Title
#beginners-need-help
f

fanzipei

12/19/2021, 1:15 PM
To reproduce my problem, please use the official iris-example and add this to catalog.yml.
example_iris_data_gz:
  type: pandas.CSVDataSet
  filepath: data/02_intermediate/iris.csv.gz
  load_args:
    header: null
    compression: gzip
  save_args:
    index: null
    compression: gzip
and add a node which load the example_iris_data and export the example_iris_data_gz. Here I added the new node
def compression(df):
    return df
and added it to the pipeline as:
node(
                compression,
                'example_iris_data',
                'example_iris_data_gz',
                name='compression'
            )
Then run
kedro run --from-nodes='compression'
There is a warning message as:
C:\Users\fanzi\anaconda3\envs\kedro\lib\site-packages\pandas\io\common.py:609: RuntimeWarning: compression has no effect when passing a non-binary object as input.
  ioargs = _get_filepath_or_buffer(
Finally get a iris.csv.gz file that is actually only a text file.