fanzipei
12/19/2021, 1:15 PMexample_iris_data_gz:
type: pandas.CSVDataSet
filepath: data/02_intermediate/iris.csv.gz
load_args:
header: null
compression: gzip
save_args:
index: null
compression: gzip
and add a node which load the example_iris_data and export the example_iris_data_gz. Here I added the new node
def compression(df):
return df
and added it to the pipeline as:
node(
compression,
'example_iris_data',
'example_iris_data_gz',
name='compression'
)
Then run
kedro run --from-nodes='compression'
There is a warning message as:
C:\Users\fanzi\anaconda3\envs\kedro\lib\site-packages\pandas\io\common.py:609: RuntimeWarning: compression has no effect when passing a non-binary object as input.
ioargs = _get_filepath_or_buffer(
Finally get a iris.csv.gz file that is actually only a text file.