reduction
02/16/2022, 8:28 PMdatajoely
02/16/2022, 8:29 PMreduction
02/16/2022, 8:29 PMdatajoely
02/16/2022, 8:29 PMpip install kedro
?reduction
02/16/2022, 8:29 PMdatajoely
02/16/2022, 8:29 PMdatajoely
02/16/2022, 8:30 PMreduction
02/16/2022, 8:31 PMdatajoely
02/16/2022, 8:32 PMreduction
02/16/2022, 8:32 PMreduction
02/16/2022, 8:32 PMdatajoely
02/16/2022, 8:32 PMconda create -n "env_name" python=3.8 -y
datajoely
02/16/2022, 8:33 PMreduction
02/16/2022, 8:35 PMreduction
02/16/2022, 8:37 PMdatajoely
02/16/2022, 8:37 PMdatajoely
02/16/2022, 8:37 PMreduction
02/16/2022, 8:38 PMdatajoely
02/16/2022, 8:38 PMdatajoely
02/16/2022, 8:39 PMreduction
02/16/2022, 8:40 PMdatajoely
02/16/2022, 8:40 PMdatajoely
02/16/2022, 8:41 PMreduction
02/16/2022, 8:43 PMEdak
02/17/2022, 8:58 PMDhaval
02/17/2022, 9:01 PMkedro.io.core.DataSetError: Save path `/home/thakkar/Work/kedro_project/data/03_primary/Master_table.pkl/2022-02-17T20.20.56.877Z/Master_table.pkl` for PickleDataSet(backend=<module 'pickle' from '/home/thakkar/anaconda3/envs/kedro_project/lib/python3.8/pickle.py'>, filepath=/home/thakkar/Work/kedro_project/data/03_primary/Master_table.pkl, load_args={}, protocol=file, save_args={}, version=Version(load=None, save='2022-02-17T20.20.56.877Z')) must not exist if versioning is enabled.
I am currently using the example code for prefect from the kedro tutorials and there's this weird bug that I came across. Whenever I register a flow, when the folders in the data
folder are empty, the first run works completely fine but when the same flow is run again, it gives the error.
PS: I have enabled versioned=True for this dataset. Ideally every run should have it's own timestamped folder and then the pkl/csv file but that's not the case when working with Prefect. I don't know what is going on under the hood over there so I could really appreciate some helpdatajoely
02/17/2022, 9:11 PMdatajoely
02/17/2022, 9:12 PMDhaval
02/17/2022, 9:13 PMversioned=True
the code works as expected but then I miss out on seeing what the results for the previous runs look likeDhaval
02/17/2022, 9:22 PM