We'll have 3.9, 3.10 support shortly

we can update it quikcly because it will introduce a breaking change and thus will be in 0.18.0

thanks for the help

Any recommendations on how to add "great expectations" into an existing kedro project? Not too familiar with using hooks so any example code would be really useful to start with.

kedro.io.core.DataSetError: Save path `/home/thakkar/Work/kedro_project/data/03_primary/Master_table.pkl/2022-02-17T20.20.56.877Z/Master_table.pkl` for PickleDataSet(backend=<module 'pickle' from '/home/thakkar/anaconda3/envs/kedro_project/lib/python3.8/pickle.py'>, filepath=/home/thakkar/Work/kedro_project/data/03_primary/Master_table.pkl, load_args={}, protocol=file, save_args={}, version=Version(load=None, save='2022-02-17T20.20.56.877Z')) must not exist if versioning is enabled.

I am currently using the example code for prefect from the kedro tutorials and there's this weird bug that I came across. Whenever I register a flow, when the folders in the

data

folder are empty, the first run works completely fine but when the same flow is run again, it gives the error. PS: I have enabled versioned=True for this dataset. Ideally every run should have it's own timestamped folder and then the pkl/csv file but that's not the case when working with Prefect. I don't know what is going on under the hood over there so I could really appreciate some help

Hi Dhuval - I think there is an old run (without versioning) at the same location. Would you remove and try again?

Hi @User there are some examples on the docs on the hooks page - but my top tip if you are only using pandas then pandera is much easier to use

I have completely emptied the folder location and reran the flow again. The error still persists that's why I thought I'll reach out for help. After removing

versioned=True

the code works as expected but then I miss out on seeing what the results for the previous runs look like

@User For your reference. I have created a run, the first run is executed when there are no folders inside the data/03_primary folder. The second time, as expected, there is a folder present but the run is not able to save the versioned dataset in the same folder because it picks the timestamp for the first run of the flow

message has been deleted

Hi Dhaval, don't think I might be right but just throwing my thoughts on why this could be happening. Is it possible that the kedro context initiated first time is being reused on the second run?

I'm guessing it's the same thing too. The flow that is being used has a fixed timestamp at all times, which doesn't make sense. There's something that goes in the backend and I don't know how to change the timestamp for the runs 😅

I'll try to replicate the issue, would following this guide[https://kedro.readthedocs.io/en/stable/10_deployment/05_prefect.html] replicate your setup?

Yes, this is exactly what I've used. Versioning datasets are causing issues while executing quick runs repetitively for a flow

prefect-versioned-datasets

Hello everyone! I wanted to know why kedro owns base and local env ? and what each one dedicated for ? why i can not create my own default env ?

Hi @User this should explain everything... https://kedro.readthedocs.io/en/latest/04_kedro_project_setup/02_configuration.html#additional-configuration-environments

not sure how I added that sticker

¯\_(ツ)_/¯

but the logic is that

base

is committed to git,

local

isn't and overrides any duplicate keys

you can then add additional `env`s in the docs above

Hi! is there a way to show the list of all the versions of a dataset in the catalog?

Hi! has anyone ever tried to run a pipeline in jupyter returning a plot? everything works till the and the figure is rendered correctly, but it raises the error

AttributeError: 'Affine2D' object has no attribute '_boxout'

what kind of plot is this? Matplotlib?

yes

I'm running it in this way :

SequentialRunner().run(pipeline=my_pipeline.from_nodes("node1").to_nodes("node10"),catalog=test)

the plot node is a memory dataset and is the last one. it is returned as a dict

{"node10": <Figure size 720x720 with 1 Axes>}

If it gets to that point I'm not sure it's a Kedro issue

it's an issue with Matplotlib render step