pip and conda weren't successful

did you clone Kedro itself from github or a project which uses Kedro?

I cloned kedro from github

Okay and you were unable to do

pip install kedro

?

I was not

if that's the case it may be because you're on a version greater than 3.8

You either need to follow these instructions or create a virtual env https://discord.com/channels/778216384475693066/846330075535769601/943508669008605214

Interesting. I was initially on Ubuntu Lucid which is fairly conservative, but am now on Jammy, which is a standard troubleshooting step

Sorry I was talking about python version to be clear

In both cases I was using Anaconda

Ok, IIRC the Kedro docs had advised to use Anaconda

in the docs we also say to make a new venv for each project

Do you know if McKinsey still uses Kedro?

ie QuantumBlack

yes very much so

to both

In that case, I'm going to file an issue that Kedro fail quickly if the right version of Python isn't available

3.9 is a known issue - it's being fixed in the next 3 weeks

but 3.8 is still a supported version of Python receiving security updates

The badge with the supported Python versions is a little discreet

We'll have 3.9, 3.10 support shortly

we can update it quikcly because it will introduce a breaking change and thus will be in 0.18.0

thanks for the help

Any recommendations on how to add "great expectations" into an existing kedro project? Not too familiar with using hooks so any example code would be really useful to start with.

kedro.io.core.DataSetError: Save path `/home/thakkar/Work/kedro_project/data/03_primary/Master_table.pkl/2022-02-17T20.20.56.877Z/Master_table.pkl` for PickleDataSet(backend=<module 'pickle' from '/home/thakkar/anaconda3/envs/kedro_project/lib/python3.8/pickle.py'>, filepath=/home/thakkar/Work/kedro_project/data/03_primary/Master_table.pkl, load_args={}, protocol=file, save_args={}, version=Version(load=None, save='2022-02-17T20.20.56.877Z')) must not exist if versioning is enabled.

I am currently using the example code for prefect from the kedro tutorials and there's this weird bug that I came across. Whenever I register a flow, when the folders in the

data

folder are empty, the first run works completely fine but when the same flow is run again, it gives the error. PS: I have enabled versioned=True for this dataset. Ideally every run should have it's own timestamped folder and then the pkl/csv file but that's not the case when working with Prefect. I don't know what is going on under the hood over there so I could really appreciate some help

Hi Dhuval - I think there is an old run (without versioning) at the same location. Would you remove and try again?

Hi @User there are some examples on the docs on the hooks page - but my top tip if you are only using pandas then pandera is much easier to use

I have completely emptied the folder location and reran the flow again. The error still persists that's why I thought I'll reach out for help. After removing

versioned=True

the code works as expected but then I miss out on seeing what the results for the previous runs look like

@User For your reference. I have created a run, the first run is executed when there are no folders inside the data/03_primary folder. The second time, as expected, there is a folder present but the run is not able to save the versioned dataset in the same folder because it picks the timestamp for the first run of the flow