Hello I m using Kedro to train a model and predict values wi

Question

Hello I m using Kedro to train a model and predict values with this model afterwards The model is saved versioned after training and for prediction I usually take the lastes version But if I want to use a specific version I have to define this in the terminal after kedro run I read on the Kedro documentation Is there an option to get the defined version I want to write a log for my prediction to keep things reproducible Thanks Anna

antony.milne · Answer

Hello The best way to do this is by defining an `after catalog created` hook a bit like this one https kedro readthedocs io en stable extend kedro hooks html hook implementation `load versions` will be a dictionary of the form ` dataset name load version ` which you can then log by using `self logger info load versions `

antony.milne · Answer

You might also like to check out the experiment tracking functionality which is exactly for this sort of thing https kedro readthedocs io en stable tutorial set up experiment tracking html

antony.milne · Answer

This way you d save the prediction as a `MetricsDataSet` Each time you do a kedro run the run command you use is saved as part of the information that can be showed in kedro viz So you can ensure reproducibility that way

AnnaRie · Answer

Thanks a lot I will try to do this

AnnaRie · Answer

Hi again I tried different things but it s still not working for me I used `after catalog created` but I don t understand how to access the load version I do have a catalog yml and some information from my in catalog yml specified information is added to a logfile I also have a hooks py where I added `after catalog created` some parameters one of it is `load versions Dict str str ` and I return `DataCalatlog from config load versions ` In catalog yml I predefine a `model` with type PickleDataSet datapath versioned=True layer I start the pipeline with `kedro run pipeline pr load version= model 2022 05 15T05 24 31 017Z` How can I accedd now this load version 2022 05 15

antony.milne · Answer

You d need to do something like this ``` hooks py class DataCatalogHooks property def logger self return logging getLogger self class name hook impl def after catalog created self catalog DataCatalog load versions Dict str str > None self logger info load versions ``` And then make sure `DataCatalogHooks ` is in `HOOKS` in settings py

AnnaRie · Answer

It worked thanks a lot I didn t write load versions into the log file