https://kedro.org/ logo
Join Discord
Powered by
Hi all, I am using Kedro v0.18.2 and I have a ques...
# advanced-need-help
n
Hi all, I am using Kedro v0.18.2 and I have a question. When I want to config the catalog I got an error (Error.png). The error is in the globals.yml (globals yml.png) where I specified the dataset that is initialised in my folder 'base' (__init__ from base). When the file, where I config the catalog, is a folder level above 'base' it works, but when the file (nodes.py) is on a lower level (Folder structure) it doesn't work. I tried to specify the whole path in the globals.yml, but that doesn't work. I hope you can help me out!
n
let's move it here.
n
Good one!
n
Is the problem with your custom dataset only?
n
No also with the ImageDataSet
n
How are you running the program to get the error that you get above?
n
python src/certifai/pipelines/data/nodes.py
I am trying to run nodes.py
So, when I hard code like this "jpg: kedro.extras.datasets.pillow.ImageDataSet" it is working. But I import it from base. From globals.yml the templated catalog is not reading it. The normal datasets work, but I still have to import the custom datasets from base.
n
I am a bit confused, can you share your 
nodes.py
? It's shouldn't import any datasets
n
Maybe I am confused. Where should I load catalogs?
n
Without any context, I am just assuming you are trying to run a kedro pipeline with custom dataset implementation.
If so, 
kedro run
is still the entrypoint, and you don't need to load up 
catalog
in any of your code.
Assuming your screenshot above is under your package folder, which I call "x" Then your dataset type should be named: 
x.base.custom_datasets.THE_CLASS_NAME
the 
type
argument you put int 
catalog.yml
is just the path to import any class, same as you do 
import pandas
. For Kedro's built-in datasets, we provide a shortcut so instead of 
kedro.extras.datasets.pandas.CSVDataSet
, you can do 
pandas.CSVDataSet
.
n
Perfect! This worked! Thank you very much! It reads my custom_dataset now!
n
Awesome! Glad it helps
2 Views
PreviousNext
Powered by