https://kedro.org/ logo
#advanced-need-help
Title
# advanced-need-help
p

PetitLepton

10/14/2022, 6:43 PM
Hi, I bumped into an unexpected behavior and would like your feedback if I did something wrong. I built a custom dataset which inherits from SQLQueryDataSet. The latter has a parameter filepath. In the former, I, first, added a parameter templated_filepath which I then pass into SQLQueryDataSet as filepath=templated_filepath. Here is the catch. If I use the custom dataset in a notebook with something like data/01_raw/blabla as templated_filepath, the catalog breaks because the full path includes notebooks/, i.e. it's using the directory of the notebook to build the complete path. But, if instead of calling the parameter templated_filepath, I use the genuine naming filepath, then everything is fine. Is there some magic around the name filepath when the catalog is parsed to create the datasets?
It seems that
path
is also special, not only
filepath
.
OK, found it! It's in the context, function
_convert_paths_to_absolute_posix
, there is this line
Copy code
# only check a few conf keys that are known to specify a path string as value
    conf_keys_with_filepath = ("filename", "filepath", "path")