user
03/12/2022, 6:30 PMregister_flow.py
script, the datasets in the catalog object are named as in the file. However, in the Nodes the input and output datasets are namespaced. Therefore, when running that flow it will create only memory datasets because it will assume all the datasets don't exist in the catalog. Now if I change the register_flow.py
so that it does not create MemoryDatasets for everything, the run_node
function does not work as the input and catalog name don't match up and the save/load functions don't work anymore (it tries loading a namespaced dataset that it can't find in the catalog). Is there a way to obtain either a namespaced catalog or a pipeline object where the inputs/outputs of the nodes are not namespaced so that the run_node
function works properly? 🙂user
03/13/2022, 11:21 AM0.17.7
does not work with the starters anymore? In 0.17.6
the spaceflight starter still works fine but in 0.17.7
the new namespacing seems to break the starter? (pipelines and their nodes now include namespacing but catalog does not)datajoely
03/13/2022, 11:56 AMantony.milne
03/14/2022, 9:54 AMdatajoely
03/14/2022, 9:56 AMantony.milne
03/14/2022, 9:57 AMPhillyCheeseCake
03/14/2022, 11:28 AMdatajoely
03/14/2022, 12:37 PMWalber Moreira
03/16/2022, 12:58 AMdatajoely
03/16/2022, 8:43 AMdatajoely
03/16/2022, 8:48 AMAPIDataSet
) could work
- You could maybe do things in a before_pipeline_run
hook and catalog.add
your new data
- You could do this outside of Kedro and simply set up your catalog to expect data at a certain locationWalber Moreira
03/16/2022, 12:33 PMDeep
03/24/2022, 7:07 AMdatajoely
03/24/2022, 7:11 AMkalah
03/24/2022, 7:15 AMdatajoely
03/24/2022, 7:20 AMkalah
03/24/2022, 7:33 AMdatajoely
03/24/2022, 9:53 AMuser
03/27/2022, 7:29 AMPhillyCheeseCake
03/27/2022, 7:52 PMdatajoely
03/27/2022, 7:57 PMPhillyCheeseCake
03/29/2022, 10:11 AMwilliamc
03/29/2022, 7:12 PMTensorFlowModelDataset
has anyone noticed some strange behaviour when trying to save (overwrite) a model that already exists? Instead of overwriting the old version, it copies the entire temporal folder into the old model folder. I have replicated the issue in two different Kedro projects (0.17.5)datajoely
03/29/2022, 9:03 PMGalileo-Galilei
03/29/2022, 9:04 PMdatajoely
03/29/2022, 9:05 PMGalileo-Galilei
03/29/2022, 9:08 PMwilliamc
03/29/2022, 9:11 PMfsspec.put
operation copies the root tmp directory that TensorFlowModelDataset
creates into the dataset's filepathdatajoely
03/29/2022, 9:19 PMwilliamc
03/29/2022, 9:21 PMwilliamc
03/29/2022, 9:21 PM