rafael.gildin
09/24/2022, 9:47 PMDeemac
09/26/2022, 11:54 AMdatajoely
09/26/2022, 11:55 AMdatajoely
09/26/2022, 11:55 AMbreakpoint()
statementnoklam
09/26/2022, 12:12 PMrohan_ahire
09/26/2022, 2:33 PMdatajoely
09/26/2022, 2:51 PMdatajoely
09/26/2022, 2:51 PMdatajoely
09/26/2022, 2:51 PMrohan_ahire
09/26/2022, 3:23 PMdatajoely
09/26/2022, 3:43 PMGoss
09/26/2022, 5:33 PMkedro.io.data_catalog - INFO - Loading data from 'data_science.candidate_modelling_pipeline.metrics' (MemoryDataSet)...
Since this is Kubeflow and MemoryDataset cannot be used, this is not surprising.
I observe that data_science.candidate_modelling_pipeline.metrics
is not mentioned in catalog.yml. Is the fix to simply modify catalog.yml to add in this dataset similar to the way data_science.active_modelling_pipeline.metrics
is already in there? As in, was this just an oversight? In which case, I would have separate output files for active and cadidate metrics...Goss
09/26/2022, 8:56 PMkedro catalog create --pipeline __default__
on the space tutorial, it generates a bunch of datasets not in the catalog:
data_science.active_modelling_pipeline.X_test:
type: MemoryDataSet
data_science.active_modelling_pipeline.X_train:
type: MemoryDataSet
data_science.active_modelling_pipeline.y_test:
type: MemoryDataSet
data_science.active_modelling_pipeline.y_train:
type: MemoryDataSet
data_science.candidate_modelling_pipeline.X_test:
type: MemoryDataSet
data_science.candidate_modelling_pipeline.X_train:
type: MemoryDataSet
data_science.candidate_modelling_pipeline.y_test:
type: MemoryDataSet
data_science.candidate_modelling_pipeline.y_train:
type: MemoryDataSet
Why aren't these included in conf/base/catalog.yml
when their absence causes errors like ValueError: Pipeline input(s) {'data_science.active_modelling_pipeline.y_train', 'data_science.active_modelling_pipeline.X_train'} not found in the DataCatalog
???Goss
09/26/2022, 9:39 PMrohan_ahire
09/27/2022, 8:20 PMnoklam
09/27/2022, 8:22 PMnoklam
09/27/2022, 8:24 PMBarros
09/29/2022, 10:42 PMBarros
09/29/2022, 10:44 PMMerel
09/30/2022, 1:37 PMBarros
09/30/2022, 2:19 PMGoss
09/30/2022, 5:57 PMdatajoely
09/30/2022, 6:01 PMGoss
09/30/2022, 6:05 PMBarros
09/30/2022, 6:38 PMBarros
09/30/2022, 6:38 PMBarros
09/30/2022, 6:43 PMdatajoely
09/30/2022, 6:48 PMBarros
09/30/2022, 6:55 PMBarros
09/30/2022, 7:00 PMkedro/framework/project/__init__.py
Barros
09/30/2022, 7:00 PMkedro/framework/project/__init__.py