waylonwalker
09/13/2022, 10:05 PMGoss
09/14/2022, 5:13 PMbuild_kubeflow_pipeline.py
script, there are many references to AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the generated kfp yaml file. But I'm not using AWS at all. Why are these in there?mrjpz99
09/14/2022, 10:18 PMkedro viz
to visualize the pipeline, I got this error as the screenshot shows. I configured the "catalog.yml" with the filepath pointing to the dataset class script. Am I doing anything wrong? Or is it a bug in "kedro-viz" that it can't handle custom dataset since it's looking for the specific installed package?mrjpz99
09/14/2022, 10:25 PMdatajoely
09/14/2022, 10:25 PMmrjpz99
09/14/2022, 10:28 PMDataSetError: An exception occurred when parsing config for DataSet
'name_match_model':
Class
'name_matching_v2.extras.datasets.transformer_dataset.SentenceTransformerModel' not
found or one of its dependencies has not been installed.
datajoely
09/14/2022, 10:29 PMdatajoely
09/14/2022, 10:29 PMmrjpz99
09/14/2022, 10:33 PM{project_name}/extras
folder. Then I specify the type
of the model artifact in the catalog.yml
file as {project_name}.extras.datasets.{custom_dataset.py}.{custom_dataset_class}
. Anything else I missed?noklam
09/14/2022, 10:41 PMnoklam
09/14/2022, 10:41 PMnoklam
09/14/2022, 10:41 PMmrjpz99
09/14/2022, 10:45 PMnoklam
09/14/2022, 10:48 PMdatajoely
09/14/2022, 10:49 PMnoklam
09/14/2022, 10:49 PMmrjpz99
09/14/2022, 10:58 PMmrjpz99
09/15/2022, 8:30 AMdatajoely
09/15/2022, 8:35 AMmrjpz99
09/15/2022, 8:37 AMdatajoely
09/15/2022, 8:37 AMdatajoely
09/15/2022, 8:38 AMmrjpz99
09/15/2022, 8:49 AMsrc
folder and change the import to src.{project_name}.extras.datasets.{custom_dataset_file}
to make it work in ipython though.mrjpz99
09/15/2022, 8:52 AMcatalog.yml
with src.{project_name}.extras.datasets.{custom_dataset_file}
and execute kedro run
in the root directory, I still got the error belowmrjpz99
09/15/2022, 8:55 AMdatajoely
09/15/2022, 8:56 AMdatajoely
09/15/2022, 8:57 AMmrjpz99
09/15/2022, 9:00 AMsrc
prefix again in the catalog.yml. Not it seems working (with a different error.) Thanks for your help!! good debugging tip.:)datajoely
09/15/2022, 9:00 AMmrjpz99
09/15/2022, 9:01 AM