datajoely
05/03/2022, 2:28 PMdatajoely
05/03/2022, 2:29 PMteddycarebears🇷🇴
05/03/2022, 4:31 PMdatajoely
05/03/2022, 4:52 PMAshwin_11
05/05/2022, 7:13 AMnoklam
05/05/2022, 12:14 PMcatalog.yml
(the related entry)?Ashwin_11
05/05/2022, 12:17 PMdatajoely
05/05/2022, 12:18 PMCarlos Bonilla
05/05/2022, 2:44 PMCarlos Bonilla
05/05/2022, 2:45 PMCarlos Bonilla
05/05/2022, 2:48 PMCarlos Bonilla
05/05/2022, 2:49 PMnoklam
05/05/2022, 3:04 PMCarlos Bonilla
05/05/2022, 3:16 PMAshwin_11
05/05/2022, 3:20 PMinigohrey
05/06/2022, 3:07 PMAshwin_11
05/06/2022, 3:13 PMJA_next
05/11/2022, 12:23 AMnode(
func=split_data,
inputs=["model_input_table", "params:model_options"],
outputs=["X_train", "X_test", "y_train", "y_test"],
name="split_data_node",
)
I have a question here: here 'model_input_table' is a data frame from data catalog, while how can I know this is not a string?Carlos Bonilla
05/11/2022, 3:43 AMnoklam
05/11/2022, 4:31 PMcatalog.yml
, then it will be just a variable -> MemoryDataSet
The string is the name of the DataSetJA_next
05/11/2022, 6:21 PMJA_next
05/11/2022, 10:00 PMnoklam
05/11/2022, 10:05 PMnoklam
05/11/2022, 10:13 PMJA_next
05/11/2022, 11:59 PMpipeline_steps.append(node(merging_features_to_main_data, inputs = dict(main_df = 'main_df', feature_1_df = 'feature_df1', feature_2_df = 'feature_df2'),
outputs='main_feature_df'))
This is one pipeline, merging 2 feature_df into main_df. but what if I dont know how many feature_dfs in advance, how can I do that?datajoely
05/12/2022, 5:31 AMKastakin
05/13/2022, 7:20 AMkedro docker run
command. This command mounts as volumes the required data/conf/logs folders and then runs kedro inside of it.
All good! But now let's say I would like to migrate my finalised project from my development machine to the machine in my lab where we would run the pipeline directly, what are the steps needed to use the dockerized pipeline there?
The documentation suggest pushing the built docker image to the registry and then pull it on the "production" env but that doesn't bring with it neither the catalog, the folder structure of the data folder nor the Kedro CLI itself.antony.milne
05/13/2022, 8:00 AManna-lea
05/13/2022, 12:55 PM06_models/trained_models
. What I then see is the following file structure:
06_models/trained_models/2022-05-13T11:12:13/trained_models/
which is versioned, but not super handy.
What I would expect is more something in this direction:
06_models/trained_models/2022-05-13T11:12:13/
Do you have an idea how I can get to the second file structure, or is it a "feature" :p
Thanks!Valentin DM.
05/13/2022, 5:45 PMviz
command
(I did pip install src/requirements.txt
)
Kedro version : 0.18.1
Do you have any idea?