Thiago Poletto
08/02/2022, 9:58 AMbrewski
08/02/2022, 7:41 PMbrewski
08/02/2022, 7:51 PMbrewski
08/02/2022, 7:59 PMfrom {{project_name}}.pipelines.{{pipeline_name}}.nodes import *
and it seemed to work wellnoklam
08/02/2022, 9:29 PMbrewski
08/02/2022, 9:29 PMnoklam
08/02/2022, 9:30 PMbrewski
08/02/2022, 9:30 PMnoklam
08/02/2022, 9:30 PMnoklam
08/02/2022, 9:31 PMnoklam
08/02/2022, 9:31 PMnoklam
08/02/2022, 9:31 PMnoklam
08/02/2022, 9:32 PMbrewski
08/02/2022, 9:32 PMnoklam
08/02/2022, 9:33 PMnoklam
08/02/2022, 9:33 PMbrewski
08/02/2022, 9:38 PMbrewski
08/02/2022, 9:38 PMbrewski
08/02/2022, 9:39 PM__init__.py
noklam
08/03/2022, 9:21 AMdemo
, then you will have
demo/__init__.py
, demo/pipelines/__init__.py
, this is why you can import from there.
If you want to factor out some code into a separate module. You will create a new folder and file like demo/common/__init__.py
etc.Binky
08/04/2022, 3:48 AMraw_dataset:
type: PartitionedDataSet
dataset:
type: pandas.CSVDataSet
load_args:
index_col: False
path: data/01_raw/${asset}
filename_suffix: ".csv"
layer: raw
and my settings.py reads
CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {
"globals_pattern": "*parameters.yml",
}
When setup this way my pipeline works as expected, pulling the asset variable from parameters.yml. However now I want to be able to override this variable from the command line e.g.
kedro run --params=asset:1234
Any advice on how I can do that? I went on a wild goose hunt on Google and the Kedro github but still can't quite figure it out. It's pretty similar to this issue https://github.com/kedro-org/kedro/issues/1527#issuecomment-1127697871 but i couldn't get the suggested solution to work.datajoely
08/04/2022, 7:26 AMMyTemplatedConfigLoader
example half way down the page.
The CLI arguments will be in extra_params
and you need to ensure that these are passed to globals_dict
Binky
08/04/2022, 7:54 AMextra_params
by loading the context. I loaded it this way:
with KedroSession.create("elevator-eventmasker") as session:
context = session.load_context()
context.params
gives the variables in parameters.yml as expected. However context._extra_params
just gives None
rather than the CLI arguments.datajoely
08/04/2022, 9:29 AMnoklam
08/04/2022, 12:45 PMThiago Poletto
08/04/2022, 1:43 PMThiago Poletto
08/04/2022, 1:45 PMThiago Poletto
08/04/2022, 1:45 PMThiago Poletto
08/04/2022, 1:46 PMdatajoely
08/04/2022, 1:46 PMdatajoely
08/04/2022, 1:46 PM