Hi all. I have a beginner question, on Kedro 0.18.2. I have a TemplatedConfigLoader that loads a partitioned dataset from a specific path

raw_dataset: 
  type: PartitionedDataSet
  dataset:
    type: pandas.CSVDataSet
    load_args:
      index_col: False
  path: data/01_raw/${asset}
  filename_suffix: ".csv"
  layer: raw

and my settings.py reads

CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {
    "globals_pattern": "*parameters.yml",
}

When setup this way my pipeline works as expected, pulling the asset variable from parameters.yml. However now I want to be able to override this variable from the command line e.g.

kedro run --params=asset:1234

Any advice on how I can do that? I went on a wild goose hunt on Google and the Kedro github but still can't quite figure it out. It's pretty similar to this issue https://github.com/kedro-org/kedro/issues/1527#issuecomment-1127697871 but i couldn't get the suggested solution to work.

Yes, it is! Thank you so much, @noklam! It works.

Nice!

Hi @noklam, I just noticed that while the code in this repo works, context/session etc are no longer automatically loaded in Jupyter notebooks. I can live with it though, as I can just switch to the usual TemplatedConfigLoader when I need to use the notebook.

Hello @Binky, this should definitely work so if it doesn't then there might be some sort of bug. How are you opening your Jupyter notebook?

Hi @antony.milne sorry for the late reply. I open the Jupyter notebook with

kedro jupyter notebook

. I usually switch between datasets when I'm running the pipeline, so for example when I want to inspect the output of one run, I will execute the

%reload_kedro

magic in my notebook so the context and catalog is reloaded. In a standard Kedro project, this always works well for me. However, when I add the abovementioned modification to the TemplatedConfigLoader, I can no longer refresh the context with the`%reload_kedro` magic.

What happened when u run reload_kedro?

Hi @noklam. With the standard TemplatedConfigLoader, I'm always able to reload the context every time I run reload_kedro. With the modified one I get the error shown in the second screenshot. I've attached the entire traceback as well.

Can you share the implementation of

MyTemplatedConfigLoader

?

@noklam, yes, I can confirm it works now. Thanks!

Perfect!