Zemeio
11/06/2021, 1:25 PMdatajoely
11/08/2021, 9:39 AMZemeio
11/08/2021, 9:40 AMdatajoely
11/08/2021, 9:41 AMpython
def register_config_loader(self, conf_paths: Iterable[str]) -> ConfigLoader:
return... globals_pattern="*globals.yml",
globals_dict={
k: v for k, v in os.environ
if k.startswith("XXXXX")
},
)
Zemeio
11/08/2021, 9:42 AMkedro run --pipeline create-test-data
which is going to subset my prod data to my test data (which is sized in a way I can use in local)
While still having the ability to run
(2) kedro run --pipeline my-usual-pipeline --env test
What complicates this is that the (1) needs to run from the prod data to the test datadatajoely
11/08/2021, 9:47 AMcreate_pipeline()
functions you could start injecting some logic that changes the pipeline inputsZemeio
11/08/2021, 9:54 AMdatajoely
11/08/2021, 9:55 AMZemeio
11/08/2021, 10:42 AMdatajoely
11/08/2021, 10:42 AM