Question I got from a colleague: is there a way to...
# advanced-need-help
b
Question I got from a colleague: is there a way to pass file paths as arguments to override the dataset file paths in catalog.yml? Context: Debugging an airflow dag with spark tasks and want to quickly change a file path without needing to re-zip the config folder to pass to spark-submit. Might be faster/easier to pass overriding paths as arguments to main.py. My thoughts: This might be strange, but is it acceptable to define the relevant parameters in parameters.yml and set the templated config loader "globals_pattern" to *parameters.yml in order to pass the paths as "extra parameters" to the context initializer?