Hey guys. I am trying to make a configuration that...
# beginners-need-help
z
Hey guys. I am trying to make a configuration that I can switch between folders where my pipelines run, but can also explicitly set a pipeline to run on test data. In order to do that I was building a configuration such as this, with the templated config loader
Copy code
yml
# globals.yml
env:
  base:
    folder: "data/prod"
  test:
    folder: "data/test"

folders:
  # Base folders, where the main pipelines are run
  raw: "${env.base.folder}/01_raw"
  int: "${env.base.folder}/01_intermediate"
  # Test folders, where smaller subsets of the data designed for testing reside, and the test pipelines run
  test_raw: "${env.test.folder}/01_raw"
  test_int: "${env.test.folder}/01_intermediate"
However, when I try to run this it does not get the value from env.base. Does the templated config loader only templates for catalog.yml? Is the only official way to do this using jinja templates?
2 Views