So the discussion on dynamic pipelines comes up a lot. Our default position is that readability is the most important thing. You can use Jinja to define dynamic catalogs and regular Python to generate dynamic pipelines. That being said - we've observed that things become very difficult to maintain things when the pipeline logic only really exists at runtime rather than rest. With this in mind, if you do dynamic pipelines I'd encourage you to dynamically generate pipelines that are immutable in structure, but dynamic in terms of inputs/outputs. If you start building conditional logic into your pipeline generation it becomes very difficult for other team members or 'future you' to read in 6 months time