Title
#resources
datajoely

datajoely

06/22/2021, 7:53 AM
Good question Arnaldo! I don't think we have anything centralised in one place, we do have our recently launched 'Principles of Kedro' that talks about our philosophy when designing the framework - I think many of these points extend to any Kedro project: https://kedro.readthedocs.io/en/0.17.4/12_faq/03_kedro_principles.html From a personal perspective, I've been using what eventually became the Kedro open source project today for about 4 years and have a few personal views about what best practice could look like: - Split catalog files into multiple YAML files for maintainability - Use
TemplatedConfigLoader
- Design code to be readable 6 months later - Keep nodes simple and to the point - Avoid dynamic DAG creation in Kedro unless you really have to - If you have to, ensure that the DAG is structurally immutable and only differs in terms of dataset flow. - Most extensions to Kedro can and should happen via a Hook - Use pre-commit hooks - Write tests with fabricated data, use something like Great Expectations for defensive checks