Hello I have multiple time series with exogenous params I ha

Question

Hello I have multiple time series with exogenous params I have one dataset for each time serie and would like to create one model for each dataset following the same pipeline Is it possible to easily do it with kedro Thanks for your response

datajoely · Answer

Yes we are currently updating our docs to better explain this feature

datajoely · Answer

but you can follow the docs here https kedro readthedocs io en stable 06 nodes and pipelines 03 modular pipelines html

datajoely · Answer

and follow my example project here https github com datajoely modular spaceflights

datajoely · Answer

You can reuse the same pipeline multiple times with different inputs

datajoely · Answer

https github com datajoely modular spaceflights blob main src modular spaceflights pipelines modelling pipeline py

datajoely · Answer

here is an example how we use the same pipeline twice using diff modelling techniques

Wit · Answer

Can modular pipelines be data driven Taking the time series example I want to build models for each time series but time series ids are in db and change in time For example in a retail chain some products are removed or added

datajoely · Answer

so yes you re able to do so but it does harm reproducibility

datajoely · Answer

As a team we don t mind dynamic pipelines but believe in order to make them maintainable the inputs can be dynamic but the structure should be static

datajoely · Answer

The other way you can do this is loading saving some checkpoints to another dataset to retain some idea of state

datajoely · Answer

happy to help you think through the problem

Wit · Answer

Thanks for reply I know it harms reproducibility I personally prefer have one pipeline and run it many times with different inputs