Hello! I have multiple time series with exogenous ...
# beginners-need-helph
hoodie
01/05/2022, 5:21 PMHello! I have multiple time series with exogenous params. I have one dataset for each time serie, and would like to create one model for each dataset, following the same pipeline. Is it possible to easily do it with kedro? 🤔 Thanks for your response :)
d
datajoely
01/05/2022, 5:22 PMYes! we are currently updating our docs to better explain this feature
datajoely
01/05/2022, 5:22 PMbut you can follow the docs here https://kedro.readthedocs.io/en/stable/06_nodes_and_pipelines/03_modular_pipelines.html
datajoely
01/05/2022, 5:22 PMand follow my example project here https://github.com/datajoely/modular-spaceflights
datajoely
01/05/2022, 5:23 PMYou can reuse the same pipeline multiple times with different inputs
datajoely
01/05/2022, 5:23 PMhere is an example how we use the same pipeline twice, using diff modelling techniques
w
Wit
04/24/2022, 7:46 PMCan modular pipelines be data driven? Taking the time series example - I want to build models for each time series but time series ids are in db and change in time. For example in a retail chain some products are removed or added.
d
datajoely
04/25/2022, 9:17 AMso yes, you're able to do so - but it does harm reproducibility
datajoely
04/25/2022, 9:17 AMAs a team we don't mind dynamic pipelines, but believe in order to make them maintainable the inputs can be dynamic - but the structure should be static
datajoely
04/25/2022, 9:18 AMThe other way you can do this is loading/saving some checkpoints to another dataset to retain some idea of 'state'
datajoely
04/25/2022, 9:18 AMhappy to help you think through the problem
w
Wit
04/28/2022, 7:47 AMThanks for reply. I know it harms reproducibility. I personally prefer have one pipeline and run it many times with different inputs.
3 Views