Hello everyone. I’d like an opinion on a scalability issue I’m facing, mostly because I’m new to Kedro, and maybe yall can point me to the best way out of this. My problem is that I have a generic enough pipeline that can be applied to some factory equipments. Currently it attends only one but the client wants to scale this up. A project characteristic is that the catalog and parameters change substantially between the applications. And a restriction is that we want to be able to run the pipelines independently as needed, meaning that not only raw data should be changed between them, but the outputs too. I’m worried about duplicating code because it makes maintenance difficult and I also worried about losing the convenient environment usage to separate domains

local

,

production_vm

,

production_cloud

, etc. That said … how can I keep the pipeline reusable with substantial changes in catalog and parameters? Thanks for reading! 👍