https://kedro.org/ logo
#advanced-need-help
Title
# advanced-need-help
a

antheas

07/28/2022, 6:18 PM
Hi everyone, I'm looking to run some custom pieces of code as part of my kedro pipeline, which won't run from the same python process. At the same time, I want to avoid race conditions with saving/loading data and similar issues. I'm thinking of running a piece of C code and some Jupyter notebooks (that need to run using a kernel). My data is stored locally and will be versioned if I ever run pipelines in parallel. Any suggestions? No pieces of code available on the web.