It's a good question, I don't think it's wise to just map feature to feature as you'll end up limiting yourself to just the overlapping parts. The path of least resistance from a kedro point of view is to use a Azure VM / cluster and data lake storage. The pipeline, experiment etc primitives azure ml provides aren't great accelerators.

Could you expand on this? I have been playing around with azureml and writing some scripts and plugins similar to kedro kubeflow and airflow to “translate” dags into the azure version of it. However I find a lot of the azureml things good in theory but cumbersome in practice… however it is kind of integrated within itself so in theory one does not need a whole set of different open source solutions. Would be super interesting to hear thoughts

the best thing i came up with in our dev environment was to dockerize the kedro pipeline and run it as a single pythonscriptstep node in azureml, all works fine but we kind of lose the scalability

I wonder if you can use hooks to expose the run lifecycle in a way that azure can visualize

I did come across this plugin; we like the Get In Data team: https://github.com/getindata/kedro-azureml

this looks actually perfect, will give it a go

ah nice I guess I can stop doing my kedro-azureml then 😄

Please share you're experiences back to us, we'd love to hear how it goes

@Flow Maybe contribute to this project? We love seeing things like this! Let us know if you have feedback you can give to this team too.

Yeah for sure. There is no space for ego in open source. I’ll check their things out

If you'd like to see kedro-azureml plugin in action without getting hands dirty @marrrcin made a short tutorial video on the subject