Yes unfortunately this isn’t the right tool for building something cyclic- it’s an interesting use case so sorry we can’t help know but it’s good to know people want to do this sort of thing

In terms of drawing force directed graphs there are lots of JavaScript libraries out there like vis.js (IIRC)

Actually mine is an undirected graph. so isn't that technically acyclic only since I'm going source to targets in one direction only. Or you mean edge wise cyclic too?

Ah, thanks. I'll check this out 👍

Yeah it’s beyond direction, it can’t loop back

Right, got it. Thanks for the help! 😅

There's an image in the README on Kedro's github repo. There two nodes point to the same output. Can you explain how to achieve that?

Let me look into the specific error. Are you okay with DMing me the JSON?

Pardon but I cannot provide that csv. corporate obligation 😓 . But I can provide you with a dummy CSV that fulfills the same purpose. Let me DM you!

That will do!

I sometimes use Kedro nodes and pipelines in my projects, but I use Kedro's Data Catalog for all of my projects! It's so useful. Trying to convince others in my team to also use the same data catalog is challenging, since the data catalog only works in Python. My team uses Bash, awk, and R also (bioinformatics). I'm thinking to just enable a mapping between dataset name and file path. To do this I would use a shell script that parses the yaml file and save this mapping into environmental variables. This way they can access the dataset-file path mapping in any language. This script would be executed in the team's

.bashrc

for team-wide dataset and be placed in the Kedro project root directory for project-specific-datasets. Any suggestions of other/easier ways to utilize the same data catalog across different languages?

nice application, @User I wondering how you're using Kedro's Data Catalog in other applications. Could you provide a simple example, please?

This is really interesting - @anhoang would you mind creating a GitHub discussion thread as I’d like to better understand what you need and how we can help improve the catalog/config journey?

Agreed - this is interesting and I'd like to hear more too 🙂 What you propose sounds very sensible. You could maybe also do it the other way round using TemplatedConfigLoader (https://kedro.readthedocs.io/en/stable/kedro.config.TemplatedConfigLoader.html), where you define the dataset name to filepath mapping in environment variables and then inject those into the kedro catalog. It depends where you want to define your original source of truth (kedro data catalog vs. environment variables) Also something that is maybe not so well known... kedro supports a variety of different file formats for config files including the catalog (yaml, json, ini, pickle, properties, xml). Although yaml is overwhelmingly popular, maybe one of the other formats is more familiar to your team/easier to parse/more natively supported in those other languages

Hello! I had a query regarding kedro. Can we somehow generate pipelines on demand while the server is running?

Like everytime I make some changes in my pipelines code, I have to restart the server everytime. Can this be avoided somehow?

Hey @User - Kedro should dynamically work with pipeline changes. How are you running your pipelines - via the CLI?

yes. Since I'm visualizing stuff I do ->

kedro viz

Ah you're talking about restarting the Viz server - currently yes, we do have a hot-reload prototype - but it's not out yet

So the answer is... we're working on it 🙂

oh okay 😅 👍

In the latest update we rewrote the Viz backend to use FastAPI and that opens up a lot possibilities for the tool to become much more interactive

Yes, I'm exploring that only right now.

Hi @User . Can you explain a bit how can I use Kedro viz as a react component in another project?

Hi @User this isn't my forte - but I can point you in the direction of a project that does this

Yes, that would be helpful. Thanks 🙂

This is a hacky thing I'm working on for another purpose and isn't official

but you can see in

App.js

on this branch how we import a JSON file and also disable the sidebar on load

This is what I was looking for! As always, thanks for the help 🙂