What makes a kedro project a kedro project from the perspective of the cli? Does it change on different versions? I am currently in a project and not seeing my project commands.

These days it's done by existence of pyproject.toml with a table called

tool.kedro

- see

kedro.framework.startup._is_project

It's been that way since 0.17.0, though 0.17.1 made the check stricter (needed the

tool.kedro

section and not just pyproject.toml file)

Before that I believe it was the existence of .kedro.yml

But don't hold me on that last part 😅

Thanks @User , That helps greately. The project I was having issues with dates back to a 14 template and has been upgraded several times. Its missing a pyproject.toml, and does not use a src directory.

hi - I'm trying to run through the "How to integrate Amazon SageMaker into your Kedro pipeline" tutorial here https://kedro.readthedocs.io/en/latest/10_deployment/09_aws_sagemaker.html and running into an issue with version compatibility between s3fs, kedro, and fsspec

kedro >= 0.16.5 gives the error "cannot import name 'setup_logging' from 'fsspec.utils'" when importing s3fs

Hi @User what version of python are you running?

Unfortunately it’s quite late in the day here in London so you may not get a detailed response until Monday. I’ll be sure to follow up then if you don’t get any other responses tonight.

3.7.10

appreciate it @User . I can raise the issue on GitHub too

This is now the right forum for support! Just a little late in the day 😅

no worries, totally understand. thank you!

Hi @User I've replied on GitHub but - thanks for the detailed issue. We think it's related to the the fact

fsspec

is using the Calendar versioning system.

thanks so much for the quick reply 👍

Not sure if here is the right place to ask but if I wanted to use the catalog to pull data from ElasticSearch would I be able to use the APIDataSet to pull the dataset or would I have to implement my own Dataset? And if I can pull it using the APIDataset how do would I go about passing in query parameters? Thank you!

I might be wrong, but I believe you can use the

data

,

params

or

json

of the APIDataSet

Hi @User - Absolutely the right place to ask. You and @User are correct here. - The

APIDataSet

is a simple wrapper for the

requests

library so that you can use the ES REST API that way. - It may prudent to implement your own dataset which wraps Elastic Search python client, but I'd personally only resort to this if the API mechanism isn't working for you. If you do end up implementing a custom dataset, please do PR it back!

Hi! I initialized a Kedro project with

kedro new --starter=pandas-iris

and then I started modifying the

catalog.yml

file. In meantime, a new Kedro version came out, which I promptly installed. Is it possible to update my project to the new version, without losing my modifications? Or should I stash the files I changed, initialize the project again with the new Kedro and then apply my modifications?

Honestly, I would start fresh just for simplicity's sake

The kedro team has done a really great job of keeping the catalog, node, and pipeline, api consistent. A simple way to upgrade is to render a new template and copy in the files that you have changed.

Will do that, thanks!

Hello! I've been using Kedro for a simple data visualization project. It basically involves visualizing a csv dataset (of a graph database containing many source nodes and target nodes ) through Kedro viz.

But I'm running into some errors. In my dataset one valid use case is different nodes can point to same targets (outputs) but that is giving me the

Node outputs must be unique

error. Can anybody help here?

Hi @sigma since Kedro is a tool for building DAG execution paths we have a hard constraints that the graph must be acyclic

Am I right in saying that you are just using the Viz library without the core side of kedro?

yes @User . That's right. I'm just using Kedro-viz by building a pipeline with a dummy function, since all I need is the graph db to be visualized

Oh, I see.