Name: setuptools Version: 45.2.0 Summary: Easily download, build, install, upgrade, and uninstall Python packages Home-page: https://github.com/pypa/setuptools Author: Python Packaging Authority Author-email: distutils-sig@python.org License: UNKNOWN Location: /usr/lib/python3/dist-packages Requires: Required-by: anyconfig, astroid, Fiona, ipython, pandas-gbq, pip-tools, pydata-google-auth, tensorboard, tensorflow --- Name: pip Version: 21.3.1 Summary: The PyPA recommended tool for installing Python packages. Home-page: https://pip.pypa.io/ Author: The pip developers Author-email: distutils-sig@python.org License: MIT Location: /usr/local/lib/python3.8/dist-packages Requires: Required-by: pip-tools

Try doing

pip install -U setuptools

to upgrade setuptools. Version 45.2.0 is quite old.

And then

pytest tests/framework/cli/pipeline/test_pipeline_requirements.py --no-cov

to try again 🙂

Thanks a lot @User ! It is working now.

Awesome, pleased to hear it. BTW don't worry if the windows 3.6 tests fail - there's a problem on CI with that at the moment

These are the failing cases. All these have something to do with CLI?

Ahhh that's a lot of failures. I just checked and these are all expected though. We've got a problem at the moment running CI on forked branches - not your fault at all. All looks good with the PR 👍

Thanks again! And sorry for bugging you back and again

No worries, thank you for your contribution!

Hahah. That's such a minor change. But okay i will take that. Looking forward to contributing more in coming time. Hopefully 🤞

Hi Team, Can I include

conf/**/*.yml

files in egg file when I execute

kedro package

?

Yes as per the docs https://kedro.readthedocs.io/en/stable/03_tutorial/05_package_a_project.html we don't ship

conf/

,

data/

and

logs/

our reasons for not doing so are because we follow this philosophy as our bible https://12factor.net/config

My kedro project needs the parameters from yml files from conf folder

i'm testing https://github.com/quantumblacklabs/kedro-airflow

it complains about conf folder

the suggestions are here: > the resulting package only contains the Python source code of your Kedro pipeline, not any of the conf/, data/ and logs/ subfolders. This means that you can distribute the project to run elsewhere, such as on a separate computer with different configuration, data and logging. When distributed, the packaged project must be run from within a directory that contains the conf/ subfolder (and data/ and logs/ if your pipeline loads/saves local data or uses logging).

you will need to copy the folders to where it's installed

i see, thank you

If you want to have modular pipelines, but also have some common code between them, like custom types, what would be the best way of sharing this across the pipelines without breaking the modularity?

@User are you able to help with this one?

Hello there, Is there a way to get data from a google spreadsheet using the data catalog?

Hi @User it doesn't exist today - you would need to define one yourself which should be relatively straight forward, here is a tutorial on how to do it with pure python: https://towardsdatascience.com/how-to-import-google-sheets-data-into-a-pandas-dataframe-using-googles-api-v4-2020-f50e84ea4530 Our tutorial for defining your own can be found here: https://kedro.readthedocs.io/en/stable/07_extend_kedro/03_custom_datasets.html As always we'd appreciate if your PRd your custom dataset back into the open source project 🙂

I am having trouble with Minio credentials stored in

credentials.yml

. It doesn't work and it seems that the credentials aren't used like it should be. The strange thing is I use also credentials for another modular pipeline before the one with Minio but with Binance credentials and it works. I'm confused about the error I got because I do exactly the same process

Is there a problem between params type given by kedro credentials thing and minio expected inputs ?

Btw when I directly give Minio client the credentials in the python script it works, but I don't really want them written in the file 😄

Is it possible that credentials are loaded only for the first modular pipeline and not for all modular pipelines during the run ?

Hello there! I'm new to Kedro. I was reading the release notes for 0.18 and noticed

kedro install

is being depreciated in favour of

pip install -r src/requirements.txt

to install dependencies. Does this effectively mean that managing dependencies via conda alone is now depreciated, given that

kedro install

currently supports installing from

src/environment.yml

?

@User I'm not 100% sure[just another user], but I think the main motto here is letting the users handle their environment. So you'll likely be free to choose between various dependency management tools, conda/pip or anything else .

Hi @User , If I understand right, you're reading credentials as parameters directly on the node input and initiating a client to read/write files. I don't think credentials could be read as params, unless I'm missing some extra magic in between. An alternate approach might be to use Example: 14 from https://kedro.readthedocs.io/en/stable/05_data/01_data_catalog.html#using-the-data-catalog-with-the-yaml-api

I will try the example but it seems to be for defining a dataset in the catalog using minio as source. In my case I don't use minio as source but only as target 🙂