I did this for an API client on the first node of the first modular pipeline and it worked well 🙂

That's why I'm surprised it doesn't work for another node on anothe modular pipeline of the same project

The same should work even for writing to minio. I tested it locally, but I was using access keys instead.

what do you give to him inside the client ?

You won't need to create a client at all in the node. Ideally nodes should only contain any transformation logic. The dataset abstraction is available so that you can just change the destination/source filesystems without having to change your pipelines at all.

companies: type: pandas.CSVDataSet filepath: s3://test-bucket/primary/companies.csv credentials: dev_minio preprocessed_companies: type: pandas.CSVDataSet filepath: s3://test-bucket/primary/preprocessed_companies.csv credentials: dev_minio

Thx for the hint

Interesting use case, I'm also a python-binance user. I take it you're using Kedro purely for analysis rather than executing trades?

Another general question: it's clear that the design of Kedro is quite heavily inspired around ML use cases. Is it still advised to use Kedro for data science projects that have no ML involved?

I'm using it to automatically train then serve LSTM models

And I'm using Binance API to feed data for training and inference

Very cool! I've been wanting to do something similar, but it'll be summer before I have the time

Deprecation of

kedro install

Kedro without ML

Thanks for helping out the community @User ! You have been upgraded to status 🙂

with kedro.extras.datasets.matplotlib.matplotlib_writer I sometimes get the following error: Fail to create pixmap with Tk_GetPixmap in TkImgPhotoInstanceSetSize, anyone recognize this error and how to resolve it?

As far as I can work out - it's a Matplotlib error not a Kedro one as we're just passing things to that API behind the scenes

Yea definitely a matplotlib error, I just wanted to see if anyone else had seen the same thing with kedro, but clearly not. Thanks anyway.

I've created a new conda environment with

python=3.8

, and I've installed kedro and jupyter using

-c conda-forge

. However, I'm getting a "no such command 'jupyter'" error. I've set up Kedro and Jupyter many times before in conda environments, and I can't figure out what I'm doing wrong. Thanks!

There’s a chance this conversation is relevant In case useful there is a conda related discussion going on here https://github.com/kedro-org/kedro/discussions/1220

I’m not really sure thought - I don’t use conda outside of venvs

Hey All! New Kedro user here.. I'm trying to use Kedro test - is there any option to get detailed test summary and assertion errors? I can only see short test summary

What's the naming convention in catalogs when using modular pipelines? In https://github.com/datajoely/modular-spaceflights *

catalog_01_raw.yml

has "companies", "reviews" (no prefixes) *

catalog_02_int.yml

has "data_ingestion.int_typed_companies", "data_ingestion.int_typed_shuttles" (prefix with dot) *

catalog_03_prm.yml

has "prm_shuttle_company_reviews", "prm_spine_table" (prefix with underscore) Or does is it a matter of taste?

kedro-test

Data Catalog

Hi all, I was wondering if anyone knows how I can grab the kedro env flag from within a pipeline.py file, if I run

kedro run --env=test

for instance. I want to pick up the

test

string. Thanks,

So two ways - you can use a

hook

or if you use the

KEDRO_ENV

variable to set it you could just get that within your node

Cool, thanks maybe the environment variable seems like an easier fix!

There is an argument a

before_node_run

hook is the best way to do this, but I think the env variable one is neat