I agree, I think they're useful for those reasons, so would like to commit them to the repository! Do you use any tools, like nbconvert or anything to deal with committing notebooks to avoid merge conflicts?

Hi, Question regarding dependencies - would anyone know if fsspec upper boundary is going to be reviewed soon? We are limited to use fsspec >=2022.5.0 so it doesn't play with Kedro 0.18.2. which is limited to <=2022.1.

Can you share why you need a relatively new version of fsspec? Would be great if you can open an GitHub issue about this. We generally don't set upperbound unless there is something breaking. I will have a look today.

There is a chain of dependencies on our side in fact. We need to use package developed by another team that requires s3fs = ">=2022.5.0", which requires mentioned version of fsspec. I read through the history of the fsspec requirement limitation in Kedro repo and seems that there was some breaking changes in the past hence the cautiousness.

Issue opened. Thank you @noklam

Thank you

The python package "nbstripout" works well for me. Install it, add three lines to your .gitattributes file, and it strips all outputs and metadata from a notebook upon "git add *", which makes for a clean git workflow. https://github.com/kynan/nbstripout

hey guys, I was wondering if you guys know any tool that could be used with Kedro that works well obviously, that provides the template for docstrings... I knew some, but they are not up to date anymore...

I'm not sure what you mean - I rely on my IDE configuration to define my doc string templates. Pycharm is best at this, Vs code is ok. For validating format there are also lining tools available

I don't use pycharm, but I mean, there was a tool that was like flake8 that you run it and it does create a docstring template for every function you have in the project...

so that you wouldn't do it manually

I'm not sure if any tools like that. Flake8 will tell you where they're missing but I don't think it will change you code for you. Googling suggests there is a pycharm function to bulk add missing docstrings

nice...I'll take a look at that

thank you

I think in Pycharm if u type """ it should just generate the docstring for you?

On vscode u have autodocstring

There is a setting to do this in bulk apparently https://stackoverflow.com/questions/37549741/is-it-possible-to-bulk-add-docstrings-to-all-the-functions-in-pycharm

Hello guys, I was doing some research about how to run kedro pipelines in containers and I found that the only thing needed besides the dependencies of the project is to have a python virtualenv, am I right or am I missing something?

No that's it! There is also a kedro-docker plugin designed to get you started

Nope, that is one of the great benefits of kerdro being written in python. I started with the kedro docker plugin. That gives you a good starting point for your docker journey, but you might need a few specific things for your environment. Such as system dependencies or database drivers. Once you have on that works for your environment you probably don't need the kedro docker plugin anymore unless you prefer the convenience of kedro cli.

hey guys I'm facing some issues when trying to write data from one BQ table to another, I did set the if_exists: 'append' but when I do it throws error

message has been deleted

Can you show us your catalog entry? To may have to put it under your save_args

I have two pipelines for: monthlypipeline = daily_data_pipeline + monthlyaggregate_pipe weeklypipeline = daily_data_pipeline +weeklyaggregate_pipe Since the pipelines share nodes, i get error. what is the best way to write this in kedro?

What kind of error are you getting? These kinds of dependencies are what Kedro is designed for, it should resolves the DAGs and merged any repeated nodes automatically.

i get error for "Pipeline nodes must have unique names". what is the best way to handle this? any sample code

Provide a variation of the

name

arg, or actually drop the explicit name altogether

sorry answering you just now, I did check the save args and now its working, I just miss place a args...

Hi, i would like to ask you something guys ... i have a question regarding the execution of the nodes. nodes that come from the same dataset are executed one after the other as if they were in a serial pipeline?, if so, is there a way to open a thread for each process to make it go faster? Here i attach some pictures to have a reference of what i am talking about