https://kedro.org/ logo
Join the conversationJoin Discord
Channels
advanced-need-help
announcements
beginners-need-help
introductions
job-posting
plugins-integrations
random
resources
welcome
Powered by Linen
beginners-need-help
  • d

    datajoely

    02/03/2022, 5:11 PM
    I'm not sure - can you paste the error?
  • d

    datajoely

    02/03/2022, 5:11 PM
    and are you on Windows?
  • m

    mr.robot

    02/03/2022, 5:11 PM
    Wait let me check
  • m

    mr.robot

    02/03/2022, 5:13 PM
    message has been deleted
  • m

    mr.robot

    02/03/2022, 5:14 PM
    Is this what you are looking for?
  • a

    antony.milne

    02/03/2022, 5:16 PM
    This is going to be something wrong with your local environment 🤔 Please can you do
    pip show setuptools pip
    and paste it here?
  • a

    antony.milne

    02/03/2022, 5:18 PM
    This file passes tests for me locally and on CI so you should be able to get it to pass with the right environment
  • m

    mr.robot

    02/03/2022, 5:19 PM
    Okay let me see
  • m

    mr.robot

    02/03/2022, 5:20 PM
    Name: setuptools Version: 45.2.0 Summary: Easily download, build, install, upgrade, and uninstall Python packages Home-page: https://github.com/pypa/setuptools Author: Python Packaging Authority Author-email: distutils-sig@python.org License: UNKNOWN Location: /usr/lib/python3/dist-packages Requires: Required-by: anyconfig, astroid, Fiona, ipython, pandas-gbq, pip-tools, pydata-google-auth, tensorboard, tensorflow --- Name: pip Version: 21.3.1 Summary: The PyPA recommended tool for installing Python packages. Home-page: https://pip.pypa.io/ Author: The pip developers Author-email: distutils-sig@python.org License: MIT Location: /usr/local/lib/python3.8/dist-packages Requires: Required-by: pip-tools
  • a

    antony.milne

    02/03/2022, 5:22 PM
    Try doing
    pip install -U setuptools
    to upgrade setuptools. Version 45.2.0 is quite old.
  • a

    antony.milne

    02/03/2022, 5:24 PM
    And then
    pytest tests/framework/cli/pipeline/test_pipeline_requirements.py --no-cov
    to try again 🙂
  • m

    mr.robot

    02/03/2022, 5:25 PM
    Thanks a lot @User ! It is working now.
  • a

    antony.milne

    02/03/2022, 5:25 PM
    Awesome, pleased to hear it. BTW don't worry if the windows 3.6 tests fail - there's a problem on CI with that at the moment
  • m

    mr.robot

    02/03/2022, 5:33 PM
    These are the failing cases. All these have something to do with CLI?
  • a

    antony.milne

    02/03/2022, 5:36 PM
    Ahhh that's a lot of failures. I just checked and these are all expected though. We've got a problem at the moment running CI on forked branches - not your fault at all. All looks good with the PR 👍
  • m

    mr.robot

    02/03/2022, 5:38 PM
    Thanks again! And sorry for bugging you back and again
  • a

    antony.milne

    02/03/2022, 5:55 PM
    No worries, thank you for your contribution!
  • m

    mr.robot

    02/03/2022, 6:26 PM
    Hahah. That's such a minor change. But okay i will take that. Looking forward to contributing more in coming time. Hopefully 🤞
  • d

    Daehyun Kim

    02/03/2022, 6:35 PM
    Hi Team, Can I include
    conf/**/*.yml
    files in egg file when I execute
    kedro package
    ?
  • d

    datajoely

    02/03/2022, 6:37 PM
    Yes as per the docs https://kedro.readthedocs.io/en/stable/03_tutorial/05_package_a_project.html we don't ship
    conf/
    ,
    data/
    and
    logs/
    our reasons for not doing so are because we follow this philosophy as our bible https://12factor.net/config
  • d

    Daehyun Kim

    02/03/2022, 6:38 PM
    My kedro project needs the parameters from yml files from conf folder
  • d

    Daehyun Kim

    02/03/2022, 6:38 PM
    i'm testing https://github.com/quantumblacklabs/kedro-airflow
  • d

    Daehyun Kim

    02/03/2022, 6:38 PM
    it complains about conf folder
  • d

    datajoely

    02/03/2022, 6:38 PM
    the suggestions are here: > the resulting package only contains the Python source code of your Kedro pipeline, not any of the conf/, data/ and logs/ subfolders. This means that you can distribute the project to run elsewhere, such as on a separate computer with different configuration, data and logging. When distributed, the packaged project must be run from within a directory that contains the conf/ subfolder (and data/ and logs/ if your pipeline loads/saves local data or uses logging).
  • d

    datajoely

    02/03/2022, 6:39 PM
    you will need to copy the folders to where it's installed
  • d

    Daehyun Kim

    02/03/2022, 6:39 PM
    i see, thank you
  • c

    czix

    02/04/2022, 2:04 PM
    If you want to have modular pipelines, but also have some common code between them, like custom types, what would be the best way of sharing this across the pipelines without breaking the modularity?
  • d

    datajoely

    02/04/2022, 2:05 PM
    @User are you able to help with this one?
  • m

    Matheus Serpa

    02/04/2022, 4:40 PM
    Hello there, Is there a way to get data from a google spreadsheet using the data catalog?
  • d

    datajoely

    02/04/2022, 4:42 PM
    Hi @User it doesn't exist today - you would need to define one yourself which should be relatively straight forward, here is a tutorial on how to do it with pure python: https://towardsdatascience.com/how-to-import-google-sheets-data-into-a-pandas-dataframe-using-googles-api-v4-2020-f50e84ea4530 Our tutorial for defining your own can be found here: https://kedro.readthedocs.io/en/stable/07_extend_kedro/03_custom_datasets.html As always we'd appreciate if your PRd your custom dataset back into the open source project 🙂
Powered by Linen
Title
d

datajoely

02/04/2022, 4:42 PM
Hi @User it doesn't exist today - you would need to define one yourself which should be relatively straight forward, here is a tutorial on how to do it with pure python: https://towardsdatascience.com/how-to-import-google-sheets-data-into-a-pandas-dataframe-using-googles-api-v4-2020-f50e84ea4530 Our tutorial for defining your own can be found here: https://kedro.readthedocs.io/en/stable/07_extend_kedro/03_custom_datasets.html As always we'd appreciate if your PRd your custom dataset back into the open source project 🙂
View count: 1