https://kedro.org/ logo
Join the conversationJoin Discord
Channels
advanced-need-help
announcements
beginners-need-help
introductions
job-posting
plugins-integrations
random
resources
welcome
Powered by Linen
introductions
  • r

    robertson809

    07/02/2021, 2:44 AM
    Hello, I'm Michael, I'm applying for a position at Quantum Black and hoping to learn more about kedro and causalnex!
  • l

    Lorena

    07/02/2021, 10:37 AM
    Hi Michael, it's great to have you here! Good look with your application! đź’Ş
  • r

    Rohail

    07/02/2021, 1:04 PM
    Hello everyone! My name's Rohail and i'm a data scientist at UCB, a pharmaceutical company (ucb.com). I've been using kedro, or carbonai as it used to be called at some point :), since 2018? I'm very happy to have seen the project become open source and how it's grown in popularity within our data science community. The thing I like most about kedro is that it created a setup for collaboration between data scientists and engineers and helped ground teams in best practices that help promote reusable codebases.. My thanks to the kedro team for a well designed product - api, documentation and a great momentum in improving the product!
  • w

    waylonwalker

    07/02/2021, 4:13 PM
    Welcome to the discord @User! You must be one of the longest users of kedro on the Discord! What is your best tip/super power that kedro gives you after all these years of using kedro?
  • w

    waylonwalker

    07/02/2021, 4:14 PM
    Welcome @User ! Good luck on your application. What have you done to start learning kedro so far? Are there any resources that you feel like you are missing?
  • r

    robertson809

    07/02/2021, 4:15 PM
    Thank you! Hmmmm, not sure what's missing so far. The documentation is pretty thorough, but I'm just branching out to start my own MNIST project now, after finishing most of the spaceflights tutorial
  • w

    waylonwalker

    07/02/2021, 4:27 PM
    Spaceflights is a solid start
  • r

    Rohail

    07/06/2021, 9:37 AM
    @User thanks for that question! I would give five tips (or at least things that worked quite well for me): 1. Prefer code over yaml API wherever possible - it really allows you to leverage all the features of pycharm and access certain features that the yaml API doesn't provide. YAML API has readability advantages but being able to dynamically generate pipelines based on configuration files is quite powerful. Also, easier to find errors when using the code api. 2. Use templatedconfiguration files whereever possible! Using jinja syntax in the config files is really helpful! Also, good to have conventions about parameter and catalog files --> better to have many small understandable configs rather than huge config files. Creating an understandable directory structure helps! 3. Master debugging kedro pipelines and reproducing errors --> A colleague may see an error on their side that you're not able to reproduce due to the: - Current state of their code (git working state); maybe a fix was pushed that they haven't retrieved - Find out the current state of their pipeline; maybe they retrieved the fix but didn't rerun the affected nodes --> partial pipeline runs are your friend 4. Use kedro starters for recurring projects! Developing these templates has been a tricky learning path since you're constantly enriching the starter as well as generating a project from that template to see that everything works. I've been using some hacky solutions that work for me but there are probably smarter workflows that I'm not aware of 5. When using notebooks in a kedro project, the first few cells should contain the git commit SHA and any versioning configurations for the catalog entries. It becomes difficult to reproduce notebook results since catalog entries keep changing as the project evolves
  • d

    datajoely

    07/06/2021, 9:38 AM
    This is great @User are you creating your own starters internally or just using the ones off the shelf?
  • r

    Rohail

    07/06/2021, 9:38 AM
    Creating our own starters!
  • d

    datajoely

    07/06/2021, 9:38 AM
    oh that's great to hear!
  • r

    Rohail

    07/06/2021, 9:39 AM
    we're also digging deep into hooks and customized great expectations to work for our use cases
  • n

    neomatrix369

    07/20/2021, 10:12 PM
    Hello all đź‘‹
  • w

    waylonwalker

    07/21/2021, 7:11 PM
    Hello @User , cool handle. Are you a seasoned kedro user, or checking it out?
  • d

    DhruvK

    07/23/2021, 5:39 PM
    Hey guys
  • d

    DhruvK

    07/23/2021, 5:41 PM
    I installed kedro == 0.17.3 on my computer, when I am running kedro info it seems to give an error, I don't have git on my system, is it prequisitive for this system
  • d

    datajoely

    07/23/2021, 5:42 PM
    @DhruvK do you mind asking this on #846330075535769601
  • u

    user

    07/24/2021, 2:43 AM
    Hi! I’m a Mechanical Eng, worked in oil and gas around 13 years, mostly root cause analysis for drilling/workover Ops and equipment, data analysis, and lean/CI, recently moved to banking to do data eng and I’m interested in knowing more about Kedro. Big fan of anything that has to do with automation!
  • d

    datajoely

    07/24/2021, 6:26 AM
    Hey @User Kedro feels like a good fit for that sort of problem! Good luck and shout if you need any help
  • w

    waylonwalker

    07/26/2021, 12:30 PM
    Hey @User I am also a mechanical engineer that made the jump to data science/engineering back in 2017.
  • u

    _pranj.al

    08/18/2021, 5:35 AM
    Hello Everyone! I am Pranjal Gupta from India, a CSE undergrad, ML/DS practitioner and researcher. Looking forward to contribute to Kedro and interact with you all
  • d

    datajoely

    08/18/2021, 9:00 AM
    Hi @User ! Welcome to the community! Do shout if you need any help or pointers 🙂
  • w

    waylonwalker

    08/19/2021, 8:16 PM
    Welcome to the communiity @User!! Are you currently using any pipeline framework similar to kedro?
  • u

    _pranj.al

    08/21/2021, 4:09 AM
    I am extremely new to this! Relied more on the theoretical workflow of DS and ML. I have used airflow tho for a project of mine!
  • n

    neomatrix369

    08/27/2021, 7:56 AM
    On and off, I have been contributing to the core library, if you see my github handle by the same name
  • d

    datajoely

    08/27/2021, 9:37 AM
    https://giphy.com/gifs/star-wars-han-solo-rHR8qP1mC5V3G
  • w

    waylonwalker

    08/28/2021, 5:13 PM
    Oh nice, thanks for contributing to the community
  • a

    avan-sh

    10/05/2021, 5:26 PM
    Hi everyone! I'm Avaneesh. I've been at QuantumBlack Singapore as a Data Engineer for the past year and had a chance to use it on multiple projects. Looking forward to contribute directly to kedro from now.
  • s

    SandyShocks™

    10/06/2021, 2:39 AM
    Hello everyone! đź‘‹
  • d

    datajoely

    10/06/2021, 6:43 AM
    Welcome! Tell us a little about yourself @SandyShocks™
Powered by Linen
Title
d

datajoely

10/06/2021, 6:43 AM
Welcome! Tell us a little about yourself @SandyShocks™
View count: 1