https://kedro.org/ logo
Join the conversationJoin Discord
Channels
advanced-need-help
announcements
beginners-need-help
introductions
job-posting
plugins-integrations
random
resources
welcome
Powered by Linen
beginners-need-help
  • r

    rafael.gildin

    10/22/2022, 2:19 PM
    Without cli or envs
  • d

    datajoely

    10/22/2022, 3:12 PM
    As in - have two parameters with ability to override one?
  • r

    rafael.gildin

    10/22/2022, 3:43 PM
    Yes
  • d

    datajoely

    10/22/2022, 3:46 PM
    Well I'm not sure you can pick a winner if they live at the same level
  • d

    datajoely

    10/22/2022, 3:46 PM
    You could use the pure python way of having a default parameter
  • d

    datajoely

    10/22/2022, 3:47 PM
    So in your function
    def myfunc(a,b=5)
  • r

    rafael.gildin

    10/22/2022, 7:15 PM
    Right
  • r

    rafael.gildin

    10/22/2022, 7:15 PM
    What about a custom parameters loader ?
  • r

    rafael.gildin

    10/22/2022, 7:15 PM
    Like when you call in your node ‘parameters’
  • g

    Goss

    10/24/2022, 8:02 PM
    I am trying to run the Space Tutorial (starter version 0.18.3) but I keep getting
    DataSetError: Save path '.../spacetut/data/06_models/regressor.pickle/2022-10-24T19.26.35.315Z/regressor.pickle' for PickleDataSet(backend=pickle, 
    filepath=.../spacetut/data/06_models/regressor.pickle, load_args={}, protocol=file, save_args={}, version=Version(load=None, 
    save='2022-10-24T19.26.35.315Z')) must not exist if versioning is enabled.
    ...even if I clean out the file it is complaining about. And it also happened on the very first run. This is with:
    kedro                    0.18.3
    kedro-docker             0.3.0
    kedro-kubeflow           0.7.3
    kedro-telemetry          0.2.1
    kedro-viz                4.7.2
    What would cause this?
  • d

    datajoely

    10/24/2022, 8:03 PM
    Delete the folder it's in?
  • d

    datajoely

    10/24/2022, 8:03 PM
    The data that is
  • g

    Goss

    10/24/2022, 8:05 PM
    I tried deleting up to/including 06_models, but still get the error.
  • d

    datajoely

    10/24/2022, 8:05 PM
    And if you say versioning false in the catalog?
  • g

    Goss

    10/24/2022, 8:07 PM
    Then I get
    [Errno 21] Is a directory: '.../spacetut/data/06_models/regressor.pickle'
  • d

    datajoely

    10/24/2022, 8:07 PM
    So delete that directory
  • d

    datajoely

    10/24/2022, 8:07 PM
    Enable versioning and then run it
  • g

    Goss

    10/24/2022, 8:09 PM
    Back to the first error...
  • d

    datajoely

    10/24/2022, 8:11 PM
    So im out of ideas, I'd be surprised if the tutorial is fundamentally broken but I can check in the morning (it's quite late here in London!)
  • g

    Goss

    10/24/2022, 8:12 PM
    Not urgent, please let me know if you find an answer tomorrow and thanks for engaging!
  • g

    Goss

    10/24/2022, 8:14 PM
    Be aware this is with the catalog changes mentioned in https://kedro-kubeflow.readthedocs.io/en/latest/source/03_getting_started/01_quickstart.html. And for version 0.18.3 of the starter (not the 0.17.7 version mentioned in that link which I abandoned after it gave API conflicts)
  • s

    saxarb

    10/24/2022, 8:49 PM
    I'm thinking of writing a plugin for kedro for google cloud workflow and google cloud run jobs, any reason why this won't work?
  • g

    Goss

    10/25/2022, 11:46 AM
    @datajoely I believe it is because the catalog.yaml in the Quickstart specifies the same file path for two different datasets:
    data_science.active_modelling_pipeline.regressor:
      type: pickle.PickleDataSet
      filepath: data/06_models/regressor.pickle
      versioned: true
      layer: models
    ...
    data_science.candidate_modelling_pipeline.regressor:
      type: pickle.PickleDataSet
      filepath: data/06_models/regressor.pickle
      versioned: true
      layer: models
    That cannot work... right?
  • g

    Goss

    10/25/2022, 12:21 PM
    FWIW, I just changed them to
    regressor.active.pickle
    and
    regressor.candidate.pickle
    and it ran fine...
  • p

    Pedro Miyasaki

    10/25/2022, 5:05 PM
    Hey guys, can I write to parameters.yml in a kedro node?
  • t

    Thiago Poletto

    10/25/2022, 6:02 PM
    @datajoely sorry to bother, is there any way to make a node to trigger multiple other pipelines, each with their individuals nodes?
  • d

    datajoely

    10/25/2022, 6:24 PM
    Ah nice, namespacing issue will get easier in the future
  • d

    datajoely

    10/25/2022, 6:24 PM
    There are ways you could do this but I'd recommend you don't because it will break reproducibility
  • d

    datajoely

    10/25/2022, 6:25 PM
    As don't typically recommend this because the combinatorial complexity introduced makes things v hard to manage/maintain/debug
  • t

    Thiago Poletto

    10/25/2022, 6:35 PM
    I see, what would you suggest to do to make some similar effects or near as that purpose?
Powered by Linen
Title
t

Thiago Poletto

10/25/2022, 6:35 PM
I see, what would you suggest to do to make some similar effects or near as that purpose?
View count: 1