Lazy2PickName
05/25/2022, 2:01 PMnode(
func=foo,
inputs=['input', 'params:parameter'],
outputs='output'
),
Is there something similar I can do to pass a credential from the credentials.yml
to my node?datajoely
05/25/2022, 2:03 PMparameters.yml
in your local
folder, but typically your nodes should focus on data flow not IOLazy2PickName
05/25/2022, 2:04 PMwaylonwalker
05/25/2022, 4:06 PMpython
node(lambda *frames: pd.concat(frames), ["cars", "cars"], "two_cars")
How would you concatenate pandas dataframes?antony.milne
05/25/2022, 4:17 PMhello_world
05/26/2022, 3:39 PMdatajoely
05/26/2022, 3:41 PMfrom kedro.io.data_catalog import DataSetError
before you do thisdatajoely
05/26/2022, 3:42 PMexcept DataSetError
rather than the full classpathhello_world
05/26/2022, 3:43 PMJA_next
05/26/2022, 10:49 PMdatajoely
05/27/2022, 9:30 AMpickle.PickleDataSet
https://kedro.readthedocs.io/en/stable/data/data_catalog.htmlnoklam
05/27/2022, 9:37 AMdatajoely
05/27/2022, 10:23 AMlimdauto
05/27/2022, 11:56 AMdatajoely
05/27/2022, 12:10 PMJA_next
05/27/2022, 6:20 PMRRoger
05/29/2022, 10:59 AMinputs={"food": "grilled_veg"}
supposed to be outputs={"grilled_veg": "food"}
instead? Or perhaps it's the input to the lunch_pipeline
? prep_pipeline
is the upstream pipeline.noklam
05/29/2022, 11:16 AMRRoger
05/29/2022, 11:35 AMEvolute
05/29/2022, 7:54 PMEvolute
05/29/2022, 8:02 PMEvolute
05/29/2022, 8:11 PMEvolute
05/29/2022, 8:16 PMsome_random_dude
05/29/2022, 8:26 PMnoklam
05/29/2022, 8:34 PMlayer
can helps if you need to align certain nodes in a specific layer.Evolute
05/29/2022, 8:38 PMEvolute
05/29/2022, 8:53 PMnoklam
05/29/2022, 9:05 PMEvolute
05/29/2022, 9:11 PMvivecalindahl
05/30/2022, 8:24 AM2022-05-30 10:01:09,039 - kedro.io.data_catalog - INFO - Loading data from `batch` (PartitionedDataSet) (version: 2022-05-15T05.24.31.017Z)
2022-05-30 10:01:18,467 - kedro.io.data_catalog - INFO - Saving data to `processed_batch` (ParquetDataSet)... (version: 2022-05-30T08.23.54.197Z)
Would be great 🙂