If a node returns a dict with multiple keys and the dataset

Question

If a node returns a dict with multiple keys and the dataset expects a dict how do I avoid the unpacking of the node to the dataset

datajoely · Answer

Hi < czix> this comes down to how you want to structure your `Pipeline` outputs

datajoely · Answer

```python my pipelines = Pipeline node func=some func that returns dict inputs= outputs= my output node func=some func that accepts a dict inputs= my output ``` In this situation the `my output` object will be a dictionary and would pertain to a single catalog entry or a single input addressable by downstream nodes ```python my pipelines = Pipeline node func=some func that returns dict inputs= outputs= key 1 catalog 1 key 2 catalog 2 node func=some other func inputs= catalog 1 outputs= ``` In this example we demonstrate how you can actually map the keys of the dictionary to individual catalog entires or downstream inputs

datajoely · Answer

I think this answers the question what do you mean by > how do I avoid the unpacking of the node to the dataset

czix · Answer

I have the first situation where the func returns a dict and the dataset my output expects one parameter But I get `list expected at most 1 argument got 2`

czix · Answer

The dict has two keys

datajoely · Answer

could you you post more of the error message and your pipeline definition

czix · Answer

I know it is not much to go on After debugging I can see actually goes into the `save` method so it is my fault But it is very hard to see on the error message

czix · Answer

```kedro io core DataSetError Failed while saving data to data set CustomDataSet filepath= my path filepath csv load args= protocol=file save args= index False version=Version load=None save= 2022 01 17T21 51 04 634Z list expected at most 1 argument got 2```

datajoely · Answer

ah gotcha

datajoely · Answer

can you post your `CustomDataSet save ` method

datajoely · Answer

because we re doing this somewhere

datajoely · Answer

perhaps this is what s needed

czix · Answer

Yes I think it was in my refactoring process But it could be nice to see it is TypeError exception happing inside the dataset

datajoely · Answer

We can look into highlighting that it s a bit difficult to account for all the things that can happen in a custom class

czix · Answer

I ll try to figure it out but thanks it is really helpful that you give that quick feedback

datajoely · Answer

The easiest thing to do is to put a `breakpoint ` in your save method

datajoely · Answer

and go line by line

czix · Answer

Yes but I think the catch all DataSetError it is bit hard to debug from

datajoely · Answer

What would be more useful in this situation

czix · Answer

Let the exception happen where it is raised inside the custom dataset

datajoely · Answer

so we do this

datajoely · Answer

we could tell you that it was a `TypeError` it tells you the error occurred during saving Would that be helpful

datajoely · Answer

There is a bunch of info in the `exc` object at runtime

czix · Answer

That would give an indication on where the problem lies but it is easiest for me in this situation just to get the trackback from where it happened

datajoely · Answer

hmm will check if that s available in the `exc` object

datajoely · Answer

thanks

czix · Answer

No problem and thank you