I guess you might have found this already, but the...
# beginners-need-help
I guess you might have found this already, but the docstring for
explains a bit what's going on here:
Copy code
Ensure that all data sets are serializable and that we do not have non proxied memory data sets being used as outputs as their content not be synchronized across threads.
The second part about memory datasets is what's relevant here. As Joel said, default for parallel runner is that
is used rather than
for where this happens). In theory you could specify this dataset type explicitly in the catalog, but the fact that it's private means that's probably not a good idea, and I've never seen anyone do so. Just don't define them in the catalog and they will default to
and everything should work ok 🙂