So I think you can pass generators between nodes, but you can't easily persist them. That being said I wonder if you would be better served looking at PartitionedDataSet. Or a diff execution engine like Spark or Modin.