07/26/2022, 6:49 AM
I'm just trying to use certain operations like dask's client.gather on a series of returned futures, so that I can get my results back before moving into the next node -- In my previous posts I figured out how to have a dask session that dictates where the computation is done (which is held in a ProjectContext), and in trying to port some of my previous experience with dask to kedro, I have to maintain some reference to the cluster and client to do these function calls. (Under the hood I think they require an active session to be able to have a place to return the results to).