You can emulate this with `after_node_run` hook: h...
# beginners-need-help
l
You can emulate this with
after_node_run
hook: https://kedro.readthedocs.io/en/stable/kedro.framework.hooks.specs.NodeSpecs.html#kedro.framework.hooks.specs.NodeSpecs.after_node_run -- use MemoryDataSet as data flow in Kedro but save to disk in after_node_run hook as backup. I think you can also do this async (eg in another process) so you don't block the pipeline execution.
5 Views