11/08/2021, 2:07 PM
Reading about the doc on IncrementalDataset here: https://kedro.readthedocs.io/en/stable/05_data/02_kedro_io.html#incremental-dataset-confirm "Partitions that are created externally during the run will also not affect the dataset loads and won’t appear in the list of loaded partitions until the next run or until the release() method is called on the dataset object." So I was curious about this release function and when it was called. The issue is that if I override the release function, it may or may not be called depending on wether the ds is a pipeline's input/output