Hi all, I am using the IncrementalDataSet when converting pdf's to png's. I use this since I have lots of them and only want to convert the newly added pdf's. However, I am running into some problems with this.

CHECKPOINT

is using the latest partition_id from the previous run. Lets say that the CHECKPOINT value is Key456. When I add Key123 it will come in front of the CHECKPOINT since S3 is sorting it alphabetically. So my question is, why is the CHECKPOINT just the latest partition_id and not all the partition_ids? So that when there are new partition_ids it is compared with that list? And is there maybe a fix for this? Or just for me? Would like to hear from you! Regards, Nick