datajoely
10/26/2022, 2:57 PMnoklam
11/06/2022, 8:23 PMWolVez
11/07/2022, 9:10 PMNick Sieraad
11/08/2022, 12:40 PMCHECKPOINT
is using the latest partition_id from the previous run. Lets say that the CHECKPOINT value is Key456. When I add Key123 it will come in front of the CHECKPOINT since S3 is sorting it alphabetically. So my question is, why is the CHECKPOINT just the latest partition_id and not all the partition_ids? So that when there are new partition_ids it is compared with that list?
And is there maybe a fix for this? Or just for me?
Would like to hear from you!
Regards, Nickedhenry
11/08/2022, 5:44 PMs3://edhenry//data_folder
I receive a No partitions found
error when attempting to catalog.load()
.
I've tried a few things with manipulating the path string, etc. but nothing seems to work. Any ideas?Zoran
11/09/2022, 8:27 AMdatajoely
11/09/2022, 8:28 AMdatajoely
11/09/2022, 8:28 AMdatajoely
11/09/2022, 8:29 AMZoran
11/09/2022, 8:29 AMdatajoely
11/09/2022, 8:29 AMdatajoely
11/09/2022, 8:30 AMdatajoely
11/09/2022, 8:31 AMNick Sieraad
11/09/2022, 9:37 AMZoran
11/09/2022, 9:47 AMdatajoely
11/09/2022, 11:18 AMdatajoely
11/09/2022, 11:18 AMdatajoely
11/09/2022, 11:19 AMdatajoely
11/09/2022, 11:19 AMkedro run --config
this is all on the docsdatajoely
11/09/2022, 11:56 AMdatajoely
11/09/2022, 11:56 AMZoran
11/09/2022, 12:01 PMZoran
11/09/2022, 12:02 PMZoran
11/09/2022, 12:34 PMZoran
11/09/2022, 12:35 PMZoran
11/09/2022, 12:36 PMdatajoely
11/09/2022, 1:49 PMdatajoely
11/09/2022, 1:49 PMZoran
11/09/2022, 2:06 PM