bluesummers
10/23/2022, 11:26 AM<my_project>/src/<my_project>
level - but that will prevent me from using micro-packaging (as the docs states).
So what is the recommended way to work with modular pipelines that share nodes/functions/classes?Onéira
10/24/2022, 7:21 AMuser
10/24/2022, 8:19 AMmrjpz99
10/26/2022, 12:42 AMdatajoely
10/26/2022, 2:57 PMdatajoely
10/26/2022, 2:57 PMnoklam
11/06/2022, 8:23 PMWolVez
11/07/2022, 9:10 PMNick Sieraad
11/08/2022, 12:40 PMCHECKPOINT
is using the latest partition_id from the previous run. Lets say that the CHECKPOINT value is Key456. When I add Key123 it will come in front of the CHECKPOINT since S3 is sorting it alphabetically. So my question is, why is the CHECKPOINT just the latest partition_id and not all the partition_ids? So that when there are new partition_ids it is compared with that list?
And is there maybe a fix for this? Or just for me?
Would like to hear from you!
Regards, Nickedhenry
11/08/2022, 5:44 PMs3://edhenry//data_folder
I receive a No partitions found
error when attempting to catalog.load()
.
I've tried a few things with manipulating the path string, etc. but nothing seems to work. Any ideas?Zoran
11/09/2022, 8:27 AMdatajoely
11/09/2022, 8:28 AMdatajoely
11/09/2022, 8:28 AMdatajoely
11/09/2022, 8:29 AMZoran
11/09/2022, 8:29 AMdatajoely
11/09/2022, 8:29 AMdatajoely
11/09/2022, 8:30 AMdatajoely
11/09/2022, 8:31 AMNick Sieraad
11/09/2022, 9:37 AMZoran
11/09/2022, 9:47 AMdatajoely
11/09/2022, 11:18 AMdatajoely
11/09/2022, 11:18 AMdatajoely
11/09/2022, 11:19 AMdatajoely
11/09/2022, 11:19 AMkedro run --config
this is all on the docsdatajoely
11/09/2022, 11:56 AMdatajoely
11/09/2022, 11:56 AMZoran
11/09/2022, 12:01 PMZoran
11/09/2022, 12:02 PMZoran
11/09/2022, 12:34 PMZoran
11/09/2022, 12:34 PM