So...it would go something like: (1) Setup `Increm...
# beginners-need-help
b
So...it would go something like: (1) Setup
IncrementalDataSet
(2) Run logic within function. If dataset is empty, it will just return an empty
DataFrame
. (3) That
DataFrame
gets saved as a parquet within S3. (4) That parquet gets loaded within the next node. Its (4) that I'm not sure about. That would load a an empty parquet which would require some subsequent handling, I suspect.