Burn1n9m4n
04/26/2022, 7:54 PMPartitionedDataSet
, which we setup this way because it would allow us to load the information that we need from the directory. There are checks that are performed to ensure that the file is the right one (checking filename, etc). At the end, after all the processing, the function takes each DataFrame
and adds it to a list which is then concatenated into a single DataFrame
.