07/31/2021, 1:27 AM
Honestly some side projects I've done with no kedro template whatsoever, starting with a blank file, I hand rolled an auto catalog in ~10 lines (probably only 3 lines of execution). I feels so fast to just ignore the catalog. Honestly I do not care where the files go, I just care that they are of the right type, and rarely (outside of the raw layer) am I fussing with any settings on them. Id rather just say dump all dataframes into parquet, in the data folder, and name the file just like the dataset, anything else make it a pickle. I can't remember ever hand loading a file, so I really don't care about the structure, I just want there to be enough there that the option is still there if I neede it.