https://kedro.org/ logo
#beginners-need-help
Title
# beginners-need-help
r

RRoger

05/21/2022, 6:16 AM
What's the pattern for numerous files as raw data? I want to download about 2000 files of the same type with different dates, e.g. "senate_2006-03-30.xml". 1. Do I create a catalog entry for each file? 2. Does the download-node
output
to a list of length 2000, i.e.
["senate_2006-03-30", "senate_2006-03-31", ...]
, i.e. a 2000-line
pipeline.py
? Or is there some sort of clever templating?