Title
#beginners-need-help
antony.milne

antony.milne

02/02/2022, 8:04 AM
As to pattern matching instead of just suffix matching, this is not immediately possible. I had the idea a while ago to change
filename_suffix
to a more general
filename_pattern
but I don't think it ever got implemented. If there's interest in adding this we can certainly think again about it. For now, your options would be:* change your filenames to match a suffix pattern like *_this_is_it.csv (note in can be more than just the .csv file extension) * subclass
PartitionedDataSet
to make your own custom dataset type that implements
filename_pattern
instead of
filename_suffix
. This would be straightforward but a bit annoying given
partition.endswith(self._filename_suffix)
seems to appear in several methods