antony.milne
02/02/2022, 8:04 AMfilename_suffix
to a more general filename_pattern
but I don't think it ever got implemented. If there's interest in adding this we can certainly think again about it.
For now, your options would be:
* change your filenames to match a suffix pattern like *_this_is_it.csv (note in can be more than just the .csv file extension)
* subclass PartitionedDataSet
to make your own custom dataset type that implements filename_pattern
instead of filename_suffix
. This would be straightforward but a bit annoying given partition.endswith(self._filename_suffix)
seems to appear in several methods