dmb23
10/13/2021, 8:34 AMdata/01_raw/my_metadata.db
, but specify this location in the credentials.yml (which is not the first place I would look if future me has to change something). I think it might also be possible to put it in a globals.yml with a TemplatedConfigLoader, but I did not find a way to include it in a catalog.yml where I would search for it when I want to define SQLQueryDataSet(s)
- and in that context: the documentation states that it might be possible (at least for a SQLTableDataSet) to provide the connection string not in credentials, but in the load_args. But both classes SQLTableDataSet and SQLQueryDataSet have a check for credentials[con] that raises a DataSetError when it is not found. Is there some way around that which I am missing? (or should I offer to change the documentation accordingly?)datajoely
10/13/2021, 1:34 PMcatalog.yml
my SQLite reference is as follows:
yaml
chinook:
type: pandas.SQLTableDataSet
table_name: artists
credentials: chinook
credentials.yml
I declare the con
key as follows:
yaml
chinook:
con: sqlite:///${chinook_location}/chinook.db
global.yml
I declare the chinook_location
as a key:
yaml
chinook_location: 'data/01_raw'
TemplatedConfigLoader
in hooks.py
dmb23
10/13/2021, 1:44 PMdatajoely
10/13/2021, 1:44 PM