question: catalog.add() seems not change local con...
# beginners-need-help
j
question: catalog.add() seems not change local config.file Description: 1. jupyter notebook import the existing catalog.yaml to load all data (works) 2. use catalog.add and save the a new dataframe (works, data in catalog.list() and DF can be found on disk.) 3. check the local config.yaml file, cannot find the item
a
Continuing earlier convo here. One of the common ways I've seen users interact is to add them to conf files and access them in jupyter notebooks using
catalog
. As catalog objects in notebooks are the final form after resolving a lot of user specific definitions such as paths, credentials, jinja replacements of variables, so in my opinion it's not best to write/sync to global yaml directly from catalog. That said, I see how being able to write config to a temp file would be useful. If I may ask, how are you adding objects to catalog. I feel like adding each object to catalog manually might be tedious.
Something like this might for individual datasets and specific ones but I'm not confident it'd work for every dataset & will still need cleanup
j
wow! Totally. this explanation is quite useful.