778216384475693066 #beginners-need-help

from kedro.config import ConfigLoader, MissingConfigException

conf_paths = ["conf/base", "conf/local"]
conf_loader = ConfigLoader(conf_paths)

try:
    credentials = conf_loader.get("credentials*", "credentials*/**")
except MissingConfigException:
    credentials = {}
    
credentials

desrame

03/05/2022, 6:11 PM

it is able to find it as well and to print the creds, from both a conf/local and a conf/base

datajoely

03/05/2022, 6:13 PM

Okay so that has to be a working directory issue

desrame

03/05/2022, 6:13 PM

i think so, this is my first real foray into doing MLOps types of things

desrame

03/05/2022, 6:13 PM

so its a fun journey 🙂

datajoely

03/05/2022, 6:13 PM

Okay!

datajoely

03/05/2022, 6:13 PM

We were here once!

desrame

03/05/2022, 6:14 PM

File "C:\Users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\kedro\io\data_catalog.py", line 54, in _get_credentials
    raise KeyError(
KeyError: "Unable to find credentials 'abcde': check your data catalog and credentials configuration.

i think refers to the class and method within a lib file

desrame

03/05/2022, 6:15 PM

which i guess when i look at other people's repos and the training materials, their env is usually not contained in the repo

desrame

03/05/2022, 6:15 PM

and would instead be in "kedro-repos/...

datajoely

03/05/2022, 6:16 PM

I'm just running a couple of checks on my side 1 min

desrame

03/05/2022, 6:16 PM

appreciated! i'll keep digging!

datajoely

03/05/2022, 6:19 PM

Okay I've just run this on my end:

credentials.yml

yaml
dev_sql:
    con: sqlite:///?odbc_connect=DRIVER={ODBC+Driver+17+for+SQL+Server};SERVER=someserver;DATABASE=somedatabase;UID=someuser;PWD=somepwd

catalog.yml

yaml
test:
  type: pandas.SQLTableDataSet
  table_name: 'test'
  credentials: dev_sql

datajoely

03/05/2022, 6:19 PM

If I jump into an

kedro ipython

session

datajoely

03/05/2022, 6:20 PM

I can then see this all properly passed to the object

desrame

03/05/2022, 6:20 PM

ok sec, let me try the same

desrame

03/05/2022, 6:25 PM

catalog.datasets.test.__dict__
Traceback (most recent call last):
  File "C:\Users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\IPython\core\interactiveshell.py", line 3524, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-4-61e53845e746>", line 1, in <cell line: 1>
    catalog.datasets.test.__dict__
NameError: name 'catalog' is not defined

datajoely

03/05/2022, 6:26 PM

Okay I think this means you have a different underlying error

datajoely

03/05/2022, 6:26 PM

which is progress

datajoely

03/05/2022, 6:26 PM

and if you just type

kedro run

in the terminal - what comes up?

desrame

03/05/2022, 6:28 PM

2022-03-05 11:27:39,939 - kedro.framework.cli.hooks.manager - INFO - Registered CLI hooks from 1 installed plugin(s): kedro-telemetry-0.1.3
Kedro-Telemetry is installed, but you have opted out of sharing usage analytics so none will be collected.
2022-03-05 11:27:48,842 - kedro.framework.session.store - INFO - `read()` not implemented for `BaseSessionStore`. Assuming empty store.
fatal: Needed a single revision
2022-03-05 11:27:49,076 - kedro.framework.session.session - WARNING - Unable to git describe C:\Users\alexr\kedro-repos\beehiving_repo
2022-03-05 11:27:49,099 - kedro.framework.session.session - INFO - ** Kedro project beehiving_repo

datajoely

03/05/2022, 6:28 PM

and is there anything after that?

desrame

03/05/2022, 6:28 PM

Traceback (most recent call last):
  File "C:\Users\alexr\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\alexr\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\alexr\kedro-repos\beehiving_repo\env\beehiving\Scripts\kedro.exe\__main__.py", line 7, in <module>
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\kedro\framework\cli\cli.py", line 246, in main
    cli_collection()
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\kedro\framework\cli\cli.py", line 184, in main
    super().main(
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)

desrame

03/05/2022, 6:29 PM

File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\kedro\framework\cli\project.py", line 390, in run
    session.run(
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\kedro\framework\session\session.py", line 353, in run
    filtered_pipeline = context._filter_pipeline(
  File "c:\users\alexr\kedro-repos\beehiving_repo\env\beehiving\lib\site-packages\kedro\framework\context\context.py", line 528, in _filter_pipeline
    raise KedroContextError("Pipeline contains no nodes")
kedro.framework.context.context.KedroContextError: Pipeline contains no nodes

datajoely

03/05/2022, 6:29 PM

oh!

datajoely

03/05/2022, 6:29 PM

Pipeline contains no nodes

datajoely

03/05/2022, 6:29 PM

you need to put some boilerplate in there