AnalyticalMeanderings
11/14/2021, 12:26 AMdatajoely
11/15/2021, 9:48 AMuser
11/17/2021, 2:14 PMuser
11/17/2021, 7:29 PMthulasiram
11/23/2021, 10:36 AMdatajoely
11/23/2021, 10:43 AMpip installable
https://kedro.readthedocs.io/en/stable/03_tutorial/05_package_a_project.html
2 & 3 Here are the instructions on how to define a custom dataset in Kedro
https://kedro.readthedocs.io/en/stable/07_extend_kedro/03_custom_datasets.html
The other option is to do your stream pull/push operation outside of Kedro - dump stuff to a landing area, run your pipeline - dump processed data to a different landing area and have services which handle either sidethulasiram
11/23/2021, 10:49 AMdatajoely
11/23/2021, 10:52 AMbefore_pipeline_hook
which handles certain logic.
Kedro isn't an orchestrator so you may want to use kedro-airflow
to set up scheduling / triggersthulasiram
11/23/2021, 10:54 AMuser
11/23/2021, 12:42 PMuser
11/23/2021, 8:02 PMdotw
11/25/2021, 1:54 AMdatajoely
11/25/2021, 9:59 AMdotw
11/25/2021, 10:20 AMdatajoely
11/25/2021, 10:20 AMdotw
11/25/2021, 10:24 AMuser
11/25/2021, 8:29 PMuser
11/27/2021, 9:30 PMuser
12/04/2021, 11:20 PMuser
12/06/2021, 2:43 PMuser
12/13/2021, 6:02 PMEdmund M
12/13/2021, 9:21 PMdatajoely
12/13/2021, 9:25 PMdatajoely
12/13/2021, 9:25 PMEdmund M
12/13/2021, 9:27 PMuser
12/14/2021, 9:58 PMuser
12/15/2021, 3:09 PMSchoolmeister
12/20/2021, 2:33 PMproject_name
and package_name
parameters in pyproject.toml
, this turns out not to be the case.
When starting a notebook using kedro jupyter lab
the Kedro magic line functions are not loaded anymore.
> %reload_kedro
UsageError: Line magic function `%reload_kedro` not found.
datajoely
12/20/2021, 2:34 PMpython
In [1]: %load_ext kedro.extras.extensions.ipython
In [2]: %reload_kedro <path_to_project_root>
datajoely
12/20/2021, 2:34 PMkedro run
still work?