datajoely
11/18/2021, 2:28 PMdatajoely
11/18/2021, 2:28 PMIsaac89
11/18/2021, 2:32 PMdatajoely
11/18/2021, 2:33 PMIsaac89
11/18/2021, 3:21 PMdatajoely
11/18/2021, 4:05 PMIsaac89
11/18/2021, 5:01 PMRRoger
11/21/2021, 4:00 AMhttps://youtu.be/cKlrkIgGYEw?t=531▾
kedro new
produces run.py
.RRoger
11/21/2021, 4:25 AMbefore_node_run
hook to execute?
I added the following in src/bnhm/hooks.py
(as suggested in https://kedro.readthedocs.io/en/stable/07_extend_kedro/02_hooks.html#use-hooks-to-extend-a-node-s-behaviour):
def say_hello(node: Node):
"""An extra behaviour for a node to say hello before running."""
print(f"Hello from {node.name}")
class ProjectHooks:
@hook_impl
def register_config_loader(...):
return ConfigLoader(conf_paths)
@hook_impl
def register_catalog(...):
return ....
@hook_impl
def before_node_run(self, node: Node):
# adding extra behaviour to a single node
if node.name == "Country Plot":
say_hello(node)
The pipeline is as attached.
The output doesn't have the expected "Hellow from Country Plot":
2021-11-21 15:17:45,228 - kedro.framework.session.store - INFO - `read()` not implemented for `BaseSessionStore`. Assuming empty store.
2021-11-21 15:17:45,306 - root - INFO - ** Kedro project bnhm
2021-11-21 15:17:45,768 - kedro.io.data_catalog - INFO - Loading data from `GBIF` (GBIFRequestDataSet)...
2021-11-21 15:17:59,472 - kedro.pipeline.node - INFO - Running node: get_country_plot([GBIF]) -> [country_plot]
2021-11-21 15:17:59,534 - kedro.io.data_catalog - INFO - Saving data to `country_plot` (MatplotlibWriter)...
2021-11-21 15:17:59,608 - kedro.runner.sequential_runner - INFO - Completed 1 out of 1 tasks
2021-11-21 15:17:59,609 - kedro.runner.sequential_runner - INFO - Pipeline execution completed successfully.
2021-11-21 15:17:59,609 - kedro.framework.session.store - INFO - `save()` not implemented for `BaseSessionStore`. Skipping the step.
dmb23
11/21/2021, 8:44 AMcountry_plot
) and second this is a dataset, not a node. So either you change your hook into a dataset hook, or you check for something like if node.name == "get_country_plot"
to get the node which creates this dataset. Does something like this work?RRoger
11/21/2021, 9:06 AM@hook_impl
def before_node_run(self, node: Node):
# adding extra behaviour to a all nodes
print(f"Hello from {node.name}")
Which should print before each node right? But still nothing.dmb23
11/21/2021, 9:14 AMRRoger
11/21/2021, 9:15 AMProjectHooks
is in settings.py
by default:
HOOKS = (ProjectHooks(),)
And the before_node_run
method was added to in the ProjectHooks
class.datajoely
11/21/2021, 9:16 AMbreakpoint()
in your hook and inspect the variables at runtime to make sense of what’s going onRRoger
11/21/2021, 9:17 AMkhern
11/22/2021, 8:19 AMdatajoely
11/22/2021, 9:29 AMdatajoely
11/22/2021, 9:30 AMantony.milne
11/22/2021, 1:23 PMclass ProjectHooks:
@hook_impl
def before_node_run(self, node):
print("Hello from ", node.name)
If you're doing exactly the same as that and it's not appearing, try doing log.info
and see if that appears.
Also can confirm that the structure of the kedro project template has changed since the video you were watching was made - there's now no need to define the run
command in your projectkhern
11/22/2021, 3:58 PMdatajoely
11/22/2021, 4:03 PMRRoger
11/23/2021, 6:23 AMhooks.py
but it didn't even activate.Isaac89
11/23/2021, 12:07 PMIsaac89
11/23/2021, 12:08 PMdatajoely
11/23/2021, 12:18 PMIsaac89
11/23/2021, 12:19 PMRRoger
11/24/2021, 4:00 AMdatajoely
11/24/2021, 10:07 AMkhern
11/24/2021, 12:11 PMArnaldo
11/24/2021, 12:25 PMnode(
train_model,
["example_train_x", "example_train_y", "parameters"],
"example_model",
name="train",
),
Arnaldo
11/24/2021, 12:25 PMnode(
train_model,
["example_train_x", "example_train_y", "parameters"],
"example_model",
name="train",
),