aah yes, this seems the exact use case. But all the examples only work with the DataSet name. Is the recommended approach to have the catalog loaded, read it and get the dimmensions?

The nodes shouldn't take any responsibility for loading, if you find yourself creating a catalog within a node you've gone too far! Nodes should be pure in the functional sense. The hooks let you touch the catalog in the right way.

not within the node itself, but within the hook I'd like to log the dimensions of the data!

Then that's exactly what they're for

Hi Team, What is the best way to get Run ID from node code?

Hooks!

are you saying run_params of after_pipeline_run()? can I access run_params from node code?

So you can look at the specs for all of our hooks here https://kedro.readthedocs.io/en/stable/kedro.framework.hooks.specs.html

lets' say my code likes this ##node_07_evaluate_model.py def evaluate_model(model, X_train, y_train, test_set, params, globals_): plot_writer = MatplotlibWriter( filepath="data/08_reporting/test_model_plot.png", version=Version(load=None, save=None) )

i'd like to get run_id to save

test_model_plot.png

under the same version(run_id)

The node shouldn't have knowledge of how things get saved

i see

so we need to use catalog,?

You should get something like that by just adding

versioned: True

to you catog entry

We do that already

If you really wanted to control things you could do some hook stuff to mutate the catalog live bit I wouldn't recommend it

ok thanks

@lbonini https://kedro.readthedocs.io/en/latest/06_nodes_and_pipelines/04_micro_packaging.html

You answered just when I founded 😅

Thanks

@User is there a catalog data types that handles png file or Matplotlib objects?

thank you

it's working great!

Guys, I'm in love with hooks 😄

Seriously, it is so fun to write them hahaha

Hello! I've been trying to find examples of credentials for azure blobs with no success. Anyone have an easy snippet to share? Thanks!

Just occur to me to look on the fsspec implementation, maybe I'l have some luck over there

Hello @User I'm sure we do - what dataset are you trying to use? and which Azure protocol?