https://kedro.org/ logo
#beginners-need-help
Title
# beginners-need-help
a

Apoorva

04/21/2022, 12:43 PM
Hey All, For one of my use-case, I have to add a node that creates histograms for each pipeline run on a dataset and then second node stitch existing stitched-histogram(from previous run) with new histogram created from previous node something like
Copy code
hist_train_candidates = node(
        func=monitor_training_cand,
        inputs="training_candidates",
        outputs="training_candidates_hist",
        name="hist_train_candidates_node")

    stitch_train_candidates = node(
        func=stitch_training_cand,
        inputs=["training_candidates_hist", "stitched_hist"],
        outputs="stitched_hist",
        name="stitch_train_candidates_node")

    create_report = node(
        func=create_report_cand,
        inputs="stitch_train_candidates",
        outputs="stitch_report",
        name="create_report_node")
Having output same as your input isn't supported, but I do need it for my usecase. Plus I have to create custom versioned dataset for stitch_hist which is leading to(after applying a hack of different name catalog entry but points to same file location ) *`raise VersionNotFoundError(f"Did not find any versions for {self}") kedro.io.core.VersionNotFoundError: Did not find any versions for HistogramDataSet(filepath=/Users/Project/data/08_reporting/stitch_train_candidates.json, protocol=file, version=Version(load=None, save='2022-04-21T12.17.11.537Z'))`* Any suggestion on how to better handle this scenario?
2 Views