You can define the default dataset with a custom runner

Got it... thanks! https://kedro.readthedocs.io/en/latest/nodes_and_pipelines/run_a_pipeline.html#custom-runners

Oh no, actually it worked partially! The rich logging handler stops logging when I do this change, but the rich backtrace (when something breaks) is still there. This can be tested when you call a pipeline that has wrong inputs or a pipeline that does not exist, for example. So this is only for the console logging, not for the actual backtrace when an exception occurs

This may be the case of an issue

You can see this here: the upper part (the console handler) and below the backtrace still using rich

So you can disable tracebacks this way I think https://github.com/kedro-org/kedro/issues/1712#issuecomment-1189517668

No, this still makes rich the main backtrace. Thanks to this I have found a related issue that is interesting for this discussion: https://github.com/Textualize/rich/issues/2461

So the workaround right now is to comment lines 217-219 in

kedro/framework/project/__init__.py

In the base package

@noklam any idea?

The workaround postsd there should work. With 0.18.3 we already disable it by default on databricks. https://github.com/Textualize/rich/issues/2461

kedro does some things when it hooks logging that it shouldn't that messes up my setup for example, loading a hardcoded default config before initialization and loading my own logging.yml. This causes the loggers to print useless config messages with the wrong formatting for numexpr and kedro project dir message It's the same case with the rich pretty hook and the callbacks. I have to redo all the rich initialization on my settings.py file.

from rich.traceback import install
from .progress import RICH_TRACEBACK_ARGS

logging.captureWarnings(True)
# TODO: verify this works
# remove handlers added by the default config
logging.getLogger("kedro").handlers = []
logging.root.handlers = []
install(**RICH_TRACEBACK_ARGS)

I also don't like how rich decides to use html elements for logging and formatting in jupyter, because it's clunky, larger (at least in vs code), and doesn't have a stable appearance when I commit my notebooks, so I need to run the following botch function after reload_kedro

def _reconfigure_rich():
    from rich import reconfigure, _console

    _rich_console_args = {
        "width": PBAR_JUP_NCOLS,
        "height": 100,
    }

    reconfigure(**_rich_console_args)

    # Disable html rendering when using jupyter
    # force_jupyter=False messes with pretty print
    _console_check_buffer = _console._check_buffer

    def non_html_check_buffer(self):
        tmp = self.is_jupyter
        self.is_jupyter = False
        _console_check_buffer.__get__(self)()
        self.is_jupyter = tmp

    _console._check_buffer = non_html_check_buffer.__get__(_console)

Started working with versioned datasets using the catalog.yml. I am now trying to define which version of the dataset my pipeline should load by using a config.yml file and passing that config to the run command. But whatever format I try in the config it only loads the latest version of the dataset. My current config file looks like this:

run:
  load-version: "dataset_name:2022-09-30T10.05.39.612Z"

Am I formatting my config file/parameters wrong?

@LeonG Can you try

load_version

instead? https://github.com/kedro-org/kedro/issues/1791 please votes or drop a comment on the issue if you could.

That worked, thank you! Also had to make sure the params were in a list, the following works:

load_version: ['dataset_name:2022-09-30T10.05.39.612Z']

I think equally a native YAML list would work - Maybe I am wrong

yaml
load_version:
  - dataset_name:2022-09-30T10.05.39.612Z

Hello, I am trying to set up a pipeline in a Kedro project and I get the error on the command line 'No such command: pipeline '

@sjster Probably you either haven't Kedro installed in your environment or you're running the command not in the root of your project

@Arnaldo - my setup is not the most ideal - I am using Kedro from within a poetry environment

you can try running

poetry run kedro pipeline --help

That gives me the error I listed above

That is the result of Kedro --help

Just tested it, that works too and is more readable. I'll use this format from now on, thank you!

Awesome

Hi, I have a question about the tracking.MetricsDataSet from kedro. I tried to load a metrics dataset through kedro viz but I got a file not found error. I can actually open it in a jupyter notebook (with catalog.load), so it is there. And also when I refer to the dataset as it being a tracking.JSONDataSet. Is this a bug in kedro or am I doing something wrong?

Could you share the error stack trace?

Are you running from the root directory?

Yes

Could u share the catalog entry for this dataswt?

As the error shows that your data is in certifai/data, usually data is at the top level of directory, thus the question