rafael.gildin09/27/2022, 2:54 PM
Vici09/27/2022, 3:12 PM
Barros10/03/2022, 3:13 PM
Barros10/03/2022, 3:14 PM
Seth10/04/2022, 12:11 PM
noklam10/04/2022, 12:21 PM
Seth10/04/2022, 12:29 PM
noklam10/04/2022, 12:32 PM
Seth10/04/2022, 12:33 PM
antheas10/05/2022, 3:36 PM
user10/07/2022, 8:07 AM
user10/07/2022, 2:04 PM
user10/07/2022, 4:46 PM
williamc10/10/2022, 2:31 AM
and I've abstracted a bunch of stuff into
. If I package it up with
and distribute it, do I just need to do
on my destination machine to use it? In this case I'm not interested in running my pipelines, just the library code. Thanks!
noklam10/10/2022, 8:04 AM
nickolas da rocha machado10/10/2022, 2:47 PM
python # pipeline_registry.py pipelines["partitioned"] = Pipeline([node(print, 'partitioned', None)])
yml # catalog.yml partitioned: type: PartitionedDataSet dataset: pandas.CSVDataSet path: abfs://...dfs.core.windows/... credentials: lab filename_suffix: .csv
log [10/10/22 14:39:54] INFO Kedro project [10/10/22 14:39:55] INFO Loading data from 'partitioned' (PartitionedDataSet)...
noklam10/10/2022, 2:49 PM
nickolas da rocha machado10/10/2022, 2:50 PM
nickolas da rocha machado10/10/2022, 2:51 PM
nickolas da rocha machado10/10/2022, 2:53 PM
. it doesn't look like a Kedro problem, but an adlfs problem, since it only stops execution when reaching its glob method.
nickolas da rocha machado10/10/2022, 3:05 PM
Onéira10/11/2022, 8:55 AM
etc. will be deprecated in 0.19. Are there other commands which are planned to replace them? Or a new procedure to build the doc or requirements?
Merel10/11/2022, 10:21 AM
specifically we have a follow up task to provide a good alternative: https://github.com/kedro-org/kedro/issues/1618 Another thought we have for the commands is to move them to a community supported plugin. Please leave your thoughts on https://github.com/kedro-org/kedro/issues/1622 if this sounds useful to you 🙂
Onéira10/11/2022, 1:41 PM
farisology10/12/2022, 7:10 AM
in a local machine and it starts executing it says
does this mean it runs a local server? 2- If we are running Kedro DAG will that mean the kedro will start a local server in our airflow? what ports will it use? 3- The kedro-airflow has been archived, what does this mean for the support of the project? 4- Since there is limited documentation on running kedro in stand-alone airflow, does this mean kedro airflow will only be a sustainable approach with astronomer going forward? 5- why there is kedro operator instead of python or base operator?
running kedro session start
Merel10/12/2022, 9:36 AM
plugin for this? 3. You can definitely run Kedro on stand-alone airflow as well. The Kedro maintainer team tries to keep documentation and deployment guides up to date, but we don't always manage to write about all possible ways of deploying Kedro projects. Let us know where we can help here! 4. I don't really understand your question, is this related to airflow/`kedro-airflow` again?
lancechua10/12/2022, 11:42 AM
class factory make sense? I'm looking to write a class factory with the following signature:
or is subclassing the
python def create_generic_dataset( name: str, load: Callable[[Path, ...], pandas.DataFrame], save: Callable[[pandas.DataFrame, Path, ...], None] ): ...
still the preferred method?
datajoely10/12/2022, 11:43 AM