Arnaldo
12/01/2021, 4:56 PMdatajoely
12/13/2021, 10:19 AMdatajoely
12/13/2021, 10:25 AMpandas.ExcelDataSet
can now save files with multiple sheets
š pandas.SQLQuerydataSet
can accept the query through a file (via a new filepath
argument)
šļø PartitionedDataSet
and MatplotlibWriter
can now delete existing partition/plots on save()
š„ Added support for arbitrary backends for pickle.PickleDataSet
šļø Added 4 new datasets, including:
* pandas.GenericDataSet
to read/write any format supported by pandas
* spark.DeltaTableDataSet
for handling DeltaLake tables
ā Added support for sum()
syntax for connecting Pipeline
objects
ā” Made some performance optimisations when loading up the catalog
š Fixed some bugs with kedro pipeline package
and kedro pipeline pull
š Removed unintentional dependency on git
See theĀ for more info.
š» Installation
Kedro 0.17.6 is aĀ non-breaking releaseĀ in the 0.17.x series, upgrade now with:
pip install kedro==0.17.6
conda install -c conda-forge kedro==0.17.6
datajoely
12/13/2021, 5:46 PMCVE-2021-44228
Ā log4j RCE - Kedro supply chain is not affected
- Kedro supports PySparkĀ pyspark>=2.2, <4.0
Ā as an optional dependency
- All versions of PySpark (and Hive use) useĀ log4jĀ versionĀ 1.2.17
Ā out of the box
- The issue affects versions log4jĀ Ā >2.10
Ā and is fixed inĀ 2.15.0
- Unless your cluster has manually updated your version of logger you should not be affected (ironically older version helps here)Yetunde
01/20/2022, 11:29 AMdatajoely
01/20/2022, 11:29 AMdatajoely
01/20/2022, 12:07 PMwulfcrona
01/20/2022, 12:57 PMdatajoely
01/21/2022, 12:47 PMdatajoely
02/14/2022, 10:31 AMkedro-dvc
something we've been keen to see for aaaaages, if you want to get involved there are lots of conversations happening on this issue on our side:
https://github.com/kedro-org/kedro/discussions/837#discussioncomment-2154419
and on their repo:
https://github.com/FactFiber/kedro-dvcSajidAlamQB
02/22/2022, 5:50 PMpandas<1.4
Ā to maintain compatibility withĀ xlrd~=1.0
.
- Bumped theĀ Pillow
Ā minimum version requirement to 9.0 (Python 3.7+ only) followingĀ CVE-2022-22817
.
- FixedĀ PickleDataSet
Ā to be copyable and hence work with the parallel runner.
- UpgradedĀ pip-tools
, which is used byĀ kedro build-reqs
, to 6.5 (Python 3.7+ only). ThisĀ pip-tools
Ā version is compatible withĀ pip>=21.2
, including the most recent releases ofĀ pip
. Python 3.6 users should continue to useĀ pip-tools
Ā 6.4 andĀ pip<22
.
- AddedĀ astro-iris
Ā as alias forĀ astro-airlow-iris
, so that old tutorials can still be followed.
- Added details aboutĀ Kedro's Technical Steering Committee and governance model.
Major features:
- pipeline
Ā now acceptsĀ tags
Ā and a collection ofĀ `Node`s and/orĀ `Pipeline`s rather than just a singleĀ Pipeline
Ā object.Ā pipeline
Ā should be used in preference toĀ Pipeline
Ā when creating a Kedro pipeline.
- pandas.SQLTableDataSet
Ā andĀ pandas.SQLQueryDataSet
Ā now only open one connection per database, at instantiation time (therefore at catalog creation time), rather than one per load/save operation.
- Added new command group,Ā micropkg
, to replaceĀ kedro pipeline pull
andĀ kedro pipeline package
Ā withĀ kedro micropkg pull
Ā and kedro micropkg package
for Kedro 0.18.0.kedro micropkg package
Ā saves packages toĀ project/dist
Ā whileĀ kedro pipeline package
Ā saves packages toĀ project/src/dist
.
Kedro 0.17.7Ā is aĀ non-breaking releaseĀ in the 0.17.x series, upgrade now with:
pip install kedro==0.17.7
conda install -c conda-forge kedro==0.17.7
Arnaldo
02/22/2022, 7:15 PMelvis
02/22/2022, 7:31 PMNero_Okwa
04/01/2022, 3:36 PMTL;DR
In this release we reduce the complexity of the project template to get us closer to a stable release of the framework. We've introduced the full micro-packaging workflow š¦, which allows you to import packages, utility functions and existing pipelines into your Kedro project. Integration with IPython and Jupyter has been streamlined in preparation for enhancements to Kedro's interactive workflow. Additionally, the release comes with long-awaited Python 3.9 and 3.10 support š. Please make sure to trial our experiment tracking workflow in Kedro.
ā ļø There are breaking changes in this release, you can read the full release notes: https://github.com/kedro-org/kedro/releases
What can you expect in this release?
- The micro-packaging workflow š¦,which allows you to package and export Kedro pipelines, Python packages, utility functions, and modules in your code base, to your choice of Python package repository.
- The only recommended way to work with Kedro in Jupyter or IPython is now the Kedro IPython extension.
- Reduced boilerplate and complexity of the project template. We have removed cli.py
from the Kedro project template.
- A packaged Kedro project will respect the same hierarchy when executed with python -m my_package
.
- Added Python 3.9 and 3.10 support š .Dropped support for Python 3.6.
- Refactored the load and save operations for pandas
datasets in order to leverage pandas
own API and delegate fsspec
operations to them. This reduces the need to have our own fsspec
wrappers, like in previous versions.
How do I get access to the latest release?
Get the latest version of Kedro with pip install kedro == 0.18.0
and await being able to install Kedro from Conda with conda install -c conda-forge kedro==0.18.0
.Nero_Okwa
04/01/2022, 3:38 PMtynan
04/05/2022, 5:36 PMexpandAllPipelines
feature flag that expands all modular pipelines by default. Turn this on by clicking the settings icon āļø
How do I get access to the latest release?
Get the latest version of Kedro-Viz with pip install kedro-viz==4.4.0
(Python) or with npm install @quantumblack/kedro-viz@latest
(React)
What's next?
Python 3.9 and 3.10 support will come in a following patch release. We'll also update our demo around that time too.
Until next time,
The Kedro Team š
Footnotes:
- #1: https://github.com/kedro-org/kedro-viz/releases/tag/v4.4.0
- #2: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html
Merel
04/14/2022, 10:30 AMArnaldo
04/14/2022, 12:44 PMMerel
04/20/2022, 2:09 PMhttps://www.youtube.com/watch?v=AUmDliHzWp0ā¾
Nero_Okwa
05/05/2022, 9:44 AMexperiment tracking
. Kindly respond with a š if you have used this feature, or know someone who has. Thank you.Merel
05/10/2022, 11:20 AMCONFIG_LOADER_CLASS
validation so that `TemplatedConfigLoader `can be specified in settings.py. Any CONFIG_LOADER_CLASS
must be a subclass of AbstractConfigLoader
.
- Fixed Jinja2
syntax loading with TemplatedConfigLoader
using globals.yml.
- Removed global _active_session
, _activate_session
and _deactivate_session
. Plugins that need to access objects such as the config loader should now do so through context in the new after_context_created
hook.
- Made hook_manager
argument optional for runner.run.
- kedro docs now opens an online version of the Kedro documentation instead of a locally built version.
š Major features:
- Added a new hook after_context_created
that passes the KedroContext
instance as context
.
- Added a new CLI hook after_command_run
.
- The Kedro package no longer contains a built version of the Kedro documentation significantly reducing the package size š³
You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/blob/main/RELEASE.md
Kedro 0.18.1 is a non-breaking release in the 0.18.x series, upgrade now with:
pip install kedro==0.18.1
conda install -c conda-forge kedro==0.18.1
Yetunde
05/24/2022, 2:23 PMWolVez
05/24/2022, 2:35 PMtynan
06/17/2022, 9:27 AMmatplotlib
images in the Metadata panel and Modal š
- Support for all plotly
chart types including Sankey diagrams š
- Python 3.9 and 3.10 support
- A whole host of other UX/UI improvements and bug fixes, too š
How do I get access to the latest release?
- Get the latest version of Kedro-Viz with pip install kedro-viz==4.7.0
(Python) or with npm install @quantumblack/kedro-viz@latest
(React)
What's next?
- An easier way to run Kedro-Viz in platforms like Databricks without a CLI and further honing of the application experience.
- Improvements on Experiment Tracking (#3), like being able to track plots and datasets with experiments
Until next time,
The Kedro Team š
Footnotes:
- #1: https://github.com/kedro-org/kedro-viz/releases/tag/v4.7.0
- #2: https://demo.kedro.org/
- #3: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html
Ahdra
07/12/2022, 9:23 AMconf/base/logging.yml
is now optional. See our documentation for details.
- Introduced a kedro.starters
entry point. This enables plugins to create custom starter aliases used by kedro starter list
and kedro new
.
- Reduced the kedro new
prompts to just one question asking for the project name.
š Bug fixes:
- Bumped pyyaml
upper bound to make Kedro compatible with the pyodide
stack.
- Kedro's framework-side logging configuration no longer performs file-based logging. Hence superfluous `info.log`/`errors.log `files are no longer created in your project root, and running Kedro on read-only file systems such as Databricks Repos is now possible.
- Required cookiecutter>=2.1.1
to address a known command injection vulnerability.
- The session store no longer fails if a username cannot be found with getpass.getuser
.
- Added generic typing for AbstractDataSet
and AbstractVersionedDataSet
as well as typing to all datasets.
To make the most of our new rich logging you can either delete conf/base/logging.yml
if it has not been customised, or add rich.logging.RichHandler
to the logging handlers. See the logging documentation for more.
You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/blob/main/RELEASE.md
Kedro 0.18.2 is a non-breaking release in the 0.18.x series, upgrade with:
pip install kedro==0.18.2
conda install -c conda-forge kedro==0.18.2
tynan
08/05/2022, 1:53 PMpip install kedro-viz==5.0.0
(Python) or with npm install @quantumblack/kedro-viz@latest
(React)
What's next?
- Improvements on Experiment Tracking (#3), like being able to track and plot metrics over time.
- And follow our roadmap (#4) for more updates.
Until next time,
The Kedro Team š
Footnotes:
- #1: https://github.com/kedro-org/kedro-viz/releases/tag/v5.0.0
- #2: https://demo.kedro.org/
- #3: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html
- #4: https://github.com/orgs/kedro-org/projects/3/views/1
user
08/15/2022, 12:14 AMuser
09/13/2022, 4:04 AMJose Alejandro M
09/13/2022, 8:47 PMuser
09/19/2022, 3:29 AMhttps://www.youtube.com/watch?v=y4MxuHNIIg0ā¾
user
09/19/2022, 3:29 AMhttps://www.youtube.com/watch?v=y4MxuHNIIg0ā¾