Quotes for Site

Come see our tech lead @User speak at mlops.community on Wednesday this week! https://www.linkedin.com/posts/mlopscommunity_kedro-activity-6874648861915262976-jw62/

🚀 🎊  KEDRO RELEASE 🎊 🚀 Kedro 0.17.6 is available on PyPI and Conda. This version of Kedro contains heaps of bugfixes and dataset improvements, including: ✍️

pandas.ExcelDataSet

can now save files with multiple sheets 📜

pandas.SQLQuerydataSet

can accept the query through a file (via a new

filepath

argument) 🗑️

PartitionedDataSet

and

MatplotlibWriter

can now delete existing partition/plots on

save()

🥒 Added support for arbitrary backends for

pickle.PickleDataSet

🗄️ Added 4 new datasets, including: *

pandas.GenericDataSet

to read/write any format supported by

pandas

*

spark.DeltaTableDataSet

for handling DeltaLake tables ➕ Added support for

sum()

syntax for connecting

Pipeline

objects ⚡ Made some performance optimisations when loading up the catalog 🐛 Fixed some bugs with

kedro pipeline package

and

kedro pipeline pull

🐞 Removed unintentional dependency on

git

See the for more info. 💻 Installation Kedro 0.17.6 is a non-breaking release in the 0.17.x series, upgrade now with:

pip install kedro==0.17.6
conda install -c conda-forge kedro==0.17.6

Regarding 

CVE-2021-44228

 log4j RCE - Kedro supply chain is not affected - Kedro supports PySpark 

pyspark>=2.2, <4.0

 as an optional dependency - All versions of PySpark (and Hive use) use log4j version 

1.2.17

 out of the box - The issue affects versions log4j  

>2.10

 and is fixed in 

2.15.0

- Unless your cluster has manually updated your version of logger you should not be affected (ironically older version helps here)

@here We have worked on this for so long, but it's finally happened! Kedro has been donated to the Linux Foundation! We specifically joined a sub-organisation within the Linux Foundation called LF AI & Data, which is also home to Feast and Amundsen. This change signifies that we're pursuing industry-wide adoption. Some have raised challenges with clients rejecting Kedro because there was a singular dependency on QuantumBlack and McKinsey, which is no longer valid. You will also see an open governance model being phased in. Read more about the changes here: https://www.mckinsey.com/about-us/new-at-mckinsey-blog/mckinsey-donates-open-source-tool-kedro-to-the-linux-foundation

So proud of this team! Excited for the future

We're also up to 4,860 ⭐ on github, fancy getting us to 5K?

An open source user has started work on

kedro-dvc

something we've been keen to see for aaaaages, if you want to get involved there are lots of conversations happening on this issue on our side: https://github.com/kedro-org/kedro/discussions/837#discussioncomment-2154419 and on their repo: https://github.com/FactFiber/kedro-dvc

Hi @everyone ! This is a quick announcement that Kedro 0.17.7 is out.  It's available on PyPI and Conda. This version has several bug fixes and major improvements including: Bug fixes: - Added tutorial documentation for experiment tracking. - Added Plotly dataset documentation. - Added the upper limit 

pandas<1.4

 to maintain compatibility with 

xlrd~=1.0

. - Bumped the 

Pillow

 minimum version requirement to 9.0 (Python 3.7+ only) following 

CVE-2022-22817

. - Fixed 

PickleDataSet

 to be copyable and hence work with the parallel runner. - Upgraded 

pip-tools

, which is used by 

kedro build-reqs

, to 6.5 (Python 3.7+ only). This 

pip-tools

 version is compatible with 

pip>=21.2

, including the most recent releases of 

pip

. Python 3.6 users should continue to use 

pip-tools

 6.4 and 

pip<22

. - Added 

astro-iris

 as alias for 

astro-airlow-iris

, so that old tutorials can still be followed. - Added details about Kedro's Technical Steering Committee and governance model. Major features: -

pipeline

 now accepts 

tags

 and a collection of `Node`s and/or `Pipeline`s rather than just a single 

Pipeline

 object. 

pipeline

 should be used in preference to 

Pipeline

 when creating a Kedro pipeline. -

pandas.SQLTableDataSet

 and 

pandas.SQLQueryDataSet

 now only open one connection per database, at instantiation time (therefore at catalog creation time), rather than one per load/save operation. - Added new command group, 

micropkg

, to replace 

kedro pipeline pull

and 

kedro pipeline package

 with 

kedro micropkg pull

 and

kedro micropkg package

for Kedro 0.18.0.

kedro micropkg package

 saves packages to 

project/dist

 while 

kedro pipeline package

 saves packages to 

project/src/dist

. Kedro 0.17.7 is a non-breaking release in the 0.17.x series, upgrade now with:

pip install kedro==0.17.7
conda install -c conda-forge kedro==0.17.7

@User @User

kedro 0.17.7

@here We're excited to announce that Kedro 0.18.0 has finally been released! 🎉

TL;DR

In this release we reduce the complexity of the project template to get us closer to a stable release of the framework. We've introduced the full micro-packaging workflow 📦, which allows you to import packages, utility functions and existing pipelines into your Kedro project. Integration with IPython and Jupyter has been streamlined in preparation for enhancements to Kedro's interactive workflow. Additionally, the release comes with long-awaited Python 3.9 and 3.10 support 🐍. Please make sure to trial our experiment tracking workflow in Kedro. ⚠️ There are breaking changes in this release, you can read the full release notes: https://github.com/kedro-org/kedro/releases What can you expect in this release? - The micro-packaging workflow 📦,which allows you to package and export Kedro pipelines, Python packages, utility functions, and modules in your code base, to your choice of Python package repository. - The only recommended way to work with Kedro in Jupyter or IPython is now the Kedro IPython extension. - Reduced boilerplate and complexity of the project template. We have removed

cli.py

from the Kedro project template. - A packaged Kedro project will respect the same hierarchy when executed with

python -m my_package

. - Added Python 3.9 and 3.10 support 🐍 .Dropped support for Python 3.6. - Refactored the load and save operations for

pandas

datasets in order to leverage

pandas

own API and delegate

fsspec

operations to them. This reduces the need to have our own

fsspec

wrappers, like in previous versions. How do I get access to the latest release? Get the latest version of Kedro with

pip install kedro == 0.18.0

and await being able to install Kedro from Conda with

conda install -c conda-forge kedro==0.18.0

.

Links to new features: - Experiment Tracking: https://kedro.readthedocs.io/en/0.18.0/tutorial/set_up_experiment_tracking.html - The new iPython/Jupyter Workflow: https://kedro.readthedocs.io/en/0.18.0/tools_integration/ipython.html - Micro-packaging Workflow: https://kedro.readthedocs.io/en/stable/nodes_and_pipelines/micro_packaging.html

Kedro-Viz 4.4.0 is out! @here We're excited to announce that Kedro-Viz 4.4.0 is out! 🥳 This version brings some great improvements and important fixes. Have a look at the full release notes (footnote #1) for everything new. What can you expect in this release? - A way to export your experiment tracking data (footnote #2) as a CSV - A new pop-up to remind you to upgrade Kedro-Viz to get the latest features - An

expandAllPipelines

feature flag that expands all modular pipelines by default. Turn this on by clicking the settings icon ⚙️ How do I get access to the latest release? Get the latest version of Kedro-Viz with

pip install kedro-viz==4.4.0

(Python) or with

npm install @quantumblack/kedro-viz@latest

(React) What's next? Python 3.9 and 3.10 support will come in a following patch release. We'll also update our demo around that time too. Until next time, The Kedro Team 💜 Footnotes: - #1: https://github.com/kedro-org/kedro-viz/releases/tag/v4.4.0 - #2: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html

I'll be giving a crash course on Kedro next week! Come along if you're new to Kedro and want to learn more about it and code along with the beginner tutorial 🚀 https://discord.gg/sCZsWWsK?event=964120084413038633

Kedro Crash Course

The recording of my talk is now available on youtube for those who missed it yesterday 🙂

@here We're currently conducting user research on kedro

experiment tracking

. Kindly respond with a 👍 if you have used this feature, or know someone who has. Thank you.

Hi @here! This is a quick announcement that Kedro 0.18.1 is out. It’s available on PyPI and Conda. This version has several bug fixes and major improvements including: 🐛 Bug fixes: - Fixed

CONFIG_LOADER_CLASS

validation so that `TemplatedConfigLoader `can be specified in settings.py. Any

CONFIG_LOADER_CLASS

must be a subclass of

AbstractConfigLoader

. - Fixed

Jinja2

syntax loading with

TemplatedConfigLoader

using globals.yml. - Removed global

_active_session

,

_activate_session

and

_deactivate_session

. Plugins that need to access objects such as the config loader should now do so through context in the new

after_context_created

hook. - Made

hook_manager

argument optional for runner.run. - kedro docs now opens an online version of the Kedro documentation instead of a locally built version. 🚀 Major features: - Added a new hook

after_context_created

that passes the

KedroContext

instance as

context

. - Added a new CLI hook

after_command_run

. - The Kedro package no longer contains a built version of the Kedro documentation significantly reducing the package size 🐳 You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/blob/main/RELEASE.md Kedro 0.18.1 is a non-breaking release in the 0.18.x series, upgrade now with:

pip install kedro==0.18.1
conda install -c conda-forge kedro==0.18.1

*@here We have two exciting announcements! 🎉 🎉🎉 * *Kedro is officially online 🌍 * We have a Kedro website: https://kedro.org/. This is our first version; please give us feedback. We're working with Databricks We're setting up a hackathon with a team from Databricks to address issues related to the Databricks / Kedro development and deployment workflows. We would love to understand the scope of issues that you have encountered. Comment in a thread below with your struggles or use the 🗓️ emoji so that we can book time in your calendar to dive deeper into your workflow.

Kedro

Kedro-Viz 4.7.0 is out! @here We're excited to announce that Kedro-Viz 4.7.0 is out! 🥳 This version brings some great improvements and important fixes. Have a look at the full release notes (#1) for everything new. Check out our hosted demo (#2). What can you expect in this release? - Support for

matplotlib

images in the Metadata panel and Modal 📊 - Support for all

plotly

chart types including Sankey diagrams 📈 - Python 3.9 and 3.10 support - A whole host of other UX/UI improvements and bug fixes, too 🐛 How do I get access to the latest release? - Get the latest version of Kedro-Viz with

pip install kedro-viz==4.7.0

(Python) or with

npm install @quantumblack/kedro-viz@latest

(React) What's next? - An easier way to run Kedro-Viz in platforms like Databricks without a CLI and further honing of the application experience. - Improvements on Experiment Tracking (#3), like being able to track plots and datasets with experiments Until next time, The Kedro Team 💜 Footnotes: - #1: https://github.com/kedro-org/kedro-viz/releases/tag/v4.7.0 - #2: https://demo.kedro.org/ - #3: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html

Hi @here! This is a quick announcement that Kedro 0.18.2 is out and is available on PyPI and Conda > TL;DR This release is special because we've worked on improving Databricks Repos support and improved the CLI experienced by adding in pretty logging to the CLI with Rich and reducing the kedro new prompts to one question. This version also has several bug fixes and improvements: 🚀 Major features: - Kedro now uses the Rich library to format terminal logs and tracebacks. - The file

conf/base/logging.yml

is now optional. See our documentation for details. - Introduced a

kedro.starters

entry point. This enables plugins to create custom starter aliases used by

kedro starter list

and

kedro new

. - Reduced the

kedro new

prompts to just one question asking for the project name. 🐛 Bug fixes: - Bumped

pyyaml

upper bound to make Kedro compatible with the

pyodide

stack. - Kedro's framework-side logging configuration no longer performs file-based logging. Hence superfluous `info.log`/`errors.log `files are no longer created in your project root, and running Kedro on read-only file systems such as Databricks Repos is now possible. - Required

cookiecutter>=2.1.1

to address a known command injection vulnerability. - The session store no longer fails if a username cannot be found with

getpass.getuser

. - Added generic typing for

AbstractDataSet

and

AbstractVersionedDataSet

as well as typing to all datasets. To make the most of our new rich logging you can either delete

conf/base/logging.yml

if it has not been customised, or add

rich.logging.RichHandler

to the logging handlers. See the logging documentation for more. You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/blob/main/RELEASE.md Kedro 0.18.2 is a non-breaking release in the 0.18.x series, upgrade with:

pip install kedro==0.18.2
conda install -c conda-forge kedro==0.18.2

Kedro-Viz 5.0.0 is out! @here We're excited to announce that Kedro-Viz 5.0.0 is out! 🥳 This new major-release version brings a great new feature and the removal of support for an older version of Kedro. Have a look at the full release notes (#1) for everything new. Check out our hosted demo (#2). What can you expect in this release? - Support for visualising Matplotlib and Plotly plots in experiment tracking 📊 - Remove support for Kedro v16 🧹 ⚠️ Note this last point is a breaking change! If you're using Kedro v17 or v18, please upgrade to continue getting the best out of Kedro-Viz. ⚠️ How do I get access to the latest release? - Get the latest version of Kedro-Viz with

pip install kedro-viz==5.0.0

(Python) or with

npm install @quantumblack/kedro-viz@latest

(React) What's next? - Improvements on Experiment Tracking (#3), like being able to track and plot metrics over time. - And follow our roadmap (#4) for more updates. Until next time, The Kedro Team 💜 Footnotes: - #1: https://github.com/kedro-org/kedro-viz/releases/tag/v5.0.0 - #2: https://demo.kedro.org/ - #3: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html - #4: https://github.com/orgs/kedro-org/projects/3/views/1

Hey Admins and Moderators, We've got an update on all things Community today! If you haven't joined our Discord Admin Community… what are you waiting for? Head over to Server Setting > Community overview and join us — we've got a slew of exciting and fun stuff planned that you won’t want to miss out on. 🪴 __*Below is a sneak peek at the Stage events we’ve got cookin’!*__ August 16th: Safety on Discord August 25th: Utilizing Automod in your Community To join our Discord Admin Community, your server will need to hit a few engagement requirements. Once your community reaches the requirements listed , any of your Admins or Moderators with the “Manage Server” permission will be able to join the Discord Admin Community via Server Settings > Community. 📚 __**Community Resources**__ We recently released our Community Resources, a one-stop shop for learning how to build, engage, grow, and manage a space where anyone can belong. New articles, showcases, and more will be added in the future, so be sure to bookmark it for later. If you haven't already taken a peek at some of our content, you can check out the latest video! https://dis.gd/H2M_Your_Discord_Community 🤖 __Apps & Bots__ If your server uses a verified app, the way you send commands has probably changed in the last year - if it hasn’t, it may very soon! Developers of verified apps have been migrating to Slash Commands. The deadline to finish migrating is the end of August, which means on Sept 1, any apps that haven’t finished may have reduced functionality. If you haven’t tried out slash commands yet, type / in your server to see all the commands you have access to. If your favorite app isn’t listed, they may not yet have enabled slash commands. Or, you might need to check out the new permissions system in Server Settings > Integrations. If you run across any issues, you can reach out to individual app developers to find out more. Learn more here:

Hey Admins and Mods! In our mission to help you keep your communities safe and lift some of that moderation work off your shoulders, we’ve set our sights on helping you more easily detect and prevent spam and raids. Today's spotlight introduces new features that help keep your communities cleaner from spam, and a preview into our early plans on building tools to deal with raids. Keep your Community clean from spam with new AutoMod spam filters We know dealing with spammy messages sucks. If only there was a way spammy messages could be blocked from being posted in your channels and threads in the first place… Thankfully, every community’s best friend in moderation - AutoMod - just got buffed with some new spam-cleansing abilities. Enable AutoMod in Server Settings > AutoMod to save hours of moderation work with these new filters: **Block Mention Spam** - Detect and block mention spam and mention raids before they happen to keep chat clean and prevent members in your community from receiving annoying mention notifications and ghost pings. Set your own limit on the number of mentions a message may contain (up to 50). Block Spam Content - Detect and block messages containing text content that has been widely flagged as spam by user reports. Great for blocking unwanted spam like those pesky free Nitro scams, unsolicited advertisements, and invite spam that disrupt your Community's friendly conversations and game nights. Check out our updated AutoMod FAQ help article: https://support.discord.com/hc/en-us/articles/4421269296535-AutoMod-FAQ Our early plans and progress to protect communities from raids We know being raided is one of the worst experiences you and your Community can go through which is why we’re giving you a heads up that we’re exploring ways to better detect and prevent raids before they take place, while making it easier to de-escalate a raid when they do happen. While we’re in the earlier stages of progress here at Discord Labs™, rest assured we’re eager to equip you with effective tools to more easily deal with raids as quickly as we can. So what can you do now? **Report Raids**: You can now directly send us feedback reports when a raid is taking place in your server by going to your Server Dropdown and selecting Report Raid. If your Community is being raided, we encourage you to partner with us in our moderation efforts by reporting the situation as we’ll be regularly reviewing your raid reports to develop tooling that combat raids. Pause invitations to your Community at anytime - You can now pause your invite links for when you want to temporarily prevent people from joining your server without having to revoking your invite links or vanity URLs - just head over to Server Settings > Invites. Because this cuts off the flow of new arrivals, you'll have an easier time halting a raid in progress. It also lets well-intentioned community members trying to join know that the stoppage is temporary, and that soon, they’ll be able to come back to the same link and try again. Learn more about pausing invites: https://support.discord.com/hc/en-us/articles/8458903738647 *How we’re protecting communities from suspicious links: * To help y’all with the legwork in dealing with messages containing malicious links, we’ve buffed every Community server with a proactive suspicious link filter designed to detect and block messages containing links that are verified to be harmful, such as those leading to malware or phishing sites that try to steal people’s personal or financial information. We’ll be continuously working on identifying links that might mislead or redirect users by reviewing a regularly updated list of unsafe and phishing websites: https://support.discord.com/hc/en-us/articles/7326747355159-Why-was-my-message-not-delivered-

kedrokedrokedrokedrokedro Kedro Viz 5 0

Hey Admins and Mods, Y’all ready for some of that ~ juicy ~ product news? After a bunch of testing, we’re excited to begin rolling out Forum Channels across Community servers! Forum Channels are a new type of channel that provide a dedicated, organized space for conversations. Since conversations are contained in individual posts, your members can browse topics before jumping in—without worrying about breaking up or getting in the way of other on-going conversations. If you’ve been waiting for this news in anticipation and are ready to dive right in, go right ahead and click that Create Channel button. You’ll see that forum channels are now an option for you. Want to do a bit more deep thonking and research around how to best use Forum Channels before diving in? The following article on our Community Resources site will guide you on your journey to reaching forum Valhalla:

@hereThis is a quick announcement that Kedro 0.18.3 is out! > TLDR: In this release, we introduce a new autodiscovery project pipelines feature, improved the IPython experience with an alias, and

%reload_kedro

now accepts keywords arguments. There are couple of fixes to make Rich’s log less verbose and work better with Databricks. Kedro 0.18.3 is a non-breaking release in the 0.18.x series, which means you can upgrade with:

pip install kedro==0.18.3

or

conda install -c conda-forge kedro==0.18.3

🚀 Major Features/Changes - [NEW] - autodiscovery of project pipelines. See our documentation for more details. - The Kedro IPython extension should now be loaded with %load_ext kedro.ipython. - The line magic

%reload_kedro

now accepts keywords arguments, e.g. %reload_kedro --env=prod. - Improved resume pipeline suggestion for

SequentialRunner

, it will backtrack the closest persisted inputs to resume. 🪲 Bug fixes and other changes - Changed default

False

value for rich logging

show_locals

, to make sure credentials and other sensitive data isn’t shown in logs. - Rich traceback handling is disabled on Databricks so that exceptions now halt execution as expected. - When using

kedro run -n [some_node]

, if

some_node

is missing a namespace the resulting error message will suggest the correct node name. - The Kedro IPython extension now surfaces errors when it cannot load a Kedro project. - And more … You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/releases/tag/0.18.3