kedro 0.17.7

@here We're excited to announce that Kedro 0.18.0 has finally been released! 🎉

TL;DR

In this release we reduce the complexity of the project template to get us closer to a stable release of the framework. We've introduced the full micro-packaging workflow 📦, which allows you to import packages, utility functions and existing pipelines into your Kedro project. Integration with IPython and Jupyter has been streamlined in preparation for enhancements to Kedro's interactive workflow. Additionally, the release comes with long-awaited Python 3.9 and 3.10 support 🐍. Please make sure to trial our experiment tracking workflow in Kedro. ⚠️ There are breaking changes in this release, you can read the full release notes: https://github.com/kedro-org/kedro/releases What can you expect in this release? - The micro-packaging workflow 📦,which allows you to package and export Kedro pipelines, Python packages, utility functions, and modules in your code base, to your choice of Python package repository. - The only recommended way to work with Kedro in Jupyter or IPython is now the Kedro IPython extension. - Reduced boilerplate and complexity of the project template. We have removed

cli.py

from the Kedro project template. - A packaged Kedro project will respect the same hierarchy when executed with

python -m my_package

. - Added Python 3.9 and 3.10 support 🐍 .Dropped support for Python 3.6. - Refactored the load and save operations for

pandas

datasets in order to leverage

pandas

own API and delegate

fsspec

operations to them. This reduces the need to have our own

fsspec

wrappers, like in previous versions. How do I get access to the latest release? Get the latest version of Kedro with

pip install kedro == 0.18.0

and await being able to install Kedro from Conda with

conda install -c conda-forge kedro==0.18.0

.

Links to new features: - Experiment Tracking: https://kedro.readthedocs.io/en/0.18.0/tutorial/set_up_experiment_tracking.html - The new iPython/Jupyter Workflow: https://kedro.readthedocs.io/en/0.18.0/tools_integration/ipython.html - Micro-packaging Workflow: https://kedro.readthedocs.io/en/stable/nodes_and_pipelines/micro_packaging.html

Kedro-Viz 4.4.0 is out! @here We're excited to announce that Kedro-Viz 4.4.0 is out! 🥳 This version brings some great improvements and important fixes. Have a look at the full release notes (footnote #1) for everything new. What can you expect in this release? - A way to export your experiment tracking data (footnote #2) as a CSV - A new pop-up to remind you to upgrade Kedro-Viz to get the latest features - An

expandAllPipelines

feature flag that expands all modular pipelines by default. Turn this on by clicking the settings icon ⚙️ How do I get access to the latest release? Get the latest version of Kedro-Viz with

pip install kedro-viz==4.4.0

(Python) or with

npm install @quantumblack/kedro-viz@latest

(React) What's next? Python 3.9 and 3.10 support will come in a following patch release. We'll also update our demo around that time too. Until next time, The Kedro Team 💜 Footnotes: - #1: https://github.com/kedro-org/kedro-viz/releases/tag/v4.4.0 - #2: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html

I'll be giving a crash course on Kedro next week! Come along if you're new to Kedro and want to learn more about it and code along with the beginner tutorial 🚀 https://discord.gg/sCZsWWsK?event=964120084413038633

Kedro Crash Course

The recording of my talk is now available on youtube for those who missed it yesterday 🙂

@here We're currently conducting user research on kedro

experiment tracking

. Kindly respond with a 👍 if you have used this feature, or know someone who has. Thank you.

Hi @here! This is a quick announcement that Kedro 0.18.1 is out. It’s available on PyPI and Conda. This version has several bug fixes and major improvements including: 🐛 Bug fixes: - Fixed

CONFIG_LOADER_CLASS

validation so that `TemplatedConfigLoader `can be specified in settings.py. Any

CONFIG_LOADER_CLASS

must be a subclass of

AbstractConfigLoader

. - Fixed

Jinja2

syntax loading with

TemplatedConfigLoader

using globals.yml. - Removed global

_active_session

,

_activate_session

and

_deactivate_session

. Plugins that need to access objects such as the config loader should now do so through context in the new

after_context_created

hook. - Made

hook_manager

argument optional for runner.run. - kedro docs now opens an online version of the Kedro documentation instead of a locally built version. 🚀 Major features: - Added a new hook

after_context_created

that passes the

KedroContext

instance as

context

. - Added a new CLI hook

after_command_run

. - The Kedro package no longer contains a built version of the Kedro documentation significantly reducing the package size 🐳 You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/blob/main/RELEASE.md Kedro 0.18.1 is a non-breaking release in the 0.18.x series, upgrade now with:

pip install kedro==0.18.1
conda install -c conda-forge kedro==0.18.1

*@here We have two exciting announcements! 🎉 🎉🎉 * *Kedro is officially online 🌍 * We have a Kedro website: https://kedro.org/. This is our first version; please give us feedback. We're working with Databricks We're setting up a hackathon with a team from Databricks to address issues related to the Databricks / Kedro development and deployment workflows. We would love to understand the scope of issues that you have encountered. Comment in a thread below with your struggles or use the 🗓️ emoji so that we can book time in your calendar to dive deeper into your workflow.

Kedro

Kedro-Viz 4.7.0 is out! @here We're excited to announce that Kedro-Viz 4.7.0 is out! 🥳 This version brings some great improvements and important fixes. Have a look at the full release notes (#1) for everything new. Check out our hosted demo (#2). What can you expect in this release? - Support for

matplotlib

images in the Metadata panel and Modal 📊 - Support for all

plotly

chart types including Sankey diagrams 📈 - Python 3.9 and 3.10 support - A whole host of other UX/UI improvements and bug fixes, too 🐛 How do I get access to the latest release? - Get the latest version of Kedro-Viz with

pip install kedro-viz==4.7.0

(Python) or with

npm install @quantumblack/kedro-viz@latest

(React) What's next? - An easier way to run Kedro-Viz in platforms like Databricks without a CLI and further honing of the application experience. - Improvements on Experiment Tracking (#3), like being able to track plots and datasets with experiments Until next time, The Kedro Team 💜 Footnotes: - #1: https://github.com/kedro-org/kedro-viz/releases/tag/v4.7.0 - #2: https://demo.kedro.org/ - #3: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html

Hi @here! This is a quick announcement that Kedro 0.18.2 is out and is available on PyPI and Conda > TL;DR This release is special because we've worked on improving Databricks Repos support and improved the CLI experienced by adding in pretty logging to the CLI with Rich and reducing the kedro new prompts to one question. This version also has several bug fixes and improvements: 🚀 Major features: - Kedro now uses the Rich library to format terminal logs and tracebacks. - The file

conf/base/logging.yml

is now optional. See our documentation for details. - Introduced a

kedro.starters

entry point. This enables plugins to create custom starter aliases used by

kedro starter list

and

kedro new

. - Reduced the

kedro new

prompts to just one question asking for the project name. 🐛 Bug fixes: - Bumped

pyyaml

upper bound to make Kedro compatible with the

pyodide

stack. - Kedro's framework-side logging configuration no longer performs file-based logging. Hence superfluous `info.log`/`errors.log `files are no longer created in your project root, and running Kedro on read-only file systems such as Databricks Repos is now possible. - Required

cookiecutter>=2.1.1

to address a known command injection vulnerability. - The session store no longer fails if a username cannot be found with

getpass.getuser

. - Added generic typing for

AbstractDataSet

and

AbstractVersionedDataSet

as well as typing to all datasets. To make the most of our new rich logging you can either delete

conf/base/logging.yml

if it has not been customised, or add

rich.logging.RichHandler

to the logging handlers. See the logging documentation for more. You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/blob/main/RELEASE.md Kedro 0.18.2 is a non-breaking release in the 0.18.x series, upgrade with:

pip install kedro==0.18.2
conda install -c conda-forge kedro==0.18.2

Kedro-Viz 5.0.0 is out! @here We're excited to announce that Kedro-Viz 5.0.0 is out! 🥳 This new major-release version brings a great new feature and the removal of support for an older version of Kedro. Have a look at the full release notes (#1) for everything new. Check out our hosted demo (#2). What can you expect in this release? - Support for visualising Matplotlib and Plotly plots in experiment tracking 📊 - Remove support for Kedro v16 🧹 ⚠️ Note this last point is a breaking change! If you're using Kedro v17 or v18, please upgrade to continue getting the best out of Kedro-Viz. ⚠️ How do I get access to the latest release? - Get the latest version of Kedro-Viz with

pip install kedro-viz==5.0.0

(Python) or with

npm install @quantumblack/kedro-viz@latest

(React) What's next? - Improvements on Experiment Tracking (#3), like being able to track and plot metrics over time. - And follow our roadmap (#4) for more updates. Until next time, The Kedro Team 💜 Footnotes: - #1: https://github.com/kedro-org/kedro-viz/releases/tag/v5.0.0 - #2: https://demo.kedro.org/ - #3: https://kedro.readthedocs.io/en/stable/tutorial/set_up_experiment_tracking.html - #4: https://github.com/orgs/kedro-org/projects/3/views/1

Hey Admins and Moderators, We've got an update on all things Community today! If you haven't joined our Discord Admin Community… what are you waiting for? Head over to Server Setting > Community overview and join us — we've got a slew of exciting and fun stuff planned that you won’t want to miss out on. 🪴 __*Below is a sneak peek at the Stage events we’ve got cookin’!*__ August 16th: Safety on Discord August 25th: Utilizing Automod in your Community To join our Discord Admin Community, your server will need to hit a few engagement requirements. Once your community reaches the requirements listed , any of your Admins or Moderators with the “Manage Server” permission will be able to join the Discord Admin Community via Server Settings > Community. 📚 __**Community Resources**__ We recently released our Community Resources, a one-stop shop for learning how to build, engage, grow, and manage a space where anyone can belong. New articles, showcases, and more will be added in the future, so be sure to bookmark it for later. If you haven't already taken a peek at some of our content, you can check out the latest video! https://dis.gd/H2M_Your_Discord_Community 🤖 __Apps & Bots__ If your server uses a verified app, the way you send commands has probably changed in the last year - if it hasn’t, it may very soon! Developers of verified apps have been migrating to Slash Commands. The deadline to finish migrating is the end of August, which means on Sept 1, any apps that haven’t finished may have reduced functionality. If you haven’t tried out slash commands yet, type / in your server to see all the commands you have access to. If your favorite app isn’t listed, they may not yet have enabled slash commands. Or, you might need to check out the new permissions system in Server Settings > Integrations. If you run across any issues, you can reach out to individual app developers to find out more. Learn more here:

Hey Admins and Mods! In our mission to help you keep your communities safe and lift some of that moderation work off your shoulders, we’ve set our sights on helping you more easily detect and prevent spam and raids. Today's spotlight introduces new features that help keep your communities cleaner from spam, and a preview into our early plans on building tools to deal with raids. Keep your Community clean from spam with new AutoMod spam filters We know dealing with spammy messages sucks. If only there was a way spammy messages could be blocked from being posted in your channels and threads in the first place… Thankfully, every community’s best friend in moderation - AutoMod - just got buffed with some new spam-cleansing abilities. Enable AutoMod in Server Settings > AutoMod to save hours of moderation work with these new filters: **Block Mention Spam** - Detect and block mention spam and mention raids before they happen to keep chat clean and prevent members in your community from receiving annoying mention notifications and ghost pings. Set your own limit on the number of mentions a message may contain (up to 50). Block Spam Content - Detect and block messages containing text content that has been widely flagged as spam by user reports. Great for blocking unwanted spam like those pesky free Nitro scams, unsolicited advertisements, and invite spam that disrupt your Community's friendly conversations and game nights. Check out our updated AutoMod FAQ help article: https://support.discord.com/hc/en-us/articles/4421269296535-AutoMod-FAQ Our early plans and progress to protect communities from raids We know being raided is one of the worst experiences you and your Community can go through which is why we’re giving you a heads up that we’re exploring ways to better detect and prevent raids before they take place, while making it easier to de-escalate a raid when they do happen. While we’re in the earlier stages of progress here at Discord Labs™, rest assured we’re eager to equip you with effective tools to more easily deal with raids as quickly as we can. So what can you do now? **Report Raids**: You can now directly send us feedback reports when a raid is taking place in your server by going to your Server Dropdown and selecting Report Raid. If your Community is being raided, we encourage you to partner with us in our moderation efforts by reporting the situation as we’ll be regularly reviewing your raid reports to develop tooling that combat raids. Pause invitations to your Community at anytime - You can now pause your invite links for when you want to temporarily prevent people from joining your server without having to revoking your invite links or vanity URLs - just head over to Server Settings > Invites. Because this cuts off the flow of new arrivals, you'll have an easier time halting a raid in progress. It also lets well-intentioned community members trying to join know that the stoppage is temporary, and that soon, they’ll be able to come back to the same link and try again. Learn more about pausing invites: https://support.discord.com/hc/en-us/articles/8458903738647 *How we’re protecting communities from suspicious links: * To help y’all with the legwork in dealing with messages containing malicious links, we’ve buffed every Community server with a proactive suspicious link filter designed to detect and block messages containing links that are verified to be harmful, such as those leading to malware or phishing sites that try to steal people’s personal or financial information. We’ll be continuously working on identifying links that might mislead or redirect users by reviewing a regularly updated list of unsafe and phishing websites: https://support.discord.com/hc/en-us/articles/7326747355159-Why-was-my-message-not-delivered-

kedrokedrokedrokedrokedro Kedro Viz 5 0

Hey Admins and Mods, Y’all ready for some of that ~ juicy ~ product news? After a bunch of testing, we’re excited to begin rolling out Forum Channels across Community servers! Forum Channels are a new type of channel that provide a dedicated, organized space for conversations. Since conversations are contained in individual posts, your members can browse topics before jumping in—without worrying about breaking up or getting in the way of other on-going conversations. If you’ve been waiting for this news in anticipation and are ready to dive right in, go right ahead and click that Create Channel button. You’ll see that forum channels are now an option for you. Want to do a bit more deep thonking and research around how to best use Forum Channels before diving in? The following article on our Community Resources site will guide you on your journey to reaching forum Valhalla:

@hereThis is a quick announcement that Kedro 0.18.3 is out! > TLDR: In this release, we introduce a new autodiscovery project pipelines feature, improved the IPython experience with an alias, and

%reload_kedro

now accepts keywords arguments. There are couple of fixes to make Rich’s log less verbose and work better with Databricks. Kedro 0.18.3 is a non-breaking release in the 0.18.x series, which means you can upgrade with:

pip install kedro==0.18.3

or

conda install -c conda-forge kedro==0.18.3

🚀 Major Features/Changes - [NEW] - autodiscovery of project pipelines. See our documentation for more details. - The Kedro IPython extension should now be loaded with %load_ext kedro.ipython. - The line magic

%reload_kedro

now accepts keywords arguments, e.g. %reload_kedro --env=prod. - Improved resume pipeline suggestion for

SequentialRunner

, it will backtrack the closest persisted inputs to resume. 🪲 Bug fixes and other changes - Changed default

False

value for rich logging

show_locals

, to make sure credentials and other sensitive data isn’t shown in logs. - Rich traceback handling is disabled on Databricks so that exceptions now halt execution as expected. - When using

kedro run -n [some_node]

, if

some_node

is missing a namespace the resulting error message will suggest the correct node name. - The Kedro IPython extension now surfaces errors when it cannot load a Kedro project. - And more … You can find the full list of changes in the release notes: https://github.com/kedro-org/kedro/releases/tag/0.18.3

@here There are a few things happening in October. It's

hacktoberfest

and it's time for Kedro Showcase!

hacktoberfest

contributions Look out for `hacktoberfest`(1) tickets on the Kedro GitHub issues (2) and start get contributing. We have a range of documentation tickets all the way to feature development. An easier contribution would be to add your company to our "Who Likes Kedro?" list on our README.md (3) 1. https://hacktoberfest.com/ 2. https://github.com/kedro-org/kedro/labels/hacktoberfest 3. https://github.com/kedro-org/kedro/#who-likes-kedro Kedro Showcase Join us for the Kedro Roadmap Showcase! We'll spotlight the 0.18.x releases and where we're going next. Additionally, we'll spend time discussing one of our upcoming changes to how we handle datasets in Kedro. Expect a lot of updates on new functionality and an open forum with the core team. Bring your questions and, we look forward to seeing you! - Event 1 (9AM BST on Wed, 12th of October): https://www.meetup.com/meetup-group-zltyafrj/events/288920310/ - Event 2 (4 PM BST on Wed, 12th of October): https://www.meetup.com/meetup-group-zltyafrj/events/288920571/

@here Hey everyone! We've got some updates for the Kedro community that we want to let you know; they're focused on a decision to migrate our community to Slack. We're mentioning this at the Kedro Showcases which are taking place today. Migration to Slack We've been struggling to grow our community on Discord and after some research, we've discovered that a large part of our community cannot use Discord because it's banned in their organisations 😞. We've decided to rebuild our community in Slack (#1) because it seems to be the most widely accepted. We'll follow the example set by Great Expectations, dbt and MLflow. We will continue to support the Discord server as we migrate users. The plan is to continue to support the server until the end of November. All conversations on Discord will be searchable with Linen (#2). We know that it's inconvenient to move communities so, get ready for a massive Kedro swag drop when we finish our rebrand (#3). Please shout if you have any questions for us. Links #1 Join Slack: https://join.slack.com/t/kedro-org/shared_invite/zt-1eicp0iw6-nkBvDlfAYb1AUJV7DgBIvw #2 Find Discord conversations: https://linen-discord.kedro.org/ #3 Get swag: https://www.surveys.online/jfe/form/SV_8jfTn7SQDcUiN5c

@here Reminder: the second Kedro Showcase is starting in 1 min! https://mckinsey.zoom.us/j/95837726288?pwd=SjhBenU4YjNCMW8zK0lDalFKdVAxZz09

@hereWe're in the countdown to our community migration to Slack which will take place on the 30th of November. Check our previous announcement for the rationale for why we're doing this. Thank you to everyone that has joined so far and completed our survey for Kedro swag ♥️. Links #1 Join Slack: https://join.slack.com/t/kedro-org/shared_invite/zt-1eicp0iw6-nkBvDlfAYb1AUJV7DgBIvw #2 Get swag: https://www.surveys.online/jfe/form/SV_8jfTn7SQDcUiN5c

@hereWe're in the final month of supporting our Discord server. We're all moving to Slack on the 30th of November. Check our previous announcement for the rationale for why we're doing this and remember to sign up for Kedro swag. About 400 people are in the new Slack workspace ♥️. Links #1 Join Slack: https://join.slack.com/t/kedro-org/shared_invite/zt-1eicp0iw6-nkBvDlfAYb1AUJV7DgBIvw #2 Get swag: https://www.surveys.online/jfe/form/SV_8jfTn7SQDcUiN5c

@hereWe have 20 days left in our Discord server. We're all moving to Slack on the 30th of November. Check our previous announcement for the rationale for why we're doing this and remember to sign up for Kedro swag. More than 400 people are in the new Slack workspace ♥️. Links #1 Join Slack: https://join.slack.com/t/kedro-org/shared_invite/zt-1eicp0iw6-nkBvDlfAYb1AUJV7DgBIvw #2 Get swag: https://www.surveys.online/jfe/form/SV_8jfTn7SQDcUiN5c

@hereWe'll be closing our Discord server soon. It's really important that you move to our new Slack Organisation to continue getting help. We will archive Discord and make it searchable on Linen. Check our previous announcement for the rationale for why we're doing this and remember to sign up for Kedro swag. More than 500 people are in the new Slack workspace already ♥️ Links #1 Join Slack: https://slack.kedro.org/ #2 Get swag: https://www.surveys.online/jfe/form/SV_8jfTn7SQDcUiN5c #3 Archive of Discord: https://linen-discord.kedro.org/

@hereWe're making the changes official. We have moved to Slack (#1) and you'll find 500 members in that channel supporting each other! Check #778996598880862208 for the rationale and remember to sign up for swag (#2). We're also running a survey to gauge interest in a "Software Engineering for Data Scientists and Data Engineers" course that we want to write and run next year so please fill out this survey if this is something you want (#3). We'll see you on Slack ❤️ Links #1 Join Slack: https://slack.kedro.org/ #2 Get swag when we ship our new branding: https://www.surveys.online/jfe/form/SV_8jfTn7SQDcUiN5c #3 Software Engineering for Data Scientists and Data Engineers survey: https://www.surveys.online/jfe/form/SV_37WxRUy5xkC0xkW

Hey Admins and Mods! Reach for the stars with new revenue tools for creators and communities on Discord. Server Subscriptions available in the US! Customize tiers, offer exclusive paid content, and meet future subscribers with Promo Pages—all while providing as much of a free experience to your community as you want. (https://dis.gd/CUServerSubs) Server Subscriptions are only available in the US. Join our waitlist to be notified if they're available in your area. (https://dis.gd/ServerSubsWaitlist) Private Threads are now available to all! Your community no longer needs to have Server Boosting to Level 2 to use Private Threads. Improvements to Server Insights! Dates should be more accurate. Conversations in Forum Channels will contribute to Engagement metrics, and now you can see joins via Student Hubs, Integrations, and Bots. The requirement to be a Communicator has been increased from 1 to 3 messages sent per week. This only applies to the Engagement tab and doesn’t impact Partner requirements. Forum Channels now with Gallery View! Gallery View gives posts in image-heavy Forum Channels like #fanart and #pet-pics room to shine - easily toggle between List View and Gallery View at any time. With this update, Forum Channels are now officially available on all Community servers! AutoMod keeps getting better! Alongside more Custom Rules, we’ve added Allowlists to your custom rules! Messages containing anything you’ve added to your Allowlist will not be flagged by AutoMod. We’ve also added the ability to filter messages using RegEX in your custom rules. () Enjoy your new tools, everyone!

Hey Admins and Mods, Beginning on March 31, over the next several weeks we will start to roll out a new feature called Soundboard. This feature will let users upload custom short sounds to their servers, and then use those sounds to communicate in Voice Channels. Click here to learn more about soundboard: To help you keep your server safe, we have already released some new permissions around Soundboard. You can now access and configure these permissions directly from Server Settings: - Use Soundboard: Determines who in your server is able to use soundboard. - Manage Expressions: Determines who in your server is able to upload and manage sounds. Visit the help center article linked above to read more about Soundboard permissions! Peace and Love, Discord Team

Hey Admins and Mods, Since 2015, every Discord username has been cAsE sEnSitIvE and had a number attached to it called a discriminator (e.g., #0001). This lets you have the same username as someone else as long as you have different discriminators or different case letters. However, this also means you have to remember a set of 4-digit numbers and account for case sensitivity to connect with your friends. ✨**We wanted to make it easier for you to identify and add your friends while preserving your ability to use your preferred name across Discord. So, we are removing discriminators and introducing new, unique usernames and display names.** ✨ 💠 New usernames are lowercase, alphanumeric, limited to certain special characters, and do not have discriminators, making them easier to remember, read, type, and share. Now, rather than having to remember “PhiBi#8936,” you just say “I'm phibiscool” and that’s it. Your new username will be unique to you, and when friends try to add you there won’t be any mistakes of the wrong discriminator or incorrect letter casing. 💠 Display names are what’s shown both to your friends and in communities. Think of them as server nicknames that work globally across Discord. You can change it whenever you want, and it can be whatever you want, including special characters, as long as it follows our Community Guidelines . If a username you were eyeing gets taken before you can claim it, no worries — you can still use it as a display name. 📩 __When you’re eligible to change your old username and claim a new username, you’ll be notified in-app. The older your Discord account, the sooner you will become eligible to pick a new username. 📩__ Visit our blog to hear from one of our founders about why usernames are changing. To learn more about usernames and display names, check out our Help Center Article for all the details.