This is the best place for me. Waiting for more content 😄

What would you like see more of @User ?

Ideally some working workflow a full process, on the infographic or flowchart to understand the logic and have a big picture of the process and use case at the end.

Have you tried the kedro spaceflights tutorial?

I'm just going to point you to the Kedro Spaceflights tutorial that @User mentioned: https://kedro.readthedocs.io/en/stable/03_tutorial/01_spaceflights_tutorial.html Try this and let us know how you get on.

Ok I will, thank you

This walkthrough may also be helpful 🙂 https://dev.to/waylonwalker/kedro-spaceflights-part-1-2o3p

I have both part 1 and 2 on Youtube as well.

Does kedro docker lib only for specific for kedro projects? Or can It be utilised to deploy a flask ml api?

It's very much written with Kedro in mind - especially for users who don't know much about docker. So it's not hard to do from scratch, but it does most of the Kedro stuff for you e.g. the Viz component have all the ports set up nicely.

Ahh , I seee. I tried automating the same stuff few months bck, 5-6 commands to create container but my code use to take requirements.txt and docker file. I am curious how you guys are generating it.

Here is the source code - https://github.com/quantumblacklabs/kedro-docker/blob/develop/kedro_docker/plugin.py - it is mostly just generating docker stuff for you

Thanks, I see, you guys are not utilising the database or not interacting with any databases or any cloud infrastructure except dolt. I faced lot of challenges to install configurations for database setup inside container. Most of ml/ds use cases interact with these technologies.

Hi @User I'm not sure that's true - give me a sec to point you to the right sections

So we have two ways of interacting with SQL via Pandas/SQLAlchemy: https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.pandas.SQLQueryDataSet.html And via Spark: https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.spark.SparkJDBCDataSet.html

The point is Kedro doesn't re-invent the wheel it wraps technologies like Pandas and Spark to do the actual database communication, but provides a YAML API for declaring different catalog entries

does that make sense?

Each of these connectors, like all of our connectors can be deployed locally or in the cloud

Yes, it does. This helps in understanding the interaction with different systems of Kedro. I thought you guys are using apache Livy or rest API to trigger jobs on cluster

No - we're much simpler than that

Kedro isn't an orchestrator it's a framework for organising your code when building ML pipelines and models

Thanks , let me explore more and deep dive in kedro

Keep the questions coming!

Hi guys, is it possible to run kedropackage using miniconda?

Hey @User ! As in from the CLI use

kedro package

?

Yeah, I'm having an issue with kedro package, using a python 3.8 from miniconda

I'll send a screenshot

Sure please do

2021-06-28 14:41:50,423 - root - INFO - Registered CLI hooks from 1 installed plugin(s): kedro-telemetry-0.1.1 Traceback (most recent call last): File "/home/grover/miniconda3/lib/python3.8/site-packages/kedro/framework/cli/utils.py", line 380, in load_entry_points entry_point_commands.append(entry_point.load()) File "/home/grover/miniconda3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2446, in load self.require(*args, **kwargs) File "/home/grover/miniconda3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2469, in require items = working_set.resolve(reqs, env, installer, extras=self.extras) File "/home/grover/miniconda3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 775, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.VersionConflict: (python-slugify 5.0.2 (/home/grover/miniconda3/lib/python3.8/site-packages), Requirement.parse('python-slugify~=4.0')) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/grover/miniconda3/bin/kedro", line 8, in sys.exit(main()) File "/home/grover/miniconda3/lib/python3.8/site-packages/kedro/framework/cli/cli.py", line 264, in main cli_collection = KedroCLI(project_path=Path.cwd()) File "/home/grover/miniconda3/lib/python3.8/site-packages/kedro/framework/cli/cli.py", line 187, in __init__ ("Project specific commands", self.project_groups), File "/home/grover/miniconda3/lib/python3.8/site-packages/kedro/framework/cli/cli.py", line 238, in project_groups plugins = load_entry_points("project") File "/home/grover/miniconda3/lib/python3.8/site-packages/kedro/framework/cli/utils.py", line 382, in load_entry_points raise KedroCliError(f"Loading {name} commands from {entry_point}") from exc kedro.framework.cli.utils.KedroCliError: Loading project commands from airflow = kedro_airflow.plugin:commands

I'm testing the astronomer airflow deployment