https://kedro.org/ logo
Powered by
#advanced-need-help
Title
# advanced-need-help
d

datajoely

07/15/2022, 8:03 AM
Or at least the virtual environment that airflow is looking for
a

arum

07/15/2022, 8:05 AM
d

datajoely

07/15/2022, 8:05 AM
Right, but if airflow cant import kedro it cant be installed
At least at the location it's looking for
If you're able to run terminal to do 
pipe freeze
or 
which python
you can see where it's trying to look for
a

arum

07/15/2022, 8:08 AM
$ which python /Users/sl/mambaforge-pypy3/bin/python
airflow is run in docker by astro, how can we make kedro available to airflow
d

datajoely

07/15/2022, 8:17 AM
And was kedro in the pip freeze?
a

arum

07/15/2022, 8:32 AM
yes
Copy code
kedro==0.18.1
kedro-airflow==0.5.0
kedro-docker==0.3.0
kedro-telemetry==0.2.1
kedro-viz==4.7.0
well, i have run the docker container in bash and did a pip freeze, which does not have kedro
so here is what i could debug, there is no kedro available with 
FROM quay.io/astronomer/ap-airflow:2.0.0-buster-onbuild
as is in the docs
pip freeze in the docker image did not have kedro
solutions 1) i have added RUN pip install kedro to the Docker and it worked
2) I have tried the latest image 
FROM quay.io/astronomer/ap-airflow:2.3.3-onbuild
in Docker
and this also worked
Would be good to update the docs as it could save time for others
FYI, also the demo dag new-kedro-project isn't successful
the input variables defined in catalog.yml are not the same as those used in pipeline.py and the demo DAG fails
Powered by