I fell in love with pip-compile after kedro introduced me to it

We want to use the new dependency resolver part of pip, but it's not good enough yet https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html

Hi, certainly not expecting an answer over the weekend, but I am having some trouble with build-docs and wanted to get it down

When I run the command, I get the following error:

The conflict is caused by:ERROR: Cannot install kedro-iris==0.1 and kedro-iris[docs]==0.1 because these package versions have conflicting dependencies. kedro-iris[docs] 0.1 depends on nbstripout==0.3.3 kedro-iris 0.1 depends on nbstripout~=0.4

I know that the "kedro-iris 0.1" dependency comes from kedro-iris/src/requirements.txt, but I don't know where the kedro-iris[docs] requirement comes from. I looked in the docs folder, but didn't find anything in the conf.py or index.rst files

I get the same problem in a different project:

ERROR: Cannot install kedro-mnist==0.1 and kedro-mnist[docs]==0.1 because these package versions have conflicting dependencies. The conflict is caused by: kedro-mnist[docs] 0.1 depends on ipykernel~=5.3 kedro-mnist 0.1 depends on ipykernel==6.0.1

Hi @User are you getting this error when you do

kedro install

? Could you paste your

requirements.in

file?

Also check out this medium article on the Spaceflights tutorial! https://towardsdatascience.com/learn-you-some-kedro-be67d4fc0ce7

Hi, I'm getting it when I run kedro build-docs in both the iris demo project and my own mnist project

Well, actually I checked again, and the problem isn't appearing in the iris project anymore, I'm not sure why.

So to be clear which project I'm asking about, in my own project, kedro-mnist, I run kedro build-docs, and get the following error

ERROR: Cannot install kedro-mnist==0.1 and kedro-mnist[docs]==0.1 because these package versions have conflicting dependencies. The conflict is caused by: kedro-mnist[docs] 0.1 depends on ipykernel~=5.3 kedro-mnist 0.1 depends on ipykernel==6.0.1

And here's the requirements.in file for my project

Thank you! I'll take a look through this.

@User are you running in a fresh virtual environment or one with stuff already installed?

I made this virtual environment when I made the project with kedro install. I think I manually installed pandas intro it, but nothing else

I can give you a pip freeze output if that would help

Hi @User later today I'm going to try and follow your same steps

at the moment I'm not sure what's up

Has anyone delt with an issue where kedro docker build fails due to "can't find version that satisfies the requirement pywin32==301 (from versions: none)?

Tried just deleting them before pip freeze, but then I get an error for conda files where the directory is wrong "/home/conda...."

@User This kis something we have ran into, its a side effect of running pip-compile from windows. We actually ended up with 3 requirements files. If you were to split like this, it is more complicated but will cut down on what you install in docker. I suspect that you do not need pywin32 in docker, you only need it with jupyter. * base/prod - everyone installs this everywhere (kedro, pandas...) * ci - These are neede when ci is running (pytest, docs, ...) * dev - includes handy dev tools like (jupyter, ipython...)

I am on Windows actually though so how do I get the right directories when freezings? These seem to be Linux paths

@User Are you using

pip-freeze

or

pip-compile

? Which paths? are you asking about the filepaths generated by

pip-compile

in your requirements.txt? Oh I see for conda files. What command are you running that causes this? Are you using conda in docker?

I'm currently using a conda environment to run "kedro docker build" that results in the error

Here are some options I can think of. * manually remove pywin32 like you tried * Using Linux to do your pip-comile (maybe docker can do this) * split requirements into multiple files so that jupyter does not appear in the production requirements.in * remove jupyter from requirements.in and rerun pip-compile (kedro build-reqs)

Since a lot of the vital files are all conda based and windows only that won’t really work. Does kedro-docker really only support Linux?

Hi @User I'm looking into your docker question and will see what we can do - IIRC our docker plugin was written long before docker on Windows matured.