https://kedro.org/ logo
Join Discord
Powered by
Hmm the problem seems to be happening on pillow si...
# beginners-need-help
z
Hmm the problem seems to be happening on pillow side, so maybe pillow version?
@datajoely Created a topic here What is happening is in the save from PIL. It does this check, but the file we pass is an s3 file-like object
Copy code
python
        filename = ""
        open_fp = False
        if isinstance(fp, Path):
            filename = str(fp)
            open_fp = True
        elif isPath(fp):
            filename = fp
            open_fp = True
        elif fp == sys.stdout:
            try:
                fp = sys.stdout.buffer
            except AttributeError:
                pass
        if not filename and hasattr(fp, "name") and isPath(fp.name):
            # only set the name for metadata purposes
            filename = fp.name
This is from 
PIL.Image
, and this does not support the s3 file-like that kedro is passing.
d
No one else has complained about this to my
z
It might be something from newer versions of PIL, or something like that, but I was able to reproduce the error quite easily.
d
so in 0.18.0 we have the requirement 
Pillow~=9.0
in 0.17.7 we had
Copy code
"pillow.ImageDataSet": [
        "Pillow~=9.0; python_version > '3.6'",
        "Pillow~=8.0; python_version == '3.6'",
    ]
z
Pillow==9.1.0, maybe this is the problem?
d
try downgrading to a 
0.8.x
see if it works?
z
Any restriction to s3fs?
Same error, btw.
d
there shouldn't be
this should be delegated to fsspec
I'm still struck by this not being reported, we know lots of users do this
what's your version of fsspec and s3fs?
z
I put the code here: https://github.com/Zemeio/kedrotest that I am using, just hid my personal s3
fsspec==2022.1.0 s3fs==2022.1.0
d
So in 0.17.7 we did: 
S3FS = "s3fs>=0.3.0, <0.5"
and 
fsspec>=2021.4, <=2022.1
z
Pillow is weird, the problem is happening here, maybe I should be posting this there https://github.com/python-pillow/Pillow/blob/2613429e24f94039e7517d0aff1da61d9879a922/src/PIL/Image.py#L2215
It says it accepts file, but it does not
d
maybe, just see if it works with the older versions of the libraries above
after that I'm out of ideas :S
z
Same error. I am out of ideas as well haha
I will stop for today, but I won't give up on this. Maybe I will try to raise this on pillow, or try to reproduce the error using just s3fs and pillow to raise to them
That being said, if another kind soul happens to know how to solve this, please let me know = )
Thanks for the huge help datajoely!
@datajoely hey man, I found an easy solution! It is a little counter-intuitive, but if we pass the format as the argument as well it works (since PIL ignores that part of the code). This is the commit that fixed it: https://github.com/Zemeio/kedrotest/commit/e2551f28a7bc2cfeb1859805202e1991d971bc7e
PreviousNext
Powered by