Title
z
Zemeio
04/28/2022, 1:06 PMHmm the problem seems to be happening on pillow side, so maybe pillow version?
@datajoely Created a topic here
What is happening is in the save from PIL. It does this check, but the file we pass is an s3 file-like object
python
filename = ""
open_fp = False
if isinstance(fp, Path):
filename = str(fp)
open_fp = True
elif isPath(fp):
filename = fp
open_fp = True
elif fp == sys.stdout:
try:
fp = sys.stdout.buffer
except AttributeError:
pass
if not filename and hasattr(fp, "name") and isPath(fp.name):
# only set the name for metadata purposes
filename = fp.namePIL.Imaged
datajoely
04/28/2022, 1:15 PMNo one else has complained about this to my
z
Zemeio
04/28/2022, 1:15 PMIt might be something from newer versions of PIL, or something like that, but I was able to reproduce the error quite easily.
d
datajoely
04/28/2022, 1:15 PMso in 0.18.0 we have the requirement
Pillow~=9.0in 0.17.7 we had
"pillow.ImageDataSet": [
"Pillow~=9.0; python_version > '3.6'",
"Pillow~=8.0; python_version == '3.6'",
]z
Zemeio
04/28/2022, 1:16 PMPillow==9.1.0, maybe this is the problem?
d
datajoely
04/28/2022, 1:18 PMtry downgrading to a
0.8.xz
Zemeio
04/28/2022, 1:20 PMAny restriction to s3fs?
Same error, btw.
d
datajoely
04/28/2022, 1:21 PMthere shouldn't be
this should be delegated to fsspec
I'm still struck by this not being reported, we know lots of users do this
what's your version of fsspec and s3fs?
z
Zemeio
04/28/2022, 1:24 PMI put the code here: https://github.com/Zemeio/kedrotest that I am using, just hid my personal s3
fsspec==2022.1.0
s3fs==2022.1.0
d
datajoely
04/28/2022, 1:34 PMSo in 0.17.7 we did:
S3FS = "s3fs>=0.3.0, <0.5"and
fsspec>=2021.4, <=2022.1z
Zemeio
04/28/2022, 1:35 PMPillow is weird, the problem is happening here, maybe I should be posting this there https://github.com/python-pillow/Pillow/blob/2613429e24f94039e7517d0aff1da61d9879a922/src/PIL/Image.py#L2215
It says it accepts file, but it does not
d
datajoely
04/28/2022, 1:35 PMmaybe, just see if it works with the older versions of the libraries above
after that I'm out of ideas :S
z
Zemeio
04/28/2022, 1:37 PMSame error. I am out of ideas as well haha
I will stop for today, but I won't give up on this. Maybe I will try to raise this on pillow, or try to reproduce the error using just s3fs and pillow to raise to them
That being said, if another kind soul happens to know how to solve this, please let me know = )
Thanks for the huge help datajoely!
@datajoely hey man, I found an easy solution!
It is a little counter-intuitive, but if we pass the format as the argument as well it works (since PIL ignores that part of the code). This is the commit that fixed it: https://github.com/Zemeio/kedrotest/commit/e2551f28a7bc2cfeb1859805202e1991d971bc7e