williamc
06/23/2022, 4:31 PMkedro.io.core.VersionNotFoundError
. Bucket has versions all the way up to 2022-06-07T22.04.39.460Z/
and the error says 2022-06-23T16.20.52.945Z
. Is this the intended behavior? Thanksnoklam
06/27/2022, 5:35 PMkedro run
it should grab the latest dataset. Maybe also useful to share the catalog.yml
or just the related datasetswilliamc
06/27/2022, 5:51 PMPickleDataSet
. The relevant portion of the stack trace is
File "/home/ec2-user/SageMaker/tars/tars-env/lib/python3.9/site-packages/kedro/io/core.py", line 539, in _fetch_latest_load_version
raise VersionNotFoundError(f"Did not find any versions for {self}")
kedro.io.core.VersionNotFoundError: Did not find any versions for KerasStringLookupLayer(backend=pickle, filepath=.../data/06_models/censor_rating_lookup_conf.pkl, load_args={}, protocol=s3, save_args={}, version=Version(load=None, save='2022-06-27T16.59.49.284Z'))
The relevant portion of my data catalog is as follows:
censor_rating_lookup:
type: tars.extras.datasets.tensorflow.KerasStringLookupLayer
filepath: s3:${s3_bucket}${exhibitor}/data/06_models/censor_rating_lookup_conf.pkl
backend: pickle
versioned: True
noklam
06/28/2022, 12:57 PMversioned: true
?
From your stack trace, it fails to see any versions available.williamc
06/28/2022, 1:33 PMnoklam
06/28/2022, 3:49 PMwilliamc
06/29/2022, 6:17 PMs3:*Object
which I think is insufficient for Kedro's versioned datasets. Once I set the policy to s3:*
everything worked as intended. Sorry for wasting your time here 😅noklam
06/30/2022, 3:31 PMListObjects
or ListObjectsV2
, which your regex excludedInsufficient permissions to list objects
be thrown here, @datajoely any idea about this? I am not super familiar with the S3 policy, does it just shows nothing or it will tell you permission is needed?datajoely
06/30/2022, 3:42 PMwilliamc
06/30/2022, 4:13 PMListObjects
is necessary for versioned datasets to work. I originally coded my project on an EC2 instance that had its own, much more permissive, IAM role, and it worked fine because of that. Setting the right permissions for SageMaker notebooks is a bit more involved and the original role I was passing on was different to the EC2 one. Those IAM errors don't actually tell you much more that access denied or something like thatnoklam
07/01/2022, 10:55 AMfsspec
and I don't know how it handles this, there may be a chance that they are slightly different in different storage too.
I think there may be something we can do about it but will require some more investigation. I don't have access to quickly spin up S3 storage and play with the S3 Policy myself, but I will jot some notes about this first, thank you!datajoely
07/01/2022, 10:56 AM