> * If I run `aws s3 ls s3://commoncrawl/crawl-data/` on the head node
What happens if you run it on one of the core or task nodes?
> I’m sure it’s related to the recent changes at
> the beginning of the month, but I don’t see where I’ve gone wrong.
Assumed the source code is on Git, you could grep for the following
- if using Python and boto3
git grep -F UNSIGNED
- if using Java, Scala (JVM)
git grep -F AnonymousAWSCredentialsProvider
- if the AWS CLI is used
git grep -F -e --no-sign-request
These keywords could point to unauthenticated S3 access which is
Otherwise, could you share some context?
- programming language (Scala, Java, Python, R)
- the exception stack trace
- eventually a code snippet