java.nio.file.NoSuchFileException: AWS Batch

[Sam Tischfield]

I keep getting this error (see log file) when attempting to spin off a job to AWS batch.

-IAM role is set to root (full permissions)

-Bucket is in same region (.command.run, .sh are written there by nextflow on execution)

-AWS CLI tools installed on both host and docker (running on ECS)

Additionally this is the error log produced on the AWS side

20:34:42 download failed: s3://samt-test/79/61d0a4b2e598b162f6f846a055ba03/.command.run to - An error occurred (403) when calling the HeadObject operation: Forbidden
20:34:43 upload failed: ./.command.log to s3://samt-test/79/61d0a4b2e598b162f6f846a055ba03/.command.log An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

I set aws access /secret key in nextflow.config file but also tried  sending as an env variable to the docker environment with runOptions command (also failed)

Nextflow is executed on my machine with aws cli installed (can ls bucket from cmd line)

any help appreciated!

[Paolo Di Tommaso]

The "Access Denied" is quite clear, there's a problem in the permissions when the job tries to write to the bucket .

Make sure the instance role  specified in the compute environment setting has S3 full access permissions. 

Hope it helps. 

p 

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Sam Tischfield]

thanks - i had thought the compute environment received permissions from nextflow so I had overlooked that - works now, thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.