Issues with running on AWS

[Jagga Soorma]

Hello All,

I am new to Cumulus and this group.  We are trying to run a workflow in AWS using AWS Batch and running into an issue with our jobs.  It keeps failing with the following messages:

--

[2021-08-30 16:49:24,21] [info] WorkflowManagerActor: Workflow 08c9a4c4-e9be-4922-989d-60c9fa2e05ba failed (during ExecutingWorkflowState): cromwell.engine.io.IoAttempts$EnhancedCromwellIoException: [Attempted 1 time(s)] - IOException: Could not read from s3://xxx/cromwell-execution/wf_hello/08c9a4c4-e9be-4922-989d-60c9fa2e05ba/call-hello/hello-rc.txt: s3://s3.amazonaws.com/xxx/cromwell-execution/wf_hello/08c9a4c4-e9be-4922-989d-60c9fa2e05ba/call-hello/hello-rc.txt

--

In cloudwatch I see:

--

/bin/bash: /var/scratch/fetch_and_run.sh: Is a directory

--

Here is the sample wdl file:

--

task hello {

  String addressee = "Cromwell"

  command {

    echo "Hello ${addressee}! Welcome to Cromwell . . . on AWS!"

  }

  output {

    String message = read_string(stdout())

  }

  runtime {

    docker: "quay.io/cumulus/config:0.2"

  }

}

workflow wf_hello {

  call hello

  output { hello.message }

}

--

Can you please help me with why this might be failing? 

Thanks in advance for your assistance!

[Yiming Yang]

Hello Jagga,

I think this is more related to configuring Cromwell for running on AWS. But as I also encountered the similar issue recently on AWS Batch, I'll try to provide information hopefully useful for you.

Did you set up your environment following the Genomics Workflows on AWS tutorial (https://docs.opendata.aws/genomics-workflows/index.html), or https://aws-genomics-workflows.s3.amazonaws.com/Installing+the+Genomics+Workflow+Core+and+Cromwell.pdf ? These two documents should follow the same architecture.

If so, you may want to ask the developer directly at https://github.com/aws-samples/aws-genomics-workflows.

For my own case, since my company doesn't allow public AMIs, I have to use the ECS optimized AMI that the company builds instead, and therefore cannot use the exact CloudFormation to build AWS Batch. I'm still struggling with the set up, and will let you know once I make any progress.

If you like, you may subscribe to this Google group for further discussion.

Sincerely,

Yiming

--
You received this message because you are subscribed to the Google Groups "Cumulus Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cumulus-suppo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cumulus-support/3f2be131-6165-42ab-940e-fcb647f11b28n%40googlegroups.com.

[Yiming Yang]

Alternatively, Caper's tutorial on AWS set up (https://github.com/ENCODE-DCC/caper/blob/master/scripts/aws_caper_server/README.md) worked on AWS with my personal account. However, its configuration requires almost full permissions on your AWS environment, including building up VPC, and you'll have to use their Caper package, which is a Python wrapper for Cromwell, instead of directly using Cromwell.

Sincerely,

Yiming