No write authority when docker image involved

[zhao...@gmail.com]

I tried run my nf with docker container, it throws an exception that it can not created files with "permission denied" .  Here is my example command and error message 

test.nf 

params.reads = "*_{1,2}.clean.fq.gz"

Channel
        .fromFilePairs( params.reads, size: 2 )
        .ifEmpty { error "Cannot find any reads matching: ${params.reads}" }
        .set { read_files }

hisat2_index = Channel.fromPath("/data/database/hg38/hisatIndex/grch38_snp_tran/genome_snp_tran*")
process test{
    maxForks 1
    publishDir 'result/', mode: "move"

    input:
    file hisat2_id from hisat2_index.collect()
    set val(name), file(pair) from read_files

    output:
    file "*.sam" into samout


    shell:
    file_tag_new=name

    index_base = hisat2_id[0].toString() - ~/.\d.ht2/
    '''
    
    hisat2  -p 10 --dta  -x !{index_base} -1 !{pair[0]}  -2 !{pair[1]}  -S !{file_tag_new}.sam
 
   '''
}

config file 

docker {
    enabled = true
}
process{
    container = 'bioinformatist/lncpipe:release-add-gffcompare'
}

and my error message 

[zq@S1 /data1/zq/nextflow]$ nextflow run test.nf 

N E X T F L O W  ~  version 0.26.0

Launching `test.nf` [friendly_cray] - revision: c797677305

[warm up] executor > local

[34/6c3a5d] Submitted process > test (1)

ERROR ~ Error executing process > 'test (1)'

Caused by:

  Process `test (1)` terminated with an error exit status (1)

Command executed:

  hisat2  -p 10 --dta  -x genome_snp_tran -1 N1177_1.clean.fq.gz  -2 N1177_2.clean.fq.gz  -S N1177.sam

  #touch N1177.sam

Command exit status:

  1

Command output:

  (empty)

Command error:

  Error: Could not open alignment output file N1177.sam

  Error: Encountered internal HISAT2 exception (#1)

  Command: /opt/hisat2-2.1.0/hisat2-align-s --wrapper basic-0 -p 10 --dta -x genome_snp_tran -S N1177.sam -1 /tmp/7.inpipe1 -2 /tmp/7.inpipe2 

  (ERR): hisat2-align exited with value 1

Work dir:

  /data1/zhaoqi/nextflow/work/34/6c3a5d16031fb476cc7531f88ca188

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

any suggestions ? or is there anything wrong with my containers ?

thanks ?

[Paolo Di Tommaso]

It looks a permission problem with the tool in the container. This may happen if the docker image has been created with a custom user. 

Try to add the following option in your nextflow.config file 

docker.runOptions = '-u $(id -u):$(id -g)'

Hope it helps. 

p

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[zhao...@gmail.com]

Hi Paolo, thanks for your quick reply. I have checked the permission of tools in container,  and I am sure it was created with root authority. The details information of my container can be found as flowing link 

https://hub.docker.com/r/bioinformatist/lncpipe/tags/

On the other hand, I also tried the modification of nextflow.config file with code you suggest and sudo options

docker.sudo = true 

but the error message still exist, can you provide an example of docker usage in nextflow command which involved both read and write files from host?

在 2017年11月15日星期三 UTC+8下午2:55:35，Paolo Di Tommaso写道：

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.

[Paolo Di Tommaso]

Hard to say. What you can do is to debug the failing task, changing in the task work directory and execute the docker container in interactive manner (the docker command is the `.command.run` script) and then run `.command.sh` to verify why it's not working. 

p 

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.