Error with email attachments
358 views
Skip to first unread message
Steve
Apr 12, 2018, 12:39:44 PM4/12/18
to Nextflow
I keep hitting error when running this workflow on our HPC cluster:
https://github.com/NYU-Molecular-Pathology/demux-nf
All the processes run fine, but every time I run it, I get an error like this:
And the custom email output does not work (line in question here: https://github.com/NYU-Molecular-Pathology/demux-nf/blob/078e3081435fed9c6f446ae838894ae1fbe5de61/main.nf#L273 )
Here is the error from the .nextflow.log file:
I am reluctant to post the entire .nextflow.log since it containers sample ID's which might include PHI, as does the large directory of input data produced by our NextSeq so its not really possible to provide a working example to replicate elsewhere.
The strange part is that the email attachment method I use here works in most every other Nextflow pipeline that I run. But in this particular pipeline, it constantly breaks when trying to attach files to the email.
Here is a full copy of the `main.nf` Nextflow script:
here is the `nextflow.config`:
and here is the command used to start Nextflow:
Any ideas what might be going on?
https://github.com/NYU-Molecular-Pathology/demux-nf
All the processes run fine, but every time I run it, I get an error like this:
ERROR ~ Failed to invoke `workflow.onComplete` event handler
-- Check script 'main.nf' at line: 273 or see '.nextflow.log' file for more details
And the custom email output does not work (line in question here: https://github.com/NYU-Molecular-Pathology/demux-nf/blob/078e3081435fed9c6f446ae838894ae1fbe5de61/main.nf#L273 )
Here is the error from the .nextflow.log file:
...
...
...
Apr-09 13:24:24.590 [Actor Thread 124] DEBUG nextflow.Session - <<< barrier arrive (process: fastqc)
Apr-09 13:24:24.594 [Actor Thread 124] DEBUG nextflow.Session - <<< barrier arrive (process: multiqc)
Apr-09 13:24:24.594 [main] DEBUG nextflow.Session - Session await > all process finished
Apr-09 13:24:24.595 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: sge)
Apr-09 13:24:24.642 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local)
Apr-09 13:24:24.642 [main] DEBUG nextflow.Session - Session await > all barriers passed
Apr-09 13:24:25.144 [main] ERROR nextflow.script.WorkflowMetadata - Failed to invoke `workflow.onComplete` event handler
java.lang.IllegalArgumentException: Invalid attachment argument: [/ifs/data/molecpathlab/production/Demultiplexing/180404_NB501073_0041_AHWJ2YAFXX/demux-nf/work/35/3f466a28b7174d356b70b54a2ba1cb/SampleSheet.csv, /ifs/data/molecpathlab/production/Demultiplexing/180404_NB501073_0041_AHWJ2YAFXX/demux-nf/work/35/3f466a28b7174d356b70b54a2ba1cb/Demultiplex_Stats.htm, /ifs/data/molecpathlab/production/Demultiplexing/180404_NB501073_0041_AHWJ2YAFXX/demux-nf/work/35/3f466a28b7174d356b70b54a2ba1cb/demultiplexing_report.html, /ifs/data/molecpathlab/production/Demultiplexing/180404_NB501073_0041_AHWJ2YAFXX/demux-nf/work/35/3f466a28b7174d356b70b54a2ba1cb/RunParameters.tsv, /ifs/data/molecpathlab/production/Demultiplexing/180404_NB501073_0041_AHWJ2YAFXX/demux-nf/work/35/3f466a28b7174d356b70b54a2ba1cb/RTAComplete.txt] [class java.util.ArrayList]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:77)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
at nextflow.mail.Attachment.<init>(Attachment.groovy:73)
at nextflow.mail.Attachment.<init>(Attachment.groovy:59)
at nextflow.mail.Mail$_attach_closure1.doCall(Mail.groovy:201)
at nextflow.mail.Mail$_attach_closure1.call(Mail.groovy)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:3156)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:3127)
at nextflow.mail.Mail.attach(Mail.groovy:201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
at _nf_script_cf885d43$_run_closure14$_closure42.doCall(_nf_script_cf885d43:273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:430)
at nextflow.mail.Mailer.send(Mailer.groovy:493)
at nextflow.mail.Mailer$send.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at nextflow.Nextflow.sendMail(Nextflow.groovy:381)
at nextflow.Nextflow$sendMail$0.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:206)
at _nf_script_cf885d43$_run_closure14.doCall(_nf_script_cf885d43:270)
at _nf_script_cf885d43$_run_closure14.doCall(_nf_script_cf885d43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:408)
at nextflow.script.WorkflowMetadata$_invokeOnComplete_closure4.doCall(WorkflowMetadata.groovy:352)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:430)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2040)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2025)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2066)
at nextflow.script.WorkflowMetadata.invokeOnComplete(WorkflowMetadata.groovy:350)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:925)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:908)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:76)
at nextflow.script.WorkflowMetadata$_closure3.doCall(WorkflowMetadata.groovy:216)
at nextflow.script.WorkflowMetadata$_closure3.call(WorkflowMetadata.groovy)
at nextflow.Session.cleanUp(Session.groovy:578)
at nextflow.Session.destroy(Session.groovy:550)
at nextflow.script.ScriptRunner.terminate(ScriptRunner.groovy:348)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:167)
at nextflow.cli.CmdRun.run(CmdRun.groovy:223)
at nextflow.cli.Launcher.run(Launcher.groovy:428)
at nextflow.cli.Launcher.main(Launcher.groovy:582)
Apr-09 13:24:25.760 [main] DEBUG nextflow.trace.StatsObserver - Workflow completed > WorkflowStats[succeedCount=134; failedCount=0; ignoredCount=0; cachedCount=0; succeedDuration=1h 5m 44s; failedDuration=0ms; cachedDuration=0ms]
Apr-09 13:24:25.760 [main] DEBUG nextflow.trace.TraceFileObserver - Flow completing -- flushing trace file
Apr-09 13:24:25.767 [main] DEBUG nextflow.trace.ReportObserver - Flow completing -- rendering html report
Apr-09 13:24:27.020 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data:
...
...
...I am reluctant to post the entire .nextflow.log since it containers sample ID's which might include PHI, as does the large directory of input data produced by our NextSeq so its not really possible to provide a working example to replicate elsewhere.
The strange part is that the email attachment method I use here works in most every other Nextflow pipeline that I run. But in this particular pipeline, it constantly breaks when trying to attach files to the email.
Here is a full copy of the `main.nf` Nextflow script:
// params.project = "180131_NB501073_0032_AHT5F3BGX3"
// params.sequencer_dir = "/ifs/data/molecpathlab/quicksilver"
// params.output_dir = "${params.basecalls_dir}/Demultiplexing"
params.run_dir = "${params.sequencer_dir}/${params.project}"
params.basecalls_dir = "${params.run_dir}/Data/Intensities/BaseCalls"
params.output_dir = "${params.production_dir}/${params.project}/output"
params.report_template_dir = "nextseq-report"
log.info "~~~~~~~ Demultiplexing Pipeline ~~~~~~~"
log.info "* Project: ${params.project}"
log.info "* Sequencer dir: ${params.sequencer_dir}"
log.info "* Run dir: ${params.run_dir}"
log.info "* Basecalls dir: ${params.basecalls_dir}"
log.info "* Output dir: ${params.output_dir} "
log.info "* Samplesheet: ${params.samplesheet}"
Channel.fromPath( params.samplesheet ).set { samplesheet_input }
Channel.from( "${params.run_dir}" ).into { run_dir; run_dir2; run_dir3; run_dir4 } // dont stage run dir for safety reasons
Channel.fromPath( params.report_template_dir ).set { report_template_dir }
process validate_run_completion {
tag { "${run_dir}" }
executor "local"
publishDir "${params.output_dir}/", mode: 'copy', overwrite: true
input:
val(run_dir) from run_dir2
output:
file("RTAComplete.txt") into run_RTAComplete_txt
file("RunCompletionStatus.xml") into run_CompletionStatus_xml
file("RunParameters.xml") into run_params_xml
script:
"""
cp ${run_dir}/RTAComplete.txt .
cp ${run_dir}/RunCompletionStatus.xml .
cp ${run_dir}/RunParameters.xml .
"""
}
process copy_samplesheet {
tag { "${samplesheet}" }
executor "local"
publishDir "${params.output_dir}/", mode: 'copy', overwrite: true
input:
file(samplesheet) from samplesheet_input
output:
file("${samplesheet}")
file("SampleSheet.csv") into samplesheet_copy
file("SampleSheet.csv") into samplesheet_copy2
script:
"""
cp "${samplesheet}" SampleSheet.csv
"""
}
process bcl2fastq {
tag { "${run_dir}" }
publishDir "${params.output_dir}/", mode: 'copy', overwrite: true
input:
set file(samplesheet), val(run_dir) from samplesheet_copy.combine(run_dir)
output:
file("Unaligned") into bcl2fastq_output
file("Unaligned/Demultiplex_Stats.htm") into demultiplex_stats_html
file("Unaligned/Demultiplex_Stats.htm") into demultiplex_stats_html2
file("Unaligned/**.fastq.gz") into fastq_output
script:
"""
nthreads="\${NSLOTS:-\${NTHREADS:-2}}"
# 20% of threads for demult as per Illumina manual
demult_threads="\$(( \$nthreads*2/10 ))"
# at least 2 threads
[ "\${demult_threads}" -lt "2" ] && demult_threads=2
echo "[bcl2fastq]: \$(which bcl2fastq) running with \${nthreads} threads and \${demult_threads} demultiplexing threads"
bcl2fastq --version
bcl2fastq \
--min-log-level WARNING \
--fastq-compression-level 8 \
--loading-threads 2 \
--demultiplexing-threads \${demult_threads:-2} \
--processing-threads \${nthreads:-2} \
--writing-threads 2 \
--sample-sheet ${samplesheet} \
--runfolder-dir ${run_dir} \
--output-dir ./Unaligned \
${params.bcl2fastq_params}
# create Demultiplex_Stats.htm
cat Unaligned/Reports/html/*/all/all/all/laneBarcode.html | grep -v "href=" > Unaligned/Demultiplex_Stats.htm
"""
}
// filter out 'Undetermined' fastq files
fastq_output.flatMap()
.map{ item ->
if (! "${item}".contains("Undetermined_")){
return item
}
}
.set{ fastq_filtered }
process fastqc {
tag { "${fastq}" }
publishDir "${params.output_dir}/fastqc", mode: 'copy', overwrite: true
input:
file(fastq) from fastq_filtered
output:
file(output_html)
file(output_zip)
val(fastq) into fastqc_fastqs
script:
output_html = "${fastq}".replaceFirst(/.fastq.gz$/, "_fastqc.html")
output_zip = "${fastq}".replaceFirst(/.fastq.gz$/, "_fastqc.zip")
"""
fastqc -o . "${fastq}"
"""
}
// currently broken on phoenix ??
process multiqc {
tag { "${output_dir}" }
publishDir "${params.output_dir}/multiqc", mode: 'copy', overwrite: true
executor "local"
input:
val(items) from bcl2fastq_output.mix(fastqc_fastqs)
.collect() // force it to wait for all steps to finish
file(output_dir) from Channel.fromPath("${params.output_dir}")
output:
file "multiqc_report.html" into multiqc_report_html
file "multiqc_data"
when:
params.multiqc_disable == false
script:
"""
echo \$PATH
echo \${PYTHONPATH:-"not set"}
echo \${PYTHONHOME:-"not set"}
module list
python --version
which python
which multiqc
multiqc --version
multiqc "${output_dir}"
"""
}
process demultiplexing_report {
tag { "${template_dir}" }
executor "local"
publishDir "${params.output_dir}/demultiplexing-report", mode: 'copy', overwrite: true
stageInMode "copy"
input:
set file(template_dir), file(demultiplex_stats) from report_template_dir.combine(demultiplex_stats_html2)
output:
file("demultiplexing_report.html") into demultiplexing_report_html
script:
"""
mv ${demultiplex_stats} "${template_dir}/"
compile_Rmd.R "${template_dir}/demultiplexing_report.Rmd"
mv "${template_dir}/demultiplexing_report.html" .
"""
}
process convert_run_params{
tag { "${run_params_xml_file}" }
publishDir "${params.output_dir}/", mode: 'copy', overwrite: true
executor "local"
input:
file(run_params_xml_file) from run_params_xml
output:
file("RunParameters.tsv") into run_params_tsv
script:
"""
RunParametersXML2tsv.py
"""
}
process collect_email_attachments {
tag { "${attachments}" }
publishDir "${params.output_dir}/email_attachments", mode: 'copy', overwrite: true
stageInMode "copy"
executor "local"
echo true
input:
file(attachments: "*") from samplesheet_copy2.concat(demultiplex_stats_html, demultiplexing_report_html, run_params_tsv, run_RTAComplete_txt ).collect()
output:
file(attachments) into email_attachments
script:
"""
echo "[collect_email_attachments] files to be attached: ${attachments}"
"""
}
// email_attachments.collectFile(name: 'email_attachments.txt', storeDir: ".", newLine: true)
// email_attachments.subscribe{println "${it}"}
// def attachments = samplesheet_copy2.concat(demultiplex_stats_html, demultiplexing_report_html, run_params_tsv, run_RTAComplete_txt ).toList().getVal()
// ~~~~~~~~~~~~~~~ PIPELINE COMPLETION EVENTS ~~~~~~~~~~~~~~~~~~~ //
def attachments = email_attachments.toList().getVal()
workflow.onComplete {
def status = "NA"
if(workflow.success) {
status = "SUCCESS"
} else {
status = "FAILED"
}
def msg = """
Pipeline execution summary
---------------------------
Success : ${workflow.success}
exit status : ${workflow.exitStatus}
Launch time : ${workflow.start.format('dd-MMM-yyyy HH:mm:ss')}
Ending time : ${workflow.complete.format('dd-MMM-yyyy HH:mm:ss')} (duration: ${workflow.duration})
Launch directory : ${workflow.launchDir}
Work directory : ${workflow.workDir.toUriString()}
Project directory : ${workflow.projectDir}
Run directory : ${params.run_dir}
Script name : ${workflow.scriptName ?: '-'}
Script ID : ${workflow.scriptId ?: '-'}
Workflow session : ${workflow.sessionId}
Workflow repo : ${workflow.repository ?: '-' }
Workflow revision : ${workflow.repository ? "$workflow.revision ($workflow.commitId)" : '-'}
Workflow profile : ${workflow.profile ?: '-'}
Workflow container: ${workflow.container ?: '-'}
container engine : ${workflow.containerEngine?:'-'}
Nextflow run name : ${workflow.runName}
Nextflow version : ${workflow.nextflow.version}, build ${workflow.nextflow.build} (${workflow.nextflow.timestamp})
The command used to launch the workflow was as follows:
${workflow.commandLine}
--
This email was sent by Nextflow
cite doi:10.1038/nbt.3820
http://nextflow.io
"""
.stripIndent()
// Total CPU-Hours : ${workflow.stats.getComputeTimeString() ?: '-'}
if(params.pipeline_email) {
sendMail {
to "${params.email_to}"
from "${params.email_from}"
attach attachments
subject "[${params.workflow_label}] ${status}: ${params.project}"
body
"""
${msg}
"""
.stripIndent()
}
}
}
here is the `nextflow.config`:
// params.output_dir = "output"
params.workflow_label = "Demultiplexing"
manifest {
author = 'Stephen Kelly'
homePage = 'https://github.com/NYU-Molecular-Pathology/demux-nf'
description = 'Illumina bcl2fastq demultiplexing pipeline'
mainScript = 'main.nf'
}
report {
enabled = true
file = "nextflow-report.html"
}
trace {
enabled = true
fields = "task_id,hash,native_id,process,tag,name,status,exit,module,container,cpus,time,disk,memory,attempt,submit,start,complete,duration,realtime,queue,%cpu,%mem,rss,vmem,peak_rss,peak_vmem,rchar,wchar,syscr,syscw,read_bytes,write_bytes"
file = "trace.txt"
raw = true
}
timeline {
enabled = true
file = "timeline-report.html"
}
username = System.getProperty("user.name")
params.email_host = "nyumc.org"
params.email_from = "${username}@${params.email_host}"
params.email_to = "${username}@${params.email_host}"
params.pipeline_email = true
notification {
enabled = true
to = "${params.email_to}"
from = "${params.email_from}"
}
params.beforeScript_str = 'printf "USER:\${USER:-none} JOB_ID:\${JOB_ID:-none} JOB_NAME:\${JOB_NAME:-none} HOSTNAME:\${HOSTNAME:-none} PWD:\$PWD\n"; TIMESTART=\$(date +%s)'
process.beforeScript = "${params.beforeScript_str}"
// locations to run the pipeline
profiles {
phoenix { // for NYULMC phoenix HPC
// process.queue = 'clinical.q' // experimental queue
process.queue = 'all.q'
params.sequencer_dir = "/ifs/data/molecpathlab/quicksilver"
params.production_dir = "/ifs/data/molecpathlab/production/Demultiplexing"
params.multiqc_makefile = new File("bin/multiqc.makefile").getCanonicalPath()
params.multiqc_requirements_txt = new File("bin/multiqc.requirements.txt").getCanonicalPath()
// params.multiqc_setup_cmd = "make -f ${params.multiqc_makefile} setup REQ_FILE=${params.multiqc_requirements_txt} ; export PS=\${PS:-''} ; export PS1=\${PS1:-''} ; export PYTHONPATH= ; source multiqc-venv/bin/activate"
params.multiqc_setup_cmd = 'module unload python'
params.multiqc_disable = true
process.$bcl2fastq.module = "bcl2fastq/2.17.1"
process.$bcl2fastq.executor = "sge"
process.$bcl2fastq.clusterOptions = "-pe threaded 4-16"
process.$multiqc.module = 'metaseq/0.5.5.4'
process.$multiqc.beforeScript = "${process.beforeScript} ; ${params.multiqc_setup_cmd}"
process.$fastqc.module = "fastqc/0.11.7"
process.$fastqc.executor = "sge"
process.$fastqc.clusterOptions = "-l mem_free=8G -l mem_token=8G"
process.$demultiplexing_report.module = "pandoc/1.13.1"
}
}
// pipeline run settings
profiles {
NGS580 {
// phoenix location for auto-samplesheet detection:
params.samplesheet = "/ifs/data/molecpathlab/quicksilver/to_be_demultiplexed/NGS580/${params.project}-SampleSheet.csv"
params.bcl2fastq_params = "--ignore-missing-bcls --ignore-missing-filter --ignore-missing-positions --ignore-missing-controls --auto-set-to-zero-barcode-mismatches --find-adapters-with-sliding-window --adapter-stringency 0.9 --mask-short-adapter-reads 35 --minimum-trimmed-read-length 35"
}
Archer {
// phoenix location for auto-samplesheet detection:
params.samplesheet = "/ifs/data/molecpathlab/quicksilver/to_be_demultiplexed/Archer/${params.project}-SampleSheet.csv"
params.bcl2fastq_params = "--no-lane-splitting"
}
}
and here is the command used to start Nextflow:
$ ./nextflow run main.nf -profile phoenix,NGS580 --project 180404_NB501073_0041_AHWJ2YAFXX --samplesheet 180404_NB501073_0041_AHWJ2YAFXX-SampleSheet.csv
N E X T F L O W ~ version 0.28.0
Launching `main.nf` [pedantic_minsky] - revision: 8e5fecb265
~~~~~~~ Demultiplexing Pipeline ~~~~~~~
* Project: 180404_NB501073_0041_AHWJ2YAFXX
* Sequencer dir: /ifs/data/molecpathlab/quicksilver
* Run dir: /ifs/data/molecpathlab/quicksilver/180404_NB501073_0041_AHWJ2YAFXX
* Basecalls dir: /ifs/data/molecpathlab/quicksilver/180404_NB501073_0041_AHWJ2YAFXX/Data/Intensities/BaseCalls
* Output dir: /ifs/data/molecpathlab/production/Demultiplexing/180404_NB501073_0041_AHWJ2YAFXX/output
* Samplesheet: 180404_NB501073_0041_AHWJ2YAFXX-SampleSheet.csv
[warm up] executor > local
[warm up] executor > sge
[d1/9637c4] Submitted process > copy_samplesheet (180404_NB501073_0041_AHWJ2YAFXX-SampleSheet.csv)
[0f/1e44f4] Submitted process > validate_run_completion (/ifs/data/molecpathlab/quicksilver/180404_NB501073_0041_AHWJ2YAFXX)
[05/a76a66] Submitted process > bcl2fastq (/ifs/data/molecpathlab/quicksilver/180404_NB501073_0041_AHWJ2YAFXX)
[75/7892e5] Submitted process > convert_run_params (RunParameters.xml)
[1d/f8c028] Submitted process > demultiplexing_report (nextseq-report)
...
...
...
ERROR ~ Failed to invoke `workflow.onComplete` event handler
-- Check script 'main.nf' at line: 273 or see '.nextflow.log' file for more detailsAny ideas what might be going on?
Paolo Di Tommaso
Apr 13, 2018, 8:56:08 AM4/13/18
to nextflow
Please report an issue on GH including a test case to replicate the issue.
p
at org.<span style="color: #000;" class=
Steve
Apr 25, 2018, 8:35:38 PM4/25/18
to Nextflow
I am not sure how to provide a test case, because this pipeline requires an entire NextSeq sequencer output directory to run.
Paolo Di Tommaso
Apr 26, 2018, 7:50:32 AM4/26/18
to nextflow
My suggestion is to try to replicate it in a separate snippet.
Otherwise I have little chances to debug it.
p
--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.
Steve
May 30, 2018, 12:35:16 PM5/30/18
to Nextflow
As per my other post here
I am starting to get similar issues cropping up across my pipelines. 'Broken pipe' errors when trying to send the email in a separate follow-up workflow, and various errors from the main workflow. Here is a new one, claiming that 'sendmail' is not availble even though its installed on the system:
[2b/c7b885] Submitted process > custom_sample_report (HapMap-B17-1267)
[c5/d2c852] Submitted process > custom_sample_report (NTC-H2O)
[d3/beec19] Submitted process > custom_sample_report (SeraCare-1to1-Positive)
[8f/1a6f2c] Submitted process > custom_analysis_report (NGS580-nf-demo-output-small.2018-05-29_17-37-12.analysis_report.html)
ERROR ~ Failed to invoke `workflow.onComplete` event handler
-- Check script 'main.nf' at line: 1904 or see '.nextflow.log' file for more details
WARN: Failed to deliver notification email -- See the log file for details
May-29 19:29:32.508 [Actor Thread 270] DEBUG nextflow.Session - <<< barrier arrive (process: custom_analysis_report)
May-29 19:29:32.509 [main] DEBUG nextflow.Session - Session await > all process finished
May-29 19:29:32.836 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local)
May-29 19:29:36.845 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: sge)
May-29 19:29:36.845 [main] DEBUG nextflow.Session - Session await > all barriers passed
May-29 19:29:48.708 [main] ERROR nextflow.script.WorkflowMetadata - Failed to invoke `workflow.onComplete` event handler
java.lang.IllegalArgumentException: Cannot send email message -- Make sure you have installed `sendmail` or `mail` program or configure a mail SMTP server in the nextflow config file
at nextflow.mail.Mailer.send(Mailer.groovy:446)
at nextflow.mail.Mailer.send(Mailer.groovy:496)
at nextflow.mail.Mailer$send.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
at nextflow.Nextflow.sendMail(Nextflow.groovy:385)
at nextflow.Nextflow$sendMail$2.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:197)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:209)
at _nf_script_6f41d842$_run_closure103.doCall(_nf_script_6f41d842:1904)
at _nf_script_6f41d842$_run_closure103.doCall(_nf_script_6f41d842)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at groovy.lang.Closure.call(Closure.java:418)
at groovy.lang.Closure.call(Closure.java:412)
at nextflow.script.WorkflowMetadata$_invokeOnComplete_closure4.doCall(WorkflowMetadata.groovy:352)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at groovy.lang.Closure.call(Closure.java:418)
at groovy.lang.Closure.call(Closure.java:434)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2125)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2110)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2151)
at nextflow.script.WorkflowMetadata.invokeOnComplete(WorkflowMetadata.groovy:350)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:947)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:930)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:92)
at nextflow.script.WorkflowMetadata$_closure3.doCall(WorkflowMetadata.groovy:216)
at nextflow.script.WorkflowMetadata$_closure3.call(WorkflowMetadata.groovy)
at nextflow.Session.shutdown0(Session.groovy:580)
at nextflow.Session.destroy(Session.groovy:552)
at nextflow.script.ScriptRunner.terminate(ScriptRunner.groovy:348)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:167)
at nextflow.cli.CmdRun.run(CmdRun.groovy:223)
at nextflow.cli.Launcher.run(Launcher.groovy:428)
at nextflow.cli.Launcher.main(Launcher.groovy:582)
May-29 19:29:56.966 [main] WARN nextflow.script.WorkflowMetadata - Failed to deliver notification email -- See the log file for details
java.lang.IllegalArgumentException: Cannot send email message -- Make sure you have installed `sendmail` or `mail` program or configure a mail SMTP server in the nextflow config file
at nextflow.mail.Mailer.send(Mailer.groovy:446)
at nextflow.script.WorkflowNotifier.sendNotification(WorkflowNotifier.groovy:75)
at nextflow.script.WorkflowMetadata.safeMailNotification(WorkflowMetadata.groovy:407)
at nextflow.script.WorkflowMetadata.invokeOnComplete(WorkflowMetadata.groovy:360)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:947)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:930)
at org.codehaus.groovy.runtime.InvokerHelper.invokeMethodSafe(InvokerHelper.java:92)
at nextflow.script.WorkflowMetadata$_closure3.doCall(WorkflowMetadata.groovy:216)
at nextflow.script.WorkflowMetadata$_closure3.call(WorkflowMetadata.groovy)
at nextflow.Session.shutdown0(Session.groovy:580)
at nextflow.Session.destroy(Session.groovy:552)
at nextflow.script.ScriptRunner.terminate(ScriptRunner.groovy:348)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:167)
at nextflow.cli.CmdRun.run(CmdRun.groovy:223)
at nextflow.cli.Launcher.run(Launcher.groovy:428)
at nextflow.cli.Launcher.main(Launcher.groovy:582)
May-29 19:29:57.820 [main] DEBUG nextflow.trace.StatsObserver - Workflow completed > WorkflowStats[succeedCount=89; failedCount=0; ignoredCount=0; cachedCount=62; succeedDuration=16h 29m 48s; failedDuration=0ms; cachedDuration=21m 26s]
May-29 19:29:57.820 [main] DEBUG nextflow.trace.TraceFileObserver - Flow completing -- flushing trace file
May-29 19:29:58.601 [main] DEBUG nextflow.trace.ReportObserver - Flow completing -- rendering html report
May-29 19:30:01.489 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data:
...
...
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.
Paolo Di Tommaso
May 31, 2018, 2:25:04 AM5/31/18
to nextflow
It looks that `sendmail` is not available nor a mail server is configured in the nextflow.config file.
p
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Steve
May 31, 2018, 9:28:31 AM5/31/18
to Nextflow
OK, I suspect that may be the result of some HPC cluster nodes not having installed. I'll check on that.
Any idea what might be causing the 'Broken Pipe' error here?
$ cat .nextflow.log
May-30 20:08:47.310 [main] DEBUG nextflow.cli.Launcher - $> ./nextflow run email.nf --samplesheet 180522_NB501073_0046_AH2LNJAFXY.csv --run_dir seq_dir
May-30 20:08:47.462 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 0.29.0
May-30 20:08:47.545 [main] INFO nextflow.cli.CmdRun - Launching `email.nf` [jovial_ptolemy] - revision: 433e3d087e
May-30 20:08:47.586 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: nextflow.config
May-30 20:08:47.588 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /ifs/data/molecpathlab/production/Demultiplexing/180522_NB501073_0046_AH2LNJAFXY/nextflow.config
May-30 20:08:47.620 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
May-30 20:08:48.099 [main] DEBUG nextflow.Session - Session uuid: 688f904a-fd39-4d9c-ba18-2a671738096d
May-30 20:08:48.100 [main] DEBUG nextflow.Session - Run name: jovial_ptolemy
May-30 20:08:48.100 [main] DEBUG nextflow.Session - Executor pool size: 32
May-30 20:08:48.114 [main] DEBUG nextflow.cli.CmdRun -
Version: 0.29.0 build 4803
Modified: 24-04-2018 08:09 UTC (04:09 EDT)
System: Linux 2.6.32-642.11.1.el6.x86_64
Runtime: Groovy 2.4.15 on OpenJDK 64-Bit Server VM 1.8.0_171-b10
Encoding: UTF-8 (ANSI_X3.4-1968)
Process: 13301@phoenix2 [192.168.10.3]
CPUs: 32 - Mem: 126 GB (2.1 GB) - Swap: 244.1 GB (229.6 GB)
May-30 20:08:48.182 [main] DEBUG nextflow.Session - Work-dir: /ifs/data/molecpathlab/production/Demultiplexing/180522_NB501073_0046_AH2LNJAFXY/work [nfs]
May-30 20:08:48.580 [main] DEBUG nextflow.Session - Session start invoked
May-30 20:08:48.584 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start
May-30 20:08:48.585 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /ifs/data/molecpathlab/production/Demultiplexing/180522_NB501073_0046_AH2LNJAFXY/trace.txt
May-30 20:08:48.600 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing
May-30 20:08:49.447 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
May-30 20:08:49.465 [main] DEBUG nextflow.Channel - files for syntax: glob; folder: output/email/attachments/; pattern: *; options: null
May-30 20:08:53.189 [main] DEBUG nextflow.Session - Session aborted -- Cause: Broken pipe
May-30 20:08:53.566 [main] ERROR nextflow.cli.Launcher - Broken pipe
java.io.IOException: Broken pipe
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:95)
at com.sun.mail.util.QPEncoderStream.output(QPEncoderStream.java:197)
at com.sun.mail.util.QPEncoderStream.write(QPEncoderStream.java:140)
at com.sun.mail.util.QPEncoderStream.write(QPEncoderStream.java:95)
at javax.activation.DataHandler.writeTo(DataHandler.java:309)
at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1485)
at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:865)
at javax.mail.internet.MimeMultipart.writeTo(MimeMultipart.java:462)
at com.sun.mail.handlers.multipart_mixed.writeTo(multipart_mixed.java:103)
at javax.activation.ObjectDataContentHandler.writeTo(DataHandler.java:889)
at javax.activation.DataHandler.writeTo(DataHandler.java:317)
at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:1485)
at javax.mail.internet.MimeMessage.writeTo(MimeMessage.java:1773)
at javax.mail.internet.MimeMessage.writeTo(MimeMessage.java:1749)
at nextflow.mail.Mailer.sendViaSysMail(Mailer.groovy:255)
at nextflow.mail.Mailer.send(Mailer.groovy:431)
at nextflow.mail.Mailer.send(Mailer.groovy:496)
at nextflow.mail.Mailer$send.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
at nextflow.Nextflow.sendMail(Nextflow.groovy:385)
at nextflow.Nextflow$sendMail.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:197)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:209)
at _nf_script_16082334.run(_nf_script_16082334:10)
at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:341)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:165)
at nextflow.cli.CmdRun.run(CmdRun.groovy:223)
at nextflow.cli.Launcher.run(Launcher.groovy:428)
at nextflow.cli.Launcher.main(Launcher.groovy:582)
pipeline script:
$ cat email.nf
Channel.fromPath("${params.output_dir}/email/attachments/*").set { email_attachments_channel }
String subject_line = new File("${params.output_dir}/email/subject.txt").text
def body = new File("${params.output_dir}/email/body.txt").text
def attachments = email_attachments_channel.toList().getVal()
// pause a moment before sending the email; 3s
sleep(3000)
sendMail {
from "${params.email_to}"
to "${params.email_from}"
attach attachments
subject subject_line
"""
${body}
""".stripIndent()
}
Paolo Di Tommaso
May 31, 2018, 10:12:10 AM5/31/18
to nextflow
Please open an issue including a script and data to replicate the error.
p
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages