Pipeline notifier 1.6.3 with tolist

[Kate Alpert]

Dear all,

After upgrading to 1.6.3, pipelines are erroring out on the notification step with a "tolist".  See error below.  The file exists and contains one valid email address.  Any thoughts?

Thanks,

Kate

org.nrg.pipeline.exception.PipelineEngineException:  Execution of $PIPELINE_HOME/catalog/notifications/Notifier -user $TOKEN -password $TOKEN_SECRET -from valid_email\@northwestern.edu -tolist /valid/path/to/email_file_list -cc valid_email\@northwestern.edu -subject NUNDA update: Pipeline complete -body content -host smtp.host.com 

at Step[id= NotifyA:] was unsuccessful 

Personnel notification email could not be sent class java.lang.ArrayStoreExceptionnull null

        at org.nrg.pipeline.manager.ExecutionManager.execute(ExecutionManager.java:121)

        at org.nrg.pipeline.task.StepManager.execute(StepManager.java:166)

        at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:213)

        at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:193)

        at org.nrg.pipeline.client.XNATPipelineLauncher.launch(XNATPipelineLauncher.java:70)

        at org.nrg.pipeline.client.XNATPipelineLauncher.run(XNATPipelineLauncher.java:290)

        at org.nrg.pipeline.client.XNATPipelineLauncher.main(XNATPipelineLauncher.java:254)

----

And in the *.err file:

Failed sending email. Exception: null

[Mohana Ramaratnam]

Can you send me the notify section from the pipeline XML. 

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

[Kate Alpert]

Hi Mohana,

Please see below.

Thanks,
Kate 

<pip:ResolvedStep id="NotifyA:" status="Failed" continueOnFailure="false" precondition="true" description="Notify" preconditionType="relaxed">

    <pip:resolvedResource prefix="pip">

      <pip:name>Notifier</pip:name>

      <pip:location>$PIPELINE_HOME/catalog/notifications</pip:location>

      <pip:type>Human</pip:type>

      <pip:description/>

      <pip:input>

        <pip:argument id="user">

          <pip:name>user</pip:name>

          <pip:value>TOKEN</pip:value>

          <pip:description>Mail service user</pip:description>

        </pip:argument>

        <pip:argument id="password">

          <pip:name>password</pip:name>

          <pip:value>SECRET</pip:value>

          <pip:description>Mail service password</pip:description>

        </pip:argument>

        <pip:argument id="from">

          <pip:name>from</pip:name>

          <pip:value>mye...@northwestern.edu</pip:value>

          <pip:description>Email from</pip:description>

        </pip:argument>

        <pip:argument id="tolist">

          <pip:name>tolist</pip:name>

          <pip:value>/valid/path/to/email_list</pip:value>

          <pip:description>Emails file path. Emails are to be separated by , or newline</pip:description>

        </pip:argument>

        <pip:argument id="cc">

          <pip:name>cc</pip:name>

          <pip:value>mye...@northwestern.edu</pip:value>

          <pip:description>CC Email to</pip:description>

        </pip:argument>

        <pip:argument id="subject">

          <pip:name>subject</pip:name>

          <pip:value>XNAT update: Pipeline complete</pip:value>

          <pip:description>Email subject</pip:description>

        </pip:argument>

        <pip:argument id="body">

          <pip:name>body</pip:name>

          <pip:value><![CDATA[Body]]></pip:value>

          <pip:description>Email body</pip:description>

        </pip:argument>

        <pip:argument id="host">

          <pip:name>host</pip:name>

          <pip:value>smtp.host</pip:value>

          <pip:description>SMTP host</pip:description>

        </pip:argument>

      </pip:input>

    </pip:resolvedResource>

  </pip:ResolvedStep>

[Kate Alpert]

Hi Mohana, 

Any ideas?

Thanks,

Kate

[Kate Alpert]

Hi Mohana,

I'm sorry to keep bothering you, it's just that I'm not sure how to go about debugging issues with the Notifier myself.  Perhaps you could point me in the right direction?

Thanks,

Kate

[Mohana Ramaratnam]

Hi Kate

Try adding a "to" argument (maybe to site admin) also. If the notifier works, its a bug

PS: I am on vacation and I will try to look into this over the weekend.

Mohana

[Kate Alpert]

Hi Mohana,

Adding a "to" argument with the "tolist" did not work.  However, the notifier works with just a "to" (no "tolist").

Thanks for looking into it, I'll look forward to hearing from you next week.

Kate

[Kate Alpert]

Mohana, 

My mistake -- adding a "to" argument did indeed fix the issue. 

I'll add this for now. 

Thanks,

Kate

[Kate Alpert]

Hi Mohana,

Adding a "to" argument works, but it only sends an email to that person -- not the emails in the list.  Is there a fix or work-around for this?

Thanks,

Kate

[Mohana Ramaratnam]

Hi Kate,

I tested your case with:

a) Created an email list file with just one email. (the email id must be that of a registered user of the XNAT host)

b) Created a test pipeline (attached TestNotify.xml)

c) Ran the command with:

$PIPELINE_HOME/bin/XnatPipelineLauncher -pipeline FULL_PATH_TO_TestNotify.xml -project PROJECT_ID_HERE -id XNAT_ID_HERE -dataType xnat:mrSessionData -host XNAT_HOST_URL_HERE  -label SESSION_ID_HERE -u XNAT_USER -pwd XNAT_PWD -supressNotification -notify MY_EMAIL_HERE -parameter adminemail=MY_EMAIL_HERE -parameter useremail=MY_EMAIL_HERE -parameter mailhost=SMTP_MAIL_HOST_HERE -parameter userfullname=Mohana -parameter builddir=PATH_TO_BUILD_TEST -parameter email_list_file=PATH_TO_email.lst 

I got an email as expected. 

Could you please test this. 

[Kate Alpert]

Hi Mohana,

I get the following error:

org.nrg.pipeline.exception.PipelineEngineException:  Execution of /var/lib/nunda_pipeline/catalog/notifications/Notifier -user USER -password PASSWD -from me\@email.com -tolist /home/k8/Desktop/emails.lst -subject Test update: TEST EMAIL available -body The body is blank -host smtp.host.edu at Step[id= 6:] was unsuccessful Personnel notification email could not be sent class org.apache.commons.mail.EmailExceptionAt least one receiver address required At least one receiver address required

        at org.nrg.pipeline.manager.ExecutionManager.execute(ExecutionManager.java:121)
        at org.nrg.pipeline.task.StepManager.execute(StepManager.java:166)
        at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:213)
        at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:193)
        at org.nrg.pipeline.client.XNATPipelineLauncher.launch(XNATPipelineLauncher.java:70)
        at org.nrg.pipeline.client.XNATPipelineLauncher.run(XNATPipelineLauncher.java:290)
        at org.nrg.pipeline.client.XNATPipelineLauncher.main(XNATPipelineLauncher.java:254)

I tried this with a list containing just one email, and I confirmed that it is an active user's email.

When I add the "to" argument, the pipeline runs, but the email is only sent to the address listed in the "to" (the "tolist" is ignored).

This is pipeline 1.6.3.

Thanks,

Kate

[Rick Herrick]

Hey Kate,

In the course of dealing with another issue in the pipeline engine, we came across something that MAY be causing your issue. Would it be possible for you to pull the latest version of the pipeline engine from Bitbucket?

hg clone https://bitbucket.org/nrg/pipeline_1_6dev

cd pipeline_1_6dev

./setup.sh http://your.url/xnat mail.your.server.com yo...@email.com

Point your XNAT to the new pipeline folder and then try running your pipeline.

The issues we found relate to trailing slashes in URLs and overwriting REST call outputs. I *think* that REST URLs with XNAT as the root application (e.g. http://your.server/) work OK, while REST URLs with XNAT as a non-root application (e.g. http://your.server/xnat/) fail. I've added a feature in the XnatDataClient to "smooth" the URL paths, removing slashes just from the path (i.e. anything to the right of the server and optional port and to the left of any query string or fragment). Also, retrieving notification addresses can fail if the file that holds the addresses already exists and the overwrite parameter isn't specified (I've added that overwrite parameter to the AutoRun pipeline definition).

These changes have solved a couple of different issues and it might solve yours as well. Let me know what you find.

Kate

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussion+unsubscribe@googlegroups.com.
To post to this group, send email to xnat_discussion@googlegroups.com.

Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussion+unsubscribe@googlegroups.com.
To post to this group, send email to xnat_discussion@googlegroups.com.

[Kate Alpert]

Hi Rick,

That worked for the TestNotify pipeline.  Now, I just need to update all my pipelines to use it.  Does the overwrite flag for XnatDataClient replace "overwrite=true" in the URL?

Thanks,

Kate

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.

[John Flavin]

No, those two things overwrite on different ends, so to speak. Setting "overwrite=true" on the REST path of an upload tells XNAT to overwrite any duplicate data on the server side. The overwrite parameter on XnatDataClient tells it to overwrite a local file if you're downloading one that conflicts with it.

Flavin

[Herrick, Rick]

No, that’s strictly a client-side option: it will overwrite the indicated output file. For the server side (i.e. to overwrite an existing resource within XNAT), you need to add the overwrite=true flag to the URL. Eventually I’m going to spiff XDC up to do that sort of thing (adding overwrite=true, inbody=true, content type request  header, etc.), but today is not that day yet.

 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

314-740-5961

Kate

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Kate Alpert]

Thanks, Rick and Flavin.

One other question -- I'm still using XNAT 1.6.3 (cloned xnat_builder_1_6dev -r 1.6.3), so when I run update.sh, xdat-beans-1.6.3.jar is copied into $PIPELINE_HOME/lib.  However, my newly downloaded pipeline_1_6dev expects xdat-beans-1.6.4-SNAPSHOT.jar.  I renamed xdat-beans-1.6.3.jar -> xdat-beans-1.6.4-SNAPSHOT.jar, and the pipelines seem to be working fine.  But, that's a bit of a hassle after every update (though I suppose I could just add the move to the end of the update script).  I could alternatively edit the pipeline scripts to use xdat-beans-1.6.3.jar.... Any suggestions?

[Herrick, Rick]

Either one of those should work just fine. This sort of inconvenience is the price of using dev tip code, sorry about that :)

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 827-4250

./setup.sh http://your.url/xnatmail.your.server.com yo...@email.com