Anduril wrapper in version 1.2.22

[Foivos Gypas]

Hello

I am using an old version of  anduril (v 1.2.5) and everythng seems to work properly.
I very recently tried to move to the 1.2.22 version but there seems to be a problem with my wrappers.

I attach you here an example of my old wrappers.
What should should I change in order to work in the new anduril version?

When i try to run the component independetly (anduril run-component ...) I get the following error:

[ERROR] Component extract_archive: Uncaught exception during component execution java.lang.NullPointerException
    at fi.helsinki.ltdk.csbl.anduril.core.network.launcher.PythonLauncher.getExtraEnvironment(PythonLauncher.java:64)
    at fi.helsinki.ltdk.csbl.anduril.core.network.launcher.Launcher.launch(Launcher.java:213)
    at fi.helsinki.ltdk.csbl.anduril.core.network.execmode.PrefixExecMode.launch(PrefixExecMode.java:53)
    at fi.helsinki.ltdk.csbl.anduril.core.network.componentInstance.RegularComponentInstance.launch(RegularComponentInstance.java:42)
    at fi.helsinki.ltdk.csbl.anduril.core.engine.ExecutorWorker.run(ExecutorWorker.java:53)

Thank you in advance for your help

Foivos

[Ville Rantanen]

Hi, 

The Python API has changed. The reason why you get Java errors is not quite clear to me,  did you use the binary release version? if not, are you sure you have compiled the anduril.jar ? (type ant in the ANDURIL_HOME)

The python code should look like this nowadays:

#!/usr/bin/env python


import anduril
from anduril.args import *
import sys, os
import subprocess




### Script to be executed from the anduril component_skeleton
   
# A. PREREQUISITES
# Get in and output arguments as specified in component.xml
# The inputs and parameters are set by the second import row
#    infile = cf.get_input("infile")
#    infolder = cf.get_input("infolder")
#    outfolder =  cf.get_output("outfolder")
#    extract_archive_dir = cf.get_parameter("extract_archive_dir")
   
# Check for presence of infolder(Anduril output from previous component) or infile (new user input)
if infolder:
    if infile:
        print "infile and infolder given:\nPlease give only one input\n"
        sys.exit(1)


    d = os.listdir(infolder)
    infile = os.path.join(infolder, d[0])
else:
    if not infile:
        print "no infile or infolder given\n"
        sys.exit(1)




# Make output directories
os.mkdir(outfolder)




# B. CALL TO FUNCTIONS/SCRIPTS
# Call actual component script with necessary input parameters
proc = subprocess.Popen("bash %s %s %s" % (extract_archive_dir, outfolder, infile),
                                                    stdout=subprocess.PIPE,
                                                    stderr=subprocess.PIPE,
                                                    shell=True
                        )


# Wait until script has been executed
stdout_value, stderr_value = proc.communicate()
print stdout_value
print stderr_value




# Return error
if proc.poll() > 0:
    print '\tstderr:', repr(stderr_value.rstrip())
    sys.exit(1)

[Foivos Gypas]

Thank you for your reply

I used this one: http://csbl.fimm.fi/anduril/current/anduril-1.2.22.zip

 Something else must be wrong. Even with the new wrapper I get the same error message.

Should I change something else maybe in in the component.xml file? (I attached it here)

[lauri...@gmail.com]

 I am sorry, I replied directly to your e-mail by accident...

Could you paste the exact command line with "anduril run-component" here? To me it seems the problem is caused by your component not being part of a bundle. Perhaps we lack a unit test covering the way you are using anduril run-component...

The error occurs before your wrapper is even used. An immediate fix would be to include the component in a bundle, or specify the bundle instead of the component directory (which I assume you are doing with -c)

[Foivos Gypas]

Some more details that might help

This is how i was testing the components in the old version.
anduril run-component extract_archive -I infile=test/t1.zip -O outfolder=test_out_folder -c project1/anduril/components (absolute path in the components case)

The components are not part of a bundle.
I have all components in a directory: project1/anduril/components

If i verbose, I get the following error message:

Registering documentation resource dir /path/to/bin/anduril-1.2.22/microarray/functions/RegionOverlap/doc-files
Registering documentation resource dir /path/to/bin/anduril-1.2.22/microarray/functions/ExpressionImport/doc-files
Component instance states at the beginning of execution:
extract_archive: current=NO, active=YES, precount=1
input_infile: current=NO, active=YES, precount=0
ComponentExecutor: gave job for input_infile; unfinished jobs: 1; numWorkers: 0; time: 0.001 s
ComponentExecutor: gave worker for input_infile; numWorkers: 1; time: 0.004 s
Executing input_infile (INPUT)
Worker started: component input_infile, thread Thread[Thread-1,5,main]
Worker finished: component input_infile, thread Thread[Thread-1,5,main]
ComponentExecutor: worker finished for input_infile; unfinished jobs: 0; numWorkers: 0; time: 0.007 s
extract_archive: pre-count decreased to 0
Component input_infile finished with success; READY queue: extract_archive
ComponentExecutor: gave job for extract_archive; unfinished jobs: 1; numWorkers: 0; time: 0.009 s
Clearing directory /path/to/anduril/components/extract_archive/execute/extract_archive
ComponentExecutor: gave worker for extract_archive; numWorkers: 1; time: 0.012 s
Executing extract_archive (extract_archive)
Worker started: component extract_archive, thread Thread[Thread-2,5,main]

[ERROR] Component extract_archive: Uncaught exception during component execution java.lang.NullPointerException
    at fi.helsinki.ltdk.csbl.anduril.core.network.launcher.PythonLauncher.getExtraEnvironment(PythonLauncher.java:64)
    at fi.helsinki.ltdk.csbl.anduril.core.network.launcher.Launcher.launch(Launcher.java:213)
    at fi.helsinki.ltdk.csbl.anduril.core.network.execmode.PrefixExecMode.launch(PrefixExecMode.java:53)
    at fi.helsinki.ltdk.csbl.anduril.core.network.componentInstance.RegularComponentInstance.launch(RegularComponentInstance.java:42)
    at fi.helsinki.ltdk.csbl.anduril.core.engine.ExecutorWorker.run(ExecutorWorker.java:53)

ComponentExecutor: worker finished for extract_archive; unfinished jobs: 0; numWorkers: 0; time: 0.015 s
input_infile.in: post-count decreased to 0
Component extract_archive finished with failure; READY queue is empty.
Component instance states at the end of execution:
extract_archive: current=NO, active=YES, precount=0
input_infile: current=YES, active=YES, precount=0
Done. The following components had errors:
 extract_archive (extract_archive)

[Ville Rantanen]

There might be a bug there.  The Python Launcher expects the component to be a member of a Bundle, and adds a certain folder within the bundle as a library path for python. In this case the bundle is not there at all, and the code fails.  I think this is a fixable bug.  

For you, it may be easier to create a bundle for your components.

A bundle consists of folder "components" and a bundle.xml  file.  see an example in the microarray bundle that accompanies Anduril.

give the bundle location with -b  switch, instead of using -c for component folder. 

 

[Foivos Gypas]

Yes
This was it. It finally worked.
Thank you both of you for your help.

[Ville Rantanen]

the bug was fixed, and should work in the next release

[marko.k...@gmail.com]

Hi,

I changed the launcher so that it does not fail if the bundle is missing but I could not test the code as I do not have any Python components except those in the standard auto bundles. It seems that run-component command does not support --no-auto-bundles option. That's also something we should fix. I am not sure if you've tested the code fix or did you just add a bundle to your component.

Br,
Marko

[lauri...@gmail.com]

I was able to both replicate the bug and test the fix.

[Foivos Gypas]

Hello

I just added the bundle to the component.

Foivos