Linking COPASI and Cytoscape

[Hikmet Emre Kaya]

Hello Everyone!

For my PhD project, I need to link COPASI and Cytoscape.

Cytoscape is an open source bioinformatics software platform for visualizing molecular interaction networks and integrating with gene expression profiles and other state data.

It has an app store where you can install apps for additional features, but there is no app for what I want to do, which is to run reaction kinetics simulations on the networks, particularly glycosylation networks.

So, I want to make COPASI available and compatible with Cytoscape.

Because Cytoscape is a Java application, I downloaded the java bindings for COPASI and decompiled the .class files to .java files. But, I am not sure how to proceed from here. Using Eclipse, I created a new maven project in the Cytoscape-api repository and added the .java files for COPASI.

I think I need to update the pom.xml file to add the COPASI dependencies, but I am not sure how to do that.

Sorry if the question sounds too vague, I guess I need an overall idea of how to best go about this.

Thanks,

Emre

[Frank Bergmann]

Hello Emre, 

since COPASI is written in C++, the way we provide support for it to Java, is by automatically wrapping the C++ classes via SWIG. SWIG generates the Java-wrapper files (and c++ wrapper file containing the native JNI symbols for the java wrapper files), that are then compiled into the copasi jar file. This jar file, will only work with the native library, that was generated alongside it. If you just wanted to do it for your local instance, you would then include this jar file in your java project, and ensure that the accompanying native library is in the `java.library.path`. from there you can access the api of COPASI as done in these examples: 

https://github.com/copasi/COPASI/tree/develop/copasi/bindings/java/examples

And here is where it becomes a bit more complicated, if you would like the jar file to work on all systems, you also need compilations of the same generated JNI wrapper on the other platforms you would want to support. As for how to include this multi-platform approach in a maven build i am not sure. 

Again the java files, without the native library will not work for you. 

best

Frank

[Hikmet Emre Kaya]

Hi Frank,

Thanks for this detailed explanation.

At the moment, when I copy the .jar file to CytoscapeConfiguration/3/apps/installed, Cytoscape crashes and gives the following error:

Native code library failed to load.                                                                                                                                                      
java.lang.UnsatisfiedLinkError: no CopasiJava in java.library.path: [/home/people/hkaya/Cytoscape_v3.8.2/framework/lib, /home/people/hkaya/Cytoscape_v3.8.2/framework/lib, /usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]

[Frank Bergmann]

Indeed, as i pointed out, the jar file cannot work without the C++ library. So assuming (from your paths provided) that you work on linux x64, i assume you have downloaded this archive: 

https://github.com/copasi/COPASI/releases/download/Build-240/COPASI-4.30.240-Java-Bindings-Linux-64bit.tar.gz

in that archive you find next to the `copasi.jar` file (with the java wrappers for the library, but not the implementation) the native library `libCopasiJava.so`this needs to be copied into a path that is in your `java.library.path` or you need to include its location manually into the java.library.path. 

best

Frank

[Hikmet Emre Kaya]

Hi Frank,

That makes sense; I do work on linux x64.

I will try this and keep you posted about the outcome. Thanks!

[Hikmet Emre Kaya]

Hello Frank,

So, linking the native library to my java.library.path did work, but COPASI still fails to start when I open Cytoscape and gives the following error:

org.cytoscape.app.internal.exception.AppStartupException: Bundle start error
    at org.cytoscape.app.internal.manager.BundleApp.start(BundleApp.java:84) ~[?:?]
    at org.cytoscape.app.internal.manager.AppManager.startApps(AppManager.java:404) ~[?:?]
    at org.cytoscape.app.internal.manager.AppManager.initializeApps(AppManager.java:374) ~[?:?]
    at org.cytoscape.app.internal.manager.AppManager.lambda$attemptInitialization$0(AppManager.java:235) ~[?:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.osgi.framework.BundleException: Activator start error in bundle org.cytoscape.COPASI [123].
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2290) ~[?:?]
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2146) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:984) ~[?:?]
    at org.cytoscape.app.internal.manager.BundleApp.start(BundleApp.java:82) ~[?:?]
    ... 8 more
Caused by: java.lang.UnsatisfiedLinkError: 'void org.cytoscape.COPASI.COPASIJNI.initCopasi()'
    at org.cytoscape.COPASI.COPASIJNI.initCopasi(Native Method) ~[?:?]
    at org.cytoscape.COPASI.COPASIJNI.<clinit>(COPASIJNI.java:25) ~[?:?]
    at org.cytoscape.COPASI.CCopasiMessage.<init>(CCopasiMessage.java:87) ~[?:?]
    at org.cytoscape.COPASI.CyActivator.start(CyActivator.java:52) ~[?:?]
    at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:697) ~[?:?]
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2240) ~[?:?]
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2146) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:984) ~[?:?]
    at org.cytoscape.app.internal.manager.BundleApp.start(BundleApp.java:82) ~[?:?]
    ... 8 more

So, I probably need to define the COPASI library dependencies in my pom.xml file. I followed the instructions here to compile the dependencies, but it didn't solve the problem. How can these dependencies be installed locally using maven?

[Frank Bergmann]

This is still the same issue, the line: 

Caused by: java.lang.UnsatisfiedLinkError: 'void org.cytoscape.COPASI.COPASIJNI.initCopasi()'
    at org.cytoscape.COPASI.COPASIJNI.initCopasi(Native Method) ~[?:?]

tells us, that again the native library could not be located. I'm afraid that i don't know how the cytoscape bundles work, and what additional code would be needed to path along the java.library path to your bundle. Googling hints at a "Bundle-NativeCode" header in OSGi. This seems to be the equivalent of the java.library.path ... but again i have no experiences with OSGi. 

I don't think you will have to compile the dependencies locally. There are cmake plugins available that could be used to have maven run cmake. But i dont see this helping. 

Frank

[Hikmet Emre Kaya]

Hello Frank,

I just wanted to update you on my attempts with the project.

When I added the <Bundle-NativeCode> header to my pom.xml file to specify the location of libCopasiJava.so,  I received the following error:

Invalid bundle classpath entry: libCopasiJava.so: java.util.zip.ZipException: The JAR/ZIP file (/home/people/hkaya/git/cytoscape-api/cycopasi/src/main/resources/libCopasiJava.so) seems corrupted, error: zip END header not found

Do you have any idea why this error is generated? Is the header expecting a ZIP file even though the .so file is outside copasi.jar?

[Frank Bergmann]

As mentioned before, i'm afraid i don't know how to write these plugins. If i had to interpret the error, it would seem whatever you added to the pom, is expected to be a zip file. We will have to see the full specification you added, please also have a look at: 

* http://blog.vogella.com/2010/07/27/osgi/

* http://docs.osgi.org/specification/osgi.core/7.0.0/framework.module.html#framework.module-loading.native.code.libraries

cheers

Frank

[Hikmet Emre Kaya]

Hi Frank,
I have managed to include the .so file in my jar, and the app starts without crashing, but does not appear on the menu tab in Cytoscape. I figured this might be because of the way I am activating it.

I want to clarify: when accessing the api of COPASI in the java project, does one have to do System.loadLibrary to load the native library? If this is already done in the c++ wrapper COPASIJNI.class, which many other COPASI classes depend on, do we still need to declare these classes as native in our source code?

Some examples contain System.loadLibrary and some don't, so I got confused

I hope my question makes sense.

[Frank Bergmann]

Hello, 

i did verify, that the load library call is not necessary, as it will be done at static initialization in the base JNI class, so using any class will cause the static initialization to hit. So indeed it is not necessary in the examples. 

Incidentally, i would actually like to change the behavior of this, currently the static initializer catches the exception, prints to std::error and then exits. I am not sure this is the desired behavior. I think it would be better if the exception would just be thrown. And perhaps even, that there would be an option to prevent the static initialization alltogether. What are your thoughts?

thanks

Frank

[Hikmet Emre Kaya]

Hi Frank,

I am not sure but I previously found the following links when investigating this issue. See if they are relevant?

https://github.com/LWJGL/lwjgl3/issues/277
https://stackoverflow.com/questions/11783632/how-do-i-load-and-use-native-library-in-java

I am trying to figure out if there is a way to call copasi.jar from my bundle app with a command. Because the jar doesn't have a main class, calling java -jar copasi.jar will return a "no main attribute..." error. So, I am not sure how to implement the main class and utilize this jar file as an executable. I understand that one can type java -cp copasi.jar mainclass.java (where mainclass.java can be one of the example files), but it is not really the best way to do it inside a bundle app, I suppose.

What do you think?

[Frank Bergmann]

I'm afraid it is really hard to see where you are at, and what the problem would be. Do you have a github project that i can look at to see where the issues are?

thanks
Frank

[Hikmet Emre Kaya]

https://github.com/kyxhik/CytoCopasi/blob/master/src/main/java/org/cytoscape/CytoCopasi/Example1/Example1.java

My initial idea was to add the following to the unimplemented public void run method at the end of Example1.java

ExecuteCommandTask executeCommandTask = new ExecuteCommandTask(command);

Where the command would be something like "java -jar copasi.jar" or "java -cp copasi.jar main.class" that would initialize COPASI

I hope I am making sense :D Calling myself a novice in app development would be an understatement :)

[Frank Bergmann]

I took some time to figure out how it actually would work, and created a test repository here: 

https://github.com/copasi/cytoscape-copasi

That one implements a very basic reader for COPASI files, creating nodes for compartments / species / parameters / reactions, creating edges between species involved in a reaction. That should show you how these things should work together. 

Currently the plugin requires placing the native library into the java.library.path (and there is currently no need to manually call loadLibrary). However, as indicated in my last reply, it might make sense to change that behavior, so applications can supply a custom loader for the native library. (that way you could bundle the native binaries as resources with the jar file, and extract it in a temporary place, and load the library from there, which would be needed to publish the cytoscape app in the app store i think. 

cheers

Frank

[Hikmet Emre Kaya]

Hi Frank.

Thank you so much for this! The way I wrote the pom.xml and installed copasi.jar as a local dependency is quite similar. I also found out about Cy3Sbml just after I messaged you. I initially thought I would just create a time course simulator class that uses the model directly from Cy3SBML, but the way you structured it makes more sense.

I am taking my time to fully understand how you incorporated COPASI into some of the class files. I will get back to you with some questions and take it from there.

[Hikmet Emre Kaya]

Hi Frank,

Before asking my other questions, here is how it looks on my Linux machine.



As per the ReadMe instructions, I have mvn-installed copasi.jar file, edited my pom.xml and copied the .so file to /usr/lib/ Interestingly, it reads .xml files but gives an unsatisfiedLinkError when trying to open .cps files

java.lang.UnsatisfiedLinkError: 'boolean org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(long, org.COPASI.CDataModel, java.lang.String)'                                          
    at org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(Native Method) ~[?:?]
    at org.COPASI.CDataModel.loadFromString(CDataModel.java:88) ~[?:?]
    at org.copasi.cytoscape.internal.tasks.CopasiFileReaderTask.run(CopasiFileReaderTask.java:165) ~[?:?]
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.innerRun(JDialogTaskManager.java:337) ~[93:org.cytoscape.work-swing-impl:3.8.2]
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.run(JDialogTaskManager.java:352) [93:org.cytoscape.work-swing-impl:3.8.2]

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]

java.lang.Exception: java.lang.UnsatisfiedLinkError: 'boolean org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(long, org.COPASI.CDataModel, java.lang.String)'
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.run(JDialogTaskManager.java:362)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.UnsatisfiedLinkError: 'boolean org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(long, org.COPASI.CDataModel, java.lang.String)'
    at org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(Native Method)
    at org.COPASI.CDataModel.loadFromString(CDataModel.java:88)
    at org.copasi.cytoscape.internal.tasks.CopasiFileReaderTask.run(CopasiFileReaderTask.java:165)
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.innerRun(JDialogTaskManager.java:337)
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.run(JDialogTaskManager.java:352)

Have you encountered this error when you tried it? Why would it specifically give the error with a .cps file, but not with an .xml file?

[Frank Bergmann]

Maybe you could get the java.library.path from the runtime properties and print it as well. (This will get the locations from which the so file will be picked up). Also the copasi.jar and the libCopasiJava.so are quite specific, have you taken them from: 

https://github.com/copasi/cytoscape-copasi/tree/master/lib

This should help narrow things down

best

Frank

[Hikmet Emre Kaya]

Java Library Path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib

I definitely took the copasi.jar and libCopasiJava.so files from your repository, and then I copied the .so file to /usr/lib as a root (sudo cp ...). I didn't take the copasi-gui.jar file, though.

[Hikmet Emre Kaya]

I think it might be a permission error, perhaps I don't have the right permission to retrieve libCopasiJava.so from /usr/lib/ (because this is a root file)

Interestingly, when I typed export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/people/hkaya/git/cytoscape-api/CytoCopasi/lib/ (to add an accessible location to the java.library.path) manually, I was able to import a .cps file

Perhaps, I could try to use System.setProperty to add the location of the .so file to my java.library.path without having to copy the .so file itself.

[Frank Bergmann]

I'm glad to hear that it finally works. I dont think that you can modify the java.library.path on a running instance. The only alternative to modifying the library path in advance, would be to write a custom loader, that would: 

* take the correct native library (for the currently running OS) out of a resource bundle and safe it to a temporary location

* use System.load to load it

* at shutdown clean up after itself. 

That would mean that the copasi.jar could no longer load the native library automatically though. But in the end that is the only way we can support multiple scenarios like this one. (For most people using the Java Bindings, the only thing that would change, would be that a manual System.loadLibrary call would become necessary. 

cheers

Frank

--
You received this message because you are subscribed to the Google Groups "COPASI User Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to copasi-user-fo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/50ca5aea-fb18-4589-bfd4-588eb8371d23n%40googlegroups.com.

[Hikmet Emre Kaya]

Hello Frank,

I hope you have been doing well.

I put the native library issue aside later to deal with when it's time to think about app submission. In the meantime, I was able to expand the app's features so it can make conversion between .sbml and .cps files (i.e., save one as the other)

I am now wanting to create plots for time course and steady state simulations, but am confused about how to make use of the classes in copasi.jar . More specifically, does one first have to generate a report to create a plot? Or can I generate a plot directly?

I don't really understand what purpose PlotSpecification, CPlotItem , CPlotDataChannelSpec.java, etc each serve :/

[Hikmet Emre Kaya]

My SimulationTask class will briefly pop up a dialog  (in a similar way to CellDesigner) to ask for duration, number of steps, etc, as well as the plot type (which I didn't have any problem writing)

[Frank Bergmann]

Hello, 

I really don't think that the plot specification of the copasi API will help you. It is just there, to describe what should be plotted, so that when something is run by Copasi UI, the corresponding plots pop up (you would only need those if you wanted to encode a copasi file, that displays a similar plot to the one you had in your application). The copasi.jar file only gives you access to the backend. The report definitions conversely, describe how something should be written to file. So when you use that, and set a report target to a specific task, the corresponding file will be written to hard drive. 

What you could be using is the data model directly. So for time courses, you could specify that the results should be kept in memory (i.e. in a CTimeSeries object), where you can retrieve the results after a simulation is run. For steady state, once you have run the steady state task (and a steady state was reached), the transient values of the model objects will reflect the model at steady state. So if you at that point asked the metabolites about their concentration, they would return the steady state concentrations.  

So i'd suggest you retrieve the values from there and plot them with whatever plotting library / toolkit you would like. 

I hope that helps, 

best

Frank

[Hikmet Emre Kaya]

Hi Frank!

Thank you for the suggestion. That actually worked perfectly! I used JFreeChart library to obtain plots.

My next question is; how would one go about getting flux rate vs time? I saw the getFlux() method, but the Copasi documentation says it only gives the final flux (which I tested and confirmed)

Thanks

Emre

[Hikmet Emre Kaya]

Hello Frank;

I have made quite a lot of progress with COPASI's implementation of CytoScape, and I managed to generate plots for Time-Course Simulations, as well as running Steady-State Calculations.

I am currently working on writing the Optimization Task. I created the user dialog to specify the objective function, the parameter, and the optimization method, but I got stuck at some point.

I am not sure how to create expressions for the objective function. For example, if the user wants to calculate the ratios of two reaction fluxes, how can one generate that expression? I can parse reaction names, but then the app would not understand that I am referring to their fluxes. Is there a method to specify those? I hope my question is clear enough.

Best,

Emre

[Hikmet Emre Kaya]

For example;

When you click on a reaction flux, the expression looks like this:

{(Adenylate kinase).Flux}

How does this work in COPASI's java bindings?

[Frank Bergmann]

Hello Emre, 

these expressions have to be constructed by using the CN syntax. CNs are the names COPASI uses internally for all model elements and their attributes, and pretty much any other object too. They have the form like: 

    CN=Root,Model=The Brusselator,Vector=Compartments[compartment],Vector=Metabolites[Y],Reference=Concentration

Wherever expressions are used in COPASI you can use the allowed subset of operators and functions from: 

    http://copasi.org/Support/User_Manual/Model_Creation/User_Defined_Functions/

and use them with CNs. The important thing is to surround them in '<', '>' brackets. So for example: 

    <cn1> - <cn2>

to create an expression with the difference of two other elements. 

This will take some time to begin with, but I would suggest that you manually create a plot for the elements you want to use, open it in a text editor, and look at the `ChannelSpec` element to see how they should look like. 

Programmatically you get to those expressions, by asking the value reference for its CN. So in java if `reaction` is a `CReaction` element, then it would be: 

     String cn = reaction.getValueReference().getCN().getString()

i hope this helps, 

best

Frank

You received this message because you are subscribed to a topic in the Google Groups "COPASI User Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/copasi-user-forum/g_gVsfDofOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to copasi-user-fo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/5ed6a4b9-2bac-41e2-bd72-c09b1e62e32an%40googlegroups.com.

[Hikmet Emre Kaya]

Hello Frank;

Thank you for the detailed explanation. By that logic, I can get an expression for the flux by model.getReaction(index).getFluxReference().getCN.toString(); correct?

I just didn't quite understand the following:

but I would suggest that you manually create a plot for the elements you want to use, open it in a text editor, and look at the `ChannelSpec` element to see how they should look like. 

Do you mean I should save the plot as a txt file to find the ChannelSpec?

[Hikmet Emre Kaya]

One more question;

I can't figure out how to get the CN format for the reaction parameters. I can get parameter names from model.getReaction(index).getParameters.getName(), but I don't think that can be put into an expression.

[Frank Bergmann]

I think in Java the best thing to get just the local parameter cns, would be to call: 

    reaction.getParameterCNs()

and in the result filter for those that contain the substring ',ParameterGroup=Parameters,Parameter='. 

best

Frank

[Frank Bergmann]

Thank you for the detailed explanation. By that logic, I can get an expression for the flux by model.getReaction(index).getFluxReference().getCN.toString(); correct?

yes

 

I just didn't quite understand the following:

but I would suggest that you manually create a plot for the elements you want to use, open it in a text editor, and look at the `ChannelSpec` element to see how they should look like. 

Do you mean I should save the plot as a txt file to find the ChannelSpec?

I just meant for debugging, saving the model as COPASI file and looking at it in a text editor, will show you the precise format of the CN for the elements you are looking after. For example for local parameters it would have showed you that the format is: 

   CN=Root,Model=<model name>,Vector=Reactions[<reaction name>],ParameterGroup=Parameters,Parameter=<local parameter name>

best

Frank

 

[Hikmet Emre Kaya]

Hi Frank,

Thank you for the valuable suggestions; it all makes perfect sense.

I had an idea about what to do when the user gives an expression for the input (By selecting objects from a JTree and putting math operations in between)

• Write a separate class that reads this expression, separates it into its components

•  gets the common name for each object in the expression according to the string. (e.g., if it is Flux.something, get the fluxValueReference, if it is [something], get the common name for the concentration, and so on)

• After converting them into their CN format, it surrounds each object with < > brackets. 

• It then feeds this revised formula back into the Optimization task. 

Do you think this is doable, or is it way more complex than it should've been?