Linking COPASI and Cytoscape

[Hikmet Emre Kaya]

Hello Everyone!

For my PhD project, I need to link COPASI and Cytoscape.

Cytoscape is an open source bioinformatics software platform for visualizing molecular interaction networks and integrating with gene expression profiles and other state data.

It has an app store where you can install apps for additional features, but there is no app for what I want to do, which is to run reaction kinetics simulations on the networks, particularly glycosylation networks.

So, I want to make COPASI available and compatible with Cytoscape.

Because Cytoscape is a Java application, I downloaded the java bindings for COPASI and decompiled the .class files to .java files. But, I am not sure how to proceed from here. Using Eclipse, I created a new maven project in the Cytoscape-api repository and added the .java files for COPASI.

I think I need to update the pom.xml file to add the COPASI dependencies, but I am not sure how to do that.

Sorry if the question sounds too vague, I guess I need an overall idea of how to best go about this.

Thanks,

Emre

[Frank Bergmann]

Hello Emre, 

since COPASI is written in C++, the way we provide support for it to Java, is by automatically wrapping the C++ classes via SWIG. SWIG generates the Java-wrapper files (and c++ wrapper file containing the native JNI symbols for the java wrapper files), that are then compiled into the copasi jar file. This jar file, will only work with the native library, that was generated alongside it. If you just wanted to do it for your local instance, you would then include this jar file in your java project, and ensure that the accompanying native library is in the `java.library.path`. from there you can access the api of COPASI as done in these examples: 

https://github.com/copasi/COPASI/tree/develop/copasi/bindings/java/examples

And here is where it becomes a bit more complicated, if you would like the jar file to work on all systems, you also need compilations of the same generated JNI wrapper on the other platforms you would want to support. As for how to include this multi-platform approach in a maven build i am not sure. 

Again the java files, without the native library will not work for you. 

best

Frank

[Hikmet Emre Kaya]

Hi Frank,

Thanks for this detailed explanation.

At the moment, when I copy the .jar file to CytoscapeConfiguration/3/apps/installed, Cytoscape crashes and gives the following error:

Native code library failed to load.                                                                                                                                                      
java.lang.UnsatisfiedLinkError: no CopasiJava in java.library.path: [/home/people/hkaya/Cytoscape_v3.8.2/framework/lib, /home/people/hkaya/Cytoscape_v3.8.2/framework/lib, /usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]

[Frank Bergmann]

Indeed, as i pointed out, the jar file cannot work without the C++ library. So assuming (from your paths provided) that you work on linux x64, i assume you have downloaded this archive: 

https://github.com/copasi/COPASI/releases/download/Build-240/COPASI-4.30.240-Java-Bindings-Linux-64bit.tar.gz

in that archive you find next to the `copasi.jar` file (with the java wrappers for the library, but not the implementation) the native library `libCopasiJava.so`this needs to be copied into a path that is in your `java.library.path` or you need to include its location manually into the java.library.path. 

best

Frank

[Hikmet Emre Kaya]

Hi Frank,

That makes sense; I do work on linux x64.

I will try this and keep you posted about the outcome. Thanks!

[Hikmet Emre Kaya]

Hello Frank,

So, linking the native library to my java.library.path did work, but COPASI still fails to start when I open Cytoscape and gives the following error:

org.cytoscape.app.internal.exception.AppStartupException: Bundle start error
    at org.cytoscape.app.internal.manager.BundleApp.start(BundleApp.java:84) ~[?:?]
    at org.cytoscape.app.internal.manager.AppManager.startApps(AppManager.java:404) ~[?:?]
    at org.cytoscape.app.internal.manager.AppManager.initializeApps(AppManager.java:374) ~[?:?]
    at org.cytoscape.app.internal.manager.AppManager.lambda$attemptInitialization$0(AppManager.java:235) ~[?:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.osgi.framework.BundleException: Activator start error in bundle org.cytoscape.COPASI [123].
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2290) ~[?:?]
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2146) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:984) ~[?:?]
    at org.cytoscape.app.internal.manager.BundleApp.start(BundleApp.java:82) ~[?:?]
    ... 8 more
Caused by: java.lang.UnsatisfiedLinkError: 'void org.cytoscape.COPASI.COPASIJNI.initCopasi()'
    at org.cytoscape.COPASI.COPASIJNI.initCopasi(Native Method) ~[?:?]
    at org.cytoscape.COPASI.COPASIJNI.<clinit>(COPASIJNI.java:25) ~[?:?]
    at org.cytoscape.COPASI.CCopasiMessage.<init>(CCopasiMessage.java:87) ~[?:?]
    at org.cytoscape.COPASI.CyActivator.start(CyActivator.java:52) ~[?:?]
    at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:697) ~[?:?]
    at org.apache.felix.framework.Felix.activateBundle(Felix.java:2240) ~[?:?]
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2146) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998) ~[?:?]
    at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:984) ~[?:?]
    at org.cytoscape.app.internal.manager.BundleApp.start(BundleApp.java:82) ~[?:?]
    ... 8 more

So, I probably need to define the COPASI library dependencies in my pom.xml file. I followed the instructions here to compile the dependencies, but it didn't solve the problem. How can these dependencies be installed locally using maven?

[Frank Bergmann]

This is still the same issue, the line: 

Caused by: java.lang.UnsatisfiedLinkError: 'void org.cytoscape.COPASI.COPASIJNI.initCopasi()'
    at org.cytoscape.COPASI.COPASIJNI.initCopasi(Native Method) ~[?:?]

tells us, that again the native library could not be located. I'm afraid that i don't know how the cytoscape bundles work, and what additional code would be needed to path along the java.library path to your bundle. Googling hints at a "Bundle-NativeCode" header in OSGi. This seems to be the equivalent of the java.library.path ... but again i have no experiences with OSGi. 

I don't think you will have to compile the dependencies locally. There are cmake plugins available that could be used to have maven run cmake. But i dont see this helping. 

Frank

[Hikmet Emre Kaya]

Hello Frank,

I just wanted to update you on my attempts with the project.

When I added the <Bundle-NativeCode> header to my pom.xml file to specify the location of libCopasiJava.so,  I received the following error:

Invalid bundle classpath entry: libCopasiJava.so: java.util.zip.ZipException: The JAR/ZIP file (/home/people/hkaya/git/cytoscape-api/cycopasi/src/main/resources/libCopasiJava.so) seems corrupted, error: zip END header not found

Do you have any idea why this error is generated? Is the header expecting a ZIP file even though the .so file is outside copasi.jar?

[Frank Bergmann]

As mentioned before, i'm afraid i don't know how to write these plugins. If i had to interpret the error, it would seem whatever you added to the pom, is expected to be a zip file. We will have to see the full specification you added, please also have a look at: 

* http://blog.vogella.com/2010/07/27/osgi/

* http://docs.osgi.org/specification/osgi.core/7.0.0/framework.module.html#framework.module-loading.native.code.libraries

cheers

Frank

[Hikmet Emre Kaya]

Hi Frank,
I have managed to include the .so file in my jar, and the app starts without crashing, but does not appear on the menu tab in Cytoscape. I figured this might be because of the way I am activating it.

I want to clarify: when accessing the api of COPASI in the java project, does one have to do System.loadLibrary to load the native library? If this is already done in the c++ wrapper COPASIJNI.class, which many other COPASI classes depend on, do we still need to declare these classes as native in our source code?

Some examples contain System.loadLibrary and some don't, so I got confused

I hope my question makes sense.

[Frank Bergmann]

Hello, 

i did verify, that the load library call is not necessary, as it will be done at static initialization in the base JNI class, so using any class will cause the static initialization to hit. So indeed it is not necessary in the examples. 

Incidentally, i would actually like to change the behavior of this, currently the static initializer catches the exception, prints to std::error and then exits. I am not sure this is the desired behavior. I think it would be better if the exception would just be thrown. And perhaps even, that there would be an option to prevent the static initialization alltogether. What are your thoughts?

thanks

Frank

[Hikmet Emre Kaya]

Hi Frank,

I am not sure but I previously found the following links when investigating this issue. See if they are relevant?

https://github.com/LWJGL/lwjgl3/issues/277
https://stackoverflow.com/questions/11783632/how-do-i-load-and-use-native-library-in-java

I am trying to figure out if there is a way to call copasi.jar from my bundle app with a command. Because the jar doesn't have a main class, calling java -jar copasi.jar will return a "no main attribute..." error. So, I am not sure how to implement the main class and utilize this jar file as an executable. I understand that one can type java -cp copasi.jar mainclass.java (where mainclass.java can be one of the example files), but it is not really the best way to do it inside a bundle app, I suppose.

What do you think?

[Frank Bergmann]

I'm afraid it is really hard to see where you are at, and what the problem would be. Do you have a github project that i can look at to see where the issues are?

thanks
Frank

[Hikmet Emre Kaya]

https://github.com/kyxhik/CytoCopasi/blob/master/src/main/java/org/cytoscape/CytoCopasi/Example1/Example1.java

My initial idea was to add the following to the unimplemented public void run method at the end of Example1.java

ExecuteCommandTask executeCommandTask = new ExecuteCommandTask(command);

Where the command would be something like "java -jar copasi.jar" or "java -cp copasi.jar main.class" that would initialize COPASI

I hope I am making sense :D Calling myself a novice in app development would be an understatement :)

[Frank Bergmann]

I took some time to figure out how it actually would work, and created a test repository here: 

https://github.com/copasi/cytoscape-copasi

That one implements a very basic reader for COPASI files, creating nodes for compartments / species / parameters / reactions, creating edges between species involved in a reaction. That should show you how these things should work together. 

Currently the plugin requires placing the native library into the java.library.path (and there is currently no need to manually call loadLibrary). However, as indicated in my last reply, it might make sense to change that behavior, so applications can supply a custom loader for the native library. (that way you could bundle the native binaries as resources with the jar file, and extract it in a temporary place, and load the library from there, which would be needed to publish the cytoscape app in the app store i think. 

cheers

Frank

[Hikmet Emre Kaya]

Hi Frank.

Thank you so much for this! The way I wrote the pom.xml and installed copasi.jar as a local dependency is quite similar. I also found out about Cy3Sbml just after I messaged you. I initially thought I would just create a time course simulator class that uses the model directly from Cy3SBML, but the way you structured it makes more sense.

I am taking my time to fully understand how you incorporated COPASI into some of the class files. I will get back to you with some questions and take it from there.

[Hikmet Emre Kaya]

Hi Frank,

Before asking my other questions, here is how it looks on my Linux machine.



As per the ReadMe instructions, I have mvn-installed copasi.jar file, edited my pom.xml and copied the .so file to /usr/lib/ Interestingly, it reads .xml files but gives an unsatisfiedLinkError when trying to open .cps files

java.lang.UnsatisfiedLinkError: 'boolean org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(long, org.COPASI.CDataModel, java.lang.String)'                                          
    at org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(Native Method) ~[?:?]
    at org.COPASI.CDataModel.loadFromString(CDataModel.java:88) ~[?:?]
    at org.copasi.cytoscape.internal.tasks.CopasiFileReaderTask.run(CopasiFileReaderTask.java:165) ~[?:?]
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.innerRun(JDialogTaskManager.java:337) ~[93:org.cytoscape.work-swing-impl:3.8.2]
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.run(JDialogTaskManager.java:352) [93:org.cytoscape.work-swing-impl:3.8.2]

    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]

java.lang.Exception: java.lang.UnsatisfiedLinkError: 'boolean org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(long, org.COPASI.CDataModel, java.lang.String)'
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.run(JDialogTaskManager.java:362)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.UnsatisfiedLinkError: 'boolean org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(long, org.COPASI.CDataModel, java.lang.String)'
    at org.COPASI.COPASIJNI.CDataModel_loadFromString__SWIG_3(Native Method)
    at org.COPASI.CDataModel.loadFromString(CDataModel.java:88)
    at org.copasi.cytoscape.internal.tasks.CopasiFileReaderTask.run(CopasiFileReaderTask.java:165)
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.innerRun(JDialogTaskManager.java:337)
    at org.cytoscape.work.internal.task.JDialogTaskManager$TaskRunnable.run(JDialogTaskManager.java:352)

Have you encountered this error when you tried it? Why would it specifically give the error with a .cps file, but not with an .xml file?

[Frank Bergmann]

Maybe you could get the java.library.path from the runtime properties and print it as well. (This will get the locations from which the so file will be picked up). Also the copasi.jar and the libCopasiJava.so are quite specific, have you taken them from: 

https://github.com/copasi/cytoscape-copasi/tree/master/lib

This should help narrow things down

best

Frank

[Hikmet Emre Kaya]

Java Library Path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib

I definitely took the copasi.jar and libCopasiJava.so files from your repository, and then I copied the .so file to /usr/lib as a root (sudo cp ...). I didn't take the copasi-gui.jar file, though.

[Hikmet Emre Kaya]

I think it might be a permission error, perhaps I don't have the right permission to retrieve libCopasiJava.so from /usr/lib/ (because this is a root file)

Interestingly, when I typed export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/people/hkaya/git/cytoscape-api/CytoCopasi/lib/ (to add an accessible location to the java.library.path) manually, I was able to import a .cps file

Perhaps, I could try to use System.setProperty to add the location of the .so file to my java.library.path without having to copy the .so file itself.

[Frank Bergmann]

I'm glad to hear that it finally works. I dont think that you can modify the java.library.path on a running instance. The only alternative to modifying the library path in advance, would be to write a custom loader, that would: 

* take the correct native library (for the currently running OS) out of a resource bundle and safe it to a temporary location

* use System.load to load it

* at shutdown clean up after itself. 

That would mean that the copasi.jar could no longer load the native library automatically though. But in the end that is the only way we can support multiple scenarios like this one. (For most people using the Java Bindings, the only thing that would change, would be that a manual System.loadLibrary call would become necessary. 

cheers

Frank

--
You received this message because you are subscribed to the Google Groups "COPASI User Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to copasi-user-fo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/50ca5aea-fb18-4589-bfd4-588eb8371d23n%40googlegroups.com.

[Hikmet Emre Kaya]

Hello Frank,

I hope you have been doing well.

I put the native library issue aside later to deal with when it's time to think about app submission. In the meantime, I was able to expand the app's features so it can make conversion between .sbml and .cps files (i.e., save one as the other)

I am now wanting to create plots for time course and steady state simulations, but am confused about how to make use of the classes in copasi.jar . More specifically, does one first have to generate a report to create a plot? Or can I generate a plot directly?

I don't really understand what purpose PlotSpecification, CPlotItem , CPlotDataChannelSpec.java, etc each serve :/

[Hikmet Emre Kaya]

My SimulationTask class will briefly pop up a dialog  (in a similar way to CellDesigner) to ask for duration, number of steps, etc, as well as the plot type (which I didn't have any problem writing)

[Frank Bergmann]

Hello, 

I really don't think that the plot specification of the copasi API will help you. It is just there, to describe what should be plotted, so that when something is run by Copasi UI, the corresponding plots pop up (you would only need those if you wanted to encode a copasi file, that displays a similar plot to the one you had in your application). The copasi.jar file only gives you access to the backend. The report definitions conversely, describe how something should be written to file. So when you use that, and set a report target to a specific task, the corresponding file will be written to hard drive. 

What you could be using is the data model directly. So for time courses, you could specify that the results should be kept in memory (i.e. in a CTimeSeries object), where you can retrieve the results after a simulation is run. For steady state, once you have run the steady state task (and a steady state was reached), the transient values of the model objects will reflect the model at steady state. So if you at that point asked the metabolites about their concentration, they would return the steady state concentrations.  

So i'd suggest you retrieve the values from there and plot them with whatever plotting library / toolkit you would like. 

I hope that helps, 

best

Frank

[Hikmet Emre Kaya]

Hi Frank!

Thank you for the suggestion. That actually worked perfectly! I used JFreeChart library to obtain plots.

My next question is; how would one go about getting flux rate vs time? I saw the getFlux() method, but the Copasi documentation says it only gives the final flux (which I tested and confirmed)

Thanks

Emre

[Hikmet Emre Kaya]

Hello Frank;

I have made quite a lot of progress with COPASI's implementation of CytoScape, and I managed to generate plots for Time-Course Simulations, as well as running Steady-State Calculations.

I am currently working on writing the Optimization Task. I created the user dialog to specify the objective function, the parameter, and the optimization method, but I got stuck at some point.

I am not sure how to create expressions for the objective function. For example, if the user wants to calculate the ratios of two reaction fluxes, how can one generate that expression? I can parse reaction names, but then the app would not understand that I am referring to their fluxes. Is there a method to specify those? I hope my question is clear enough.

Best,

Emre

[Hikmet Emre Kaya]

For example;

When you click on a reaction flux, the expression looks like this:

{(Adenylate kinase).Flux}

How does this work in COPASI's java bindings?

[Frank Bergmann]

Hello Emre, 

these expressions have to be constructed by using the CN syntax. CNs are the names COPASI uses internally for all model elements and their attributes, and pretty much any other object too. They have the form like: 

    CN=Root,Model=The Brusselator,Vector=Compartments[compartment],Vector=Metabolites[Y],Reference=Concentration

Wherever expressions are used in COPASI you can use the allowed subset of operators and functions from: 

    http://copasi.org/Support/User_Manual/Model_Creation/User_Defined_Functions/

and use them with CNs. The important thing is to surround them in '<', '>' brackets. So for example: 

    <cn1> - <cn2>

to create an expression with the difference of two other elements. 

This will take some time to begin with, but I would suggest that you manually create a plot for the elements you want to use, open it in a text editor, and look at the `ChannelSpec` element to see how they should look like. 

Programmatically you get to those expressions, by asking the value reference for its CN. So in java if `reaction` is a `CReaction` element, then it would be: 

     String cn = reaction.getValueReference().getCN().getString()

i hope this helps, 

best

Frank

You received this message because you are subscribed to a topic in the Google Groups "COPASI User Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/copasi-user-forum/g_gVsfDofOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to copasi-user-fo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/5ed6a4b9-2bac-41e2-bd72-c09b1e62e32an%40googlegroups.com.

[Hikmet Emre Kaya]

Hello Frank;

Thank you for the detailed explanation. By that logic, I can get an expression for the flux by model.getReaction(index).getFluxReference().getCN.toString(); correct?

I just didn't quite understand the following:

but I would suggest that you manually create a plot for the elements you want to use, open it in a text editor, and look at the `ChannelSpec` element to see how they should look like. 

Do you mean I should save the plot as a txt file to find the ChannelSpec?

[Hikmet Emre Kaya]

One more question;

I can't figure out how to get the CN format for the reaction parameters. I can get parameter names from model.getReaction(index).getParameters.getName(), but I don't think that can be put into an expression.

[Frank Bergmann]

I think in Java the best thing to get just the local parameter cns, would be to call: 

    reaction.getParameterCNs()

and in the result filter for those that contain the substring ',ParameterGroup=Parameters,Parameter='. 

best

Frank

[Frank Bergmann]

Thank you for the detailed explanation. By that logic, I can get an expression for the flux by model.getReaction(index).getFluxReference().getCN.toString(); correct?

yes

 

I just didn't quite understand the following:

but I would suggest that you manually create a plot for the elements you want to use, open it in a text editor, and look at the `ChannelSpec` element to see how they should look like. 

Do you mean I should save the plot as a txt file to find the ChannelSpec?

I just meant for debugging, saving the model as COPASI file and looking at it in a text editor, will show you the precise format of the CN for the elements you are looking after. For example for local parameters it would have showed you that the format is: 

   CN=Root,Model=<model name>,Vector=Reactions[<reaction name>],ParameterGroup=Parameters,Parameter=<local parameter name>

best

Frank

 

[Hikmet Emre Kaya]

Hi Frank,

Thank you for the valuable suggestions; it all makes perfect sense.

I had an idea about what to do when the user gives an expression for the input (By selecting objects from a JTree and putting math operations in between)

• Write a separate class that reads this expression, separates it into its components

•  gets the common name for each object in the expression according to the string. (e.g., if it is Flux.something, get the fluxValueReference, if it is [something], get the common name for the concentration, and so on)

• After converting them into their CN format, it surrounds each object with < > brackets. 

• It then feeds this revised formula back into the Optimization task. 

Do you think this is doable, or is it way more complex than it should've been?

[Frank Bergmann]

I would need to see in what forms you take the input expression, if you can post some example expressions you want to support, that would help. If you do allow for the same syntax we use in COPASI, then there are methods that help you on the CDataModel class. (`findObjectByDisplayName` tries to resolve objects by their display name, `getObject` resolves a CCommonName to its object). 

cheers

frank

[Hikmet Emre Kaya]

I am not sure if you can see the screenshot, but this is a rough version of it. Let's say I want to minimize the ratio of two fluxes, so I specify the following expression: Flux.(Glucose transport)/Flux.(Hexokinase)

I thought about using the exact syntax used in COPASI, but wasn't sure if that would help. I can change the string representations of the objects in the selection tree to look exactly like the COPASI versions, if it helps.

--

You received this message because you are subscribed to a topic in the Google Groups "COPASI User Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/copasi-user-forum/g_gVsfDofOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to copasi-user-fo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/54442d7a-3312-46d1-b812-0a3e71472e52n%40googlegroups.com.

--

Hikmet Emre Kaya

Professional Science & Wellness Writer

PhD in Computational Glycoenzymology at the University of Cape Town

Website

[Hikmet Emre Kaya]

In COPASI, the syntax that appears on the selection panel is flux(something), but when you select it, it appears on the expression field as {(Something).Flux}. Which syntax are you referring to?

[Frank Bergmann]

I think it would make sense, to use the same expression syntax as COPASI uses as well, that way they could be copied between tools as well. It would also make testing for you easier. 

For the infix expressions we do use display names (as you can obtain from the value reference objects using `getObjectDisplayName`)  and surround them with curly brackets for easy recognition in the infix editor, so it would be 

    { (Glucose Transport).Flux } /  { (Hexokinase).Flux } 

in the example from your screenshot. 

Cheers

Frank

[Hikmet Emre Kaya]

Hi Frank,

That definitely makes  life a lot easier! To be clear, do you have to put that expression in angle brackets as well, or does COPASI recognize it when you put in optItem.setObjectiveFunction(my expression)

where my expression is: say, { (Glucose Transport).Flux } /  { (Hexokinase).Flux } 

[Frank Bergmann]

Sorry for the confusion, the expressions of the form: 

 { (Glucose Transport).Flux } /  { (Hexokinase).Flux } 

is what we let the user enter, and copy / paste. The API uses the CN form we discussed before:

<cn1> / <cn2>

Frank

You received this message because you are subscribed to the Google Groups "COPASI User Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to copasi-user-fo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/a501d7a9-9e55-4c35-8c47-13034e1b8c7dn%40googlegroups.com.

[Hikmet Emre Kaya]

Oh okay, I am actually more confused now :D

So, let's say I got this user input with the curly brackets and passed it onto the

public void run (... ...)

 - Loading the model from string
 - Running time course or steady state
 - Creating an optimization task


Will I have to convert that expression into the <cn1>/<cn2> format before I set the objective function in my COptProblem?

[Hikmet Emre Kaya]

Hello again, Frank;

I gave it a try and wrote the following by looking at java bindings examples:

CModelValue variableModelValue = model.createModelValue("V");
variableModelValue.setStatus(CModelEntity.Status_ASSIGNMENT);
            
variableModelValue.setExpression(expression);

Where, the expression could be, for example = {[ADP]}/{[ATP]}

 String objectiveFunction = variableModelValue.getObject(new CCommonName("Reference=Value")).getCN().getString();

When I print out objectiveFunction, I get the following:

Objective Function: CN=Root,Model=Karlstaedt2019 - G6P accumulation via Phosphoglucose isomerase inhibition in heart muscles,Vector=Values[V],Reference=Value

But, when I enter the same expression on COPASI and look at the .cps file, the objective function is:

<ParameterText name="ObjectiveExpression" type="expression">
          &lt;CN=Root,Model=Karlstaedt2019 - G6P accumulation via Phosphoglucose isomerase inhibition in heart muscles,Vector=Compartments[cytosol],Vector=Metabolites[ADP],Reference=Concentration>/&lt;CN=Root,Model=Karlstaedt2019 - G6P accumulation via Phosphoglucose isomerase inhibition in heart muscles,Vector=Compartments[cytosol],Vector=Metabolites[ATP],Reference=Concentration>
        </ParameterText>

Does this mean, I am not getting the objective function correctly?

[Hikmet Emre Kaya]

Hi Frank,

Sorry for asking too many questions :D I actually managed to do the conversion from object display name to Common Name, which gives the correct value for the minimum objective function.

My question is about the optimization task itself. I am confused as to why COPASI gives a different best parameter value (corresponding to the minimum objective function) after each optimization run. Is there a document that you can suggest for me to understand the logic of optimization better?

Best,

Emre

[Frank Bergmann]

Hello Emre, 

can you send me the copasi file in question, and again a quick refresher of which different values you are referring to? It makes it easier to answer your question that way. 

cheers

Frank

[Hikmet Emre Kaya]

Hello Frank,

This is the file that I am using to test the app.

When I ran the optimization task in standalone COPASI, I determined an objective function of [ADP]/[ATP] and chose my parameter (to be scanned) as one of the constant k values.

Every time I run the optimization task (without changing the opt method), I get more or less the same value for the minimum objective function, but the corresponding parameter value keeps changing.

So, I feel a bit perplexed about the logic of optimization in COPASI in general. I hope it doesn't sound silly.

Best,

To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/f12e76df-5bf1-4ae7-9afd-d47a9b1ff0e7n%40googlegroups.com.

[Frank Bergmann]

The way the optimization works in your case, is that: 

 - a new random values for your parameters are chosen based on the selected  method (since you chose genetic algorithm, a population of values between 1e-6 and 1e6 are chosen for your parameter)

- each of these values are tested to see what is the minimal value of your chosen expression for those

   -- for that since you selected time course as sub task, each time a time course simulation over the interval [0 1] is run

- then a from there a new guess will be made to improve the value

It is always important to choose parameter ranges that make sense for your model. as it so happens, for the largest part of your range the parameter has very little influence. On your chosen objective function. Here is how you can visualize what is going on: 

- define a new global quantity 'obs' with the assignment expression: {[ADP]}/{[ATP]}

- create a new plot, plotting model time vs the transient value of obs

- select time course

- click the slider button: select Kadp_7, and create a large range as you have with the optimization.

- now as you move these slider along, you see how the simulation updates, but you hardly get any change in the result

You can now also go to scan, and do the same thing there. Vary the parameter in the range, maybe even select the steady state. And you will find that the value hardly changes.

I hope this explains a bit why this is happening. In conclusion:

- think about the ranges of the parameter you want to let varry, what makes sense for your model (in your model parameter values > 1) seem to make no difference

- do you really want to only look at the time course of [0 1]?

- scans / sliders help to visualize what happens.

cheers

Frank

To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/CAFJfJ8jQNnBNqwhiNcT%2B3mO2XRCN08%3Dx-p6Rgp-YVxnxgZJtuw%40mail.gmail.com.

[Pedro Mendes]

Emre,

There are many reasons why the best value of a run may be different from
another run. First you should check if "randomize start values" is
checked. If it is, then each run starts from a different place and
therefore algorithms are very likely to end up in different places
(including the deterministic algorithms).

If the initial values are _not_ randomized then you should only expect
that deterministic algorithms should end in the same place. These
algorithms are Levenberg-Marquardt, Truncated Newton, steepest descent,
NL2SOL, Praxis, Hooke-Jeeves and Nelder-Mead.

However the global optimization algorithms are based on stochasticity
and do not necessarily end up in the same value. Methods in this class
are all teh evolutionary algorithms (including genetic algorithm,
differential evolution, SRES, etc.), particle swarm and simulated
annealing.

A long time ago I published a paper on this topic (which was still for
my older Gepasi software) which talks a little about the algorithms.

Mendes P, Kell DB (1998) Non-linear optimization of biochemical
pathways: applications to metabolic engineering and parameter
estimation. Bioinformatics 14:869–883
https://doi.org/10.1093/bioinformatics/14.10.869


Pedro

On 11/4/21 4:17 AM, Hikmet Emre Kaya wrote:
> Hi Frank,
>
> Sorry for asking too many questions :D I actually managed to do the
> conversion from object display name to Common Name, which gives the
> correct value for the minimum objective function.
>
> My question is about the optimization task itself. I am confused as to
> why COPASI gives a different best parameter value (corresponding to the
> minimum objective function) after each optimization run. Is there a
> document that you can suggest for me to understand the logic of
> optimization better?
>
> Best,
>
> Emre
>
> On Friday, October 29, 2021 at 12:55:23 PM UTC+2 Hikmet Emre Kaya wrote:
>
> Hello again, Frank;
>
> I gave it a try and wrote the following by looking at java bindings
> examples:
>
> CModelValue variableModelValue = model.createModelValue("V");
> variableModelValue.setStatus(CModelEntity.Status_ASSIGNMENT);
>
> variableModelValue.setExpression(expression);
>
> Where, the expression could be, for example = {[ADP]}/{[ATP]}
>
>  String objectiveFunction = variableModelValue.getObject(new
> CCommonName("Reference=Value")).getCN().getString();
>
> When I print out objectiveFunction, I get the following:
>

> *Objective Function: CN=Root,Model=Karlstaedt2019 - G6P accumulation

> via Phosphoglucose isomerase inhibition in heart
> muscles,Vector=Values[V],Reference=Value

> *

> But, when I enter the same expression on COPASI and look at the .cps
> file, the objective function is:
>

> *<ParameterText name="ObjectiveExpression" type="expression">

>           &lt;CN=Root,Model=Karlstaedt2019 - G6P accumulation via
> Phosphoglucose isomerase inhibition in heart
> muscles,Vector=Compartments[cytosol],Vector=Metabolites[ADP],Reference=Concentration>/&lt;CN=Root,Model=Karlstaedt2019
> - G6P accumulation via Phosphoglucose isomerase inhibition in heart
> muscles,Vector=Compartments[cytosol],Vector=Metabolites[ATP],Reference=Concentration>

>         </ParameterText>*

> *

>
> Write a separate class that
> reads this expression, separates
> it into its components
>

> *

>
>  gets the common name for each
> object in the expression
> according to the string. (e.g.,
> if it is Flux.something, get the
> fluxValueReference, if it is
> [something], get the common name
> for the concentration, and so on)
>

> *

>
> After converting them into their
> CN format, it surrounds each
> object with < > brackets.
>

> *

> <https://groups.google.com/d/msgid/copasi-user-forum/54442d7a-3312-46d1-b812-0a3e71472e52n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> *Hikmet Emre Kaya*
> *
> *
> *Professional Science & Wellness Writer*
> *
> *

> PhD in Computational Glycoenzymology at the
> University of Cape Town
>

> Website <https://hikmetemrekaya.wordpress.com/>
>
>
>
>
> --
> *Hikmet Emre Kaya*
> *
> *
> *Professional Science & Wellness Writer*
> *
> *

> PhD in Computational Glycoenzymology at the
> University of Cape Town
>

> Website <https://hikmetemrekaya.wordpress.com/>

>
>
> --
>
> You received this message because you are subscribed to
> the Google Groups "COPASI User Forum" group.
>
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> copasi-user-fo...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/copasi-user-forum/a501d7a9-9e55-4c35-8c47-13034e1b8c7dn%40googlegroups.com

> <https://groups.google.com/d/msgid/copasi-user-forum/a501d7a9-9e55-4c35-8c47-13034e1b8c7dn%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "COPASI User Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to copasi-user-fo...@googlegroups.com

> <mailto:copasi-user-fo...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/copasi-user-forum/e9a6b2ab-faa5-4709-9907-d402434d7604n%40googlegroups.com
> <https://groups.google.com/d/msgid/copasi-user-forum/e9a6b2ab-faa5-4709-9907-d402434d7604n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Hikmet Emre Kaya]

Dear Frank and Pedro;

Thank you for your insightful comments. These suggestions will definitely help me grasp the logic of optimization much better.

I have one more question related to optimization. In stand alone COPASI, the best value is shown in a live-updating plot, but I am not sure if I can generate the same plot from the Java bindings. It seems that one can only retrieve the final value after the task is finished. Is there a way to iteratively access the value found after each function evaluation?

[Frank Bergmann]

Hello

 

I have one more question related to optimization. In stand alone COPASI, the best value is shown in a live-updating plot, but I am not sure if I can generate the same plot from the Java bindings. It seems that one can only retrieve the final value after the task is finished. Is there a way to iteratively access the value found after each function evaluation?

I think the idea would be to create a callback class, and attach it to the optimization task when you run the task. Then COPASI will call back your application, at which point you could could read out the current objective value. 

The example below shows how to attach a shutdown handler to the task, so that when the JVM is terminated, the task is stopped. You will have to add to the class to give it access to the optimization problem. On which you call `getSolutionValue` periodically when you want to update your information. 

https://github.com/copasi/COPASI/blob/develop/copasi/bindings/java/examples/processCallback.java

best

Frank

[Hikmet Emre Kaya]

Hello Frank!

I have also been working on bringing the Parameter Scan feature into Cytoscape. Currently working on single parameter scan for now. The task does run without errors, but I am confused about retrieving the set of results for each parameter. For example, the user makes a selection from a plot assistant to get steady state concentration for each parameter value and create a plot (as described here: https://www.youtube.com/watch?v=mRPY3m8pWJU).

Do you need to implement the callback class idea here as well?

[Frank Bergmann]

Hello, 

What you would have to do, is to create an instance of the CDataHandler class, and add all the CN's that you are interested in, to it. Then you attach the datahandler to the datamodel prior to running the scan task, and detach it later. Or you pass it along to the process method. After a successful run, you then retrieve the stored results from the data handler class. Note: it currently only collects real valued references. i have a patch ready to expand it to boolean and integer values, but that is not yet released. 

I dont have an example of doing that in Java right now, here is how it looks in python: 

https://github.com/copasi/COPASI/blob/develop/copasi/bindings/python/examples/custom_output.py

should you have any trouble with it, let me know and i'll create a java example for it. 

The alternative is to let the task write the output a report file, that you then read back in and extract the data from. 

best

Frank

[Hikmet Emre Kaya]

Hi Frank,

Thank you for the detailed response. I actually have been trying to create a report file, but it's coming out empty for some reason. Let me go over this example, and I'll get back to you if I have any questions.

Best,

Emre

[Hikmet Emre Kaya]

Hello Frank,

There is a part that confuses me about this example. Under create_datahandler, there is the line:


if isinstance(obj, COPASI.CModel): # fix for time
obj = obj.getValueReference()

in java, obj = dm.findObjectByDisplayName(name) is a CDataObject, and it does not have the getValueReference option. So, I am not sure how this part translates to Java.

[Hikmet Emre Kaya]

What also confuses me is that; it seems like the data handler class (or the report) would both store only the results from the run with maximum parameter value. Then, how does one go about getting the in-between values? That's when I thought of the processCallback method.

[Frank Bergmann]

Hello, so basically in Java you'd need a cast at this point: 

if (obj instance of CModel)

{

  obj = ((CModel)obj).getValueReference()

}

best

Frank

--
You received this message because you are subscribed to a topic in the Google Groups "COPASI User Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/copasi-user-forum/g_gVsfDofOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to copasi-user-fo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/bc0e9a69-25d3-46bc-b073-808abcd12d4cn%40googlegroups.com.

[Frank Bergmann]

Can you please provide a complete example (of model  + what you want to scan, and what output you want to have). Then it would be easier to encode that and to explain it in less abstract terms. 

In COPASI the output handlers can collect data at distinct times: before, during or after the task is run. In the data handler class you specify which CN's to collect  at these times using either the function addBeforeName, addDuringName or addAfterName to specify that. After the task is completed you get the results for the respective channel by calling: getBeforedata, getDuringData or getAfterData. 

best

Frank

--
You received this message because you are subscribed to a topic in the Google Groups "COPASI User Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/copasi-user-forum/g_gVsfDofOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to copasi-user-fo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/copasi-user-forum/01206f32-cb01-4afd-84e4-2fcb24b79184n%40googlegroups.com.

[Hikmet Emre Kaya]

Hello Frank;

I am using this model as an example.

In the parameter scan, I chose the "scan" option, specify one of the reaction parameters as my parameter object (e.g., (Phosphofructosekinase).Vmax_4 with intervals=10; min = 40, and max=160). Then, I click on plot assistant and select Scan of Concentrations, Volumes, and Global Quantity Values. Then, I go to the plot itself and select one of the species (e.g., [ATP]) and delete all the other curves. For the subtask, I chose steady state so that I can monitor how the std st concentration of that metabolite changes throughout the parameter interval. As far as I understand, COPASI then repeats the subtask for each interval by incrementing the parameter value between the min and the max. Then, you generate a plot like the one attached.

So, I am wanting to collect that specific model variable for each parameter value to see its dependency on that parameter.