Questions on Quixote and Chempound

[Oliver Stueker]

Hi everyone,

We at the Memorial University (MUN) and University of Prince Edwards Island (UPEI) are working on creating a repository for (initially) results of QM calculations. Recently we came across Quixote and Chempound and we believe CML/Compchem & RDF would be a very good data-model for us.
Actually about a week ago one of our team members (Jason Pearson at UPEI) introduced our project to Prof. Murray-Rust via Skype.

My role currently is getting an overview what tools and standards have been developed for Quixote/Chempound and see which parts would be useful for our needs. Of course we will contribute any additions to the Chempound, Quixote and jumbo-converters (e.g. to support more QM packages for CML conversion) back to the projects.

After looking into things for the past two or so weeks I came up with a number of questions for which I hope someone here can help me to answer them.


1) First of all: Is the Quixote community still active or at least available to give me some pointers?


2) Do the repositories on BitBucket [2a] contain the most up-to-date source code? After building quixote-client and quixote-repository from source and depositing some of our files, I noticed that in the local Chempound repository most fields (Task, Basis Set, Formula, Point Group, Charge, Final Energy, etc.) are shown as unknown, although the data is present in the RDF (see below) and Package and Method show up correctly.

<rdf:RDF
    xmlns:gaussian="http://www.xml-cml.org/dictionary/gaussian/"
    xmlns:compchem="http://www.xml-cml.org/dictionary/compchem/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:cml="http://www.xmlcml.org/rdf-schema#"
    xmlns:owl="http://www.w3.org/2002/07/owl#"
    xmlns:dc="http://purl.org/dc/terms/"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
  <compchem:Calculation rdf:about="">
<!-- 120 lines deleted -->
    <compchem:basis rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
    >6-31G(d)</compchem:basis>
  </compchem:Calculation>
<!-- 7 lines deleted -->
</rdf:RDF>

Also on [2b] all values are displayed correctly.
Has the RDF schema changed?
Where can I find the chempound code in which the data to be displayed is extracted from the RDF graph?


3) I also noticed that on [2b] the RDF is also available in N3 notation whereas the version built from source only created the RDF/XML file.


4) I was thinking to use lensfield2 for converting our Gaussian files to CML.
So I pulled lensfield from [4a] and tried to follow the tutorial [4b] but all I got is an error ( see [E1] below the links).
Also building lensfield didn't work. [E2]
Does lensfield only convert to CML or can it create the RDF as well?


5) Chempound (the chemistry-jmol-plugin) uses an older version of Jmol (12.0.44) which is not working anymore due to security requirements imposed by Java 1.7.0_51 and newer in January 2014 [5a,b]. Luckily JSmol (a JavaScript version of Jmol) is getting more and more performant and so I've replaced the Jmol Applet in the chemistry-jmol-plugin with JSmol 14.0.11.
I will soon make the updated repo's available on BitBucket and will post a pull-request here.


6) Just as a note: When trying to download the compiled Chempound from [6a], noticed that the hudson at [6b] seems to be offline. Not a real problem as I went and just pulled the sources from bitbucket [2a] and built it myself.

7) Is the cml-discuss list still active? In the archives on sf.net I found a message from 2012 that the list would be moved to a Google group,  but I couldn't find it.




With best regards,
Oliver
--
Oliver Stueker, Dr. rer. nat.
Postdoctoral Fellow, Poirier Lab
Department of Chemistry,
Memorial University of Newfoundland
St. John's, Canada



[2a] https://bitbucket.org/chempound/
[2b] http://quixote.ch.cam.ac.uk/
[4a] https://bitbucket.org/petermr/lensfieldjumbo
[4b] http://quixote.wikispot.org/Tutorials_and_problems#head-4bcc821e7cd47daadb3ae0898b5da42fb860f733
[5a] https://blogs.oracle.com/java-platform-group/entry/new_security_requirements_for_rias
[5b] thread http://sourceforge.net/p/jmol/mailman/message/31586308/
[6a] http://www.chempound.net/download.html
[6b] https://hudson.ch.cam.ac.uk/


[E1] Error executing lensfield:
lensfieldjumbo/examples/gau $ ../../lensfield2-0.1.1/bin/lf gaussianArchive2Cml.lf
Lensfield2 (0.1.1)
----------------------------------------
[INFO]  Checking build steps
[INFO]  Resolving build order
[DEBUG]  - files
[DEBUG]  - cml
[INFO]  Resolving dependencies
[DEBUG] Resolving global dependencies
[DEBUG] Resolving dependencies for: cml
Downloading: https://maven.ch.cam.ac.uk/m2repo/cml/jumbo-converters-compchem/0.3-SNAPSHOT/jumbo-converters-compchem-0.3-SNAPSHOT.jar
----------------------------------------
org.lensfield.LensfieldException: Unable to resolve dependency: [cml:jumbo-converters-compchem:jar:0.3-SNAPSHOT:compile]
    at org.lensfield.DependencyResolver.resolveDependencies(DependencyResolver.java:120)
    at org.lensfield.DependencyResolver.configureDependencies(DependencyResolver.java:252)
    at org.lensfield.DependencyResolver.configureDependencies(DependencyResolver.java:246)
    at org.lensfield.Lensfield.resolveDependencies(Lensfield.java:706)
    at org.lensfield.Lensfield.init(Lensfield.java:83)
    at org.lensfield.Lensfield.build(Lensfield.java:123)
    at org.lensfield.cli.LensfieldCli.run(LensfieldCli.java:87)
    at org.lensfield.cli.LensfieldCli.main(LensfieldCli.java:69)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.lensfield.launcher.load.Loader.run(Loader.java:65)
    at org.lensfield.launcher.load.Loader.main(Loader.java:94)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.lensfield.launcher.boot.Bootstrap.main(Bootstrap.java:52)

BUILD FAILED


[E2] Error compiling lensfield:
lensfieldjumbo $ mvn compile
[INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for wwmm:lensfield-jumbo:jar:0.0.1 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-compiler-plugin is missing. @ line 12, column 15 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-surefire-plugin is missing. @ line 19, column 15 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building lensfield-jumbo 0.0.1 [INFO] ------------------------------------------------------------------------
Downloading: https://maven.ch.cam.ac.uk/m2repo/cml/jumbo/5.5.1-SNAPSHOT/maven-metadata.xml Downloaded: https://maven.ch.cam.ac.uk/m2repo/cml/jumbo/5.5.1-SNAPSHOT/maven-metadata.xml (254 B at 0.1 KB/sec) Downloading: https://maven.ch.cam.ac.uk/m2repo/wwmm/wwmm-parent/2-SNAPSHOT/maven-metadata.xml Downloading: https://maven.ch.cam.ac.uk/m2repo/wwmm/wwmm-parent/2-SNAPSHOT/maven-metadata.xml Downloading: https://maven.ch.cam.ac.uk/m2repo/wwmm/wwmm-parent/2-SNAPSHOT/wwmm-parent-2-SNAPSHOT.pom Downloading: https://maven.ch.cam.ac.uk/m2repo/cml/inchi/0.1-SNAPSHOT/maven-metadata.xml Downloaded: https://maven.ch.cam.ac.uk/m2repo/cml/inchi/0.1-SNAPSHOT/maven-metadata.xml (2 KB at 1.0 KB/sec) Downloading: https://maven.ch.cam.ac.uk/m2repo/cml/cmlxom/2.5.1-SNAPSHOT/maven-metadata.xml Downloaded: https://maven.ch.cam.ac.uk/m2repo/cml/cmlxom/2.5.1-SNAPSHOT/maven-metadata.xml (255 B at 0.7 KB/sec) Downloading: https://maven.ch.cam.ac.uk/m2repo/org/restlet/jee/org.restlet/2.0.0/org.restlet-2.0.0.pom Downloading: http://repo1.maven.org/maven2/org/restlet/jee/org.restlet/2.0.0/org.restlet-2.0.0.pom [WARNING] The POM for org.restlet.jee:org.restlet:jar:2.0.0 is missing, no dependency information available Downloading: https://maven.ch.cam.ac.uk/m2repo/org/restlet/jee/org.restlet.ext.servlet/2.0.0/org.restlet.ext.servlet-2.0.0.pom Downloading: http://repo1.maven.org/maven2/org/restlet/jee/org.restlet.ext.servlet/2.0.0/org.restlet.ext.servlet-2.0.0.pom [WARNING] The POM for org.restlet.jee:org.restlet.ext.servlet:jar:2.0.0 is missing, no dependency information available Downloading: https://maven.ch.cam.ac.uk/m2repo/org/restlet/jee/org.restlet.ext.freemarker/2.0.0/org.restlet.ext.freemarker-2.0.0.pom Downloading: http://repo1.maven.org/maven2/org/restlet/jee/org.restlet.ext.freemarker/2.0.0/org.restlet.ext.freemarker-2.0.0.pom [WARNING] The POM for org.restlet.jee:org.restlet.ext.freemarker:jar:2.0.0 is missing, no dependency information available

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 32.507s
[INFO] Finished at: Mon Mar 31 18:26:00 NDT 2014
[INFO] Final Memory: 7M/78M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project lensfield-jumbo: Could not resolve dependencies for project wwmm:lensfield-jumbo:jar:0.0.1: Failed to collect dependencies for [junit:junit:jar:4.0 (test), cml:jumbo:jar:5.5.1-SNAPSHOT (compile), cml:inchi:jar:0.1-SNAPSHOT (compile), joda-time:joda-time:jar:1.6 (compile), org.restlet.jee:org.restlet:jar:2.0.0 (compile), org.restlet.jee:org.restlet.ext.servlet:jar:2.0.0 (compile), org.restlet.jee:org.restlet.ext.freemarker:jar:2.0.0 (compile)]: Failed to read artifact descriptor for cml:jumbo:jar:5.5.1-SNAPSHOT: Could not find artifact wwmm:wwmm-parent:pom:2-SNAPSHOT in wwmm (https://maven.ch.cam.ac.uk/m2repo) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[Peter Murray-Rust]

Quick answers to some of your questions...

On Tue, Apr 1, 2014 at 6:14 PM, Oliver Stueker <ostu...@gmail.com> wrote:

Hi everyone,

We at the Memorial University (MUN) and University of Prince Edwards Island (UPEI) are working on creating a repository for (initially) results of QM calculations. Recently we came across Quixote and Chempound and we believe CML/Compchem & RDF would be a very good data-model for us.
Actually about a week ago one of our team members (Jason Pearson at UPEI) introduced our project to Prof. Murray-Rust via Skype.

I'm Peter :-)

 

My role currently is getting an overview what tools and standards have been developed for Quixote/Chempound and see which parts would be useful for our needs. Of course we will contribute any additions to the Chempound, Quixote and jumbo-converters (e.g. to support more QM packages for CML conversion) back to the projects.

After looking into things for the past two or so weeks I came up with a number of questions for which I hope someone here can help me to answer them.


1) First of all: Is the Quixote community still active or at least available to give me some pointers?

It's not active but we'll all try to answer questions. Maybe we might re-gel if there's enough need :-) I've explicitly copied some

 

2) Do the repositories on BitBucket [2a] contain the most up-to-date source code?

Not sure. It's probably as good as we can find. Sam Adams copied was a main architect and there might be something on his bitbucket site.

 

After building quixote-client and quixote-repository from source and depositing some of our files, I noticed that in the local Chempound repository most fields (Task, Basis Set, Formula, Point Group, Charge, Final Energy, etc.) are shown as unknown, although the data is present in the RDF (see below) and Package and Method show up correctly.

<rdf:RDF
    xmlns:gaussian="http://www.xml-cml.org/dictionary/gaussian/"
    xmlns:compchem="http://www.xml-cml.org/dictionary/compchem/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:cml="http://www.xmlcml.org/rdf-schema#"
    xmlns:owl="http://www.w3.org/2002/07/owl#"
    xmlns:dc="http://purl.org/dc/terms/"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
  <compchem:Calculation rdf:about="">
<!-- 120 lines deleted -->
    <compchem:basis rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
    >6-31G(d)</compchem:basis>
  </compchem:Calculation>
<!-- 7 lines deleted -->
</rdf:RDF>

Also on [2b] all values are displayed correctly.
Has the RDF schema changed?

 

Where can I find the chempound code in which the data to be displayed is extracted from the RDF graph?

 

Pass... 

3) I also noticed that on [2b] the RDF is also available in N3 notation whereas the version built from source only created the RDF/XML file.

Pass...

 

4) I was thinking to use lensfield2 for converting our Gaussian files to CML.
So I pulled lensfield from [4a] and tried to follow the tutorial [4b] but all I got is an error ( see [E1] below the links).
Also building lensfield didn't work. [E2]
Does lensfield only convert to CML or can it create the RDF as well?

You should use Jumbo-converters http://quixote.wikispot.org/JUMBO-Converters and links. We've recently hacked JC to have a command line approach as well as programmatic. Not completely finished - may be driven by what you want to do!

5) Chempound (the chemistry-jmol-plugin) uses an older version of Jmol (12.0.44) which is not working anymore due to security requirements imposed by Java 1.7.0_51 and newer in January 2014 [5a,b]. Luckily JSmol (a JavaScript version of Jmol) is getting more and more performant and so I've replaced the Jmol Applet in the chemistry-jmol-plugin with JSmol 14.0.11.
I will soon make the updated repo's available on BitBucket and will post a pull-request here.

Great! Feel a complete member of everything. If you need commit rights let me know 

6) Just as a note: When trying to download the compiled Chempound from [6a], noticed that the hudson at [6b] seems to be offline. Not a real problem as I went and just pulled the sources from bitbucket [2a] and built it myself.

We've got Jenkins working again at Cambridge - http://jenkins.ch.cam.ac.uk - if it's not been working there  for some time it's probably not active

 

7) Is the cml-discuss list still active? In the archives on sf.net I found a message from 2012 that the list would be moved to a Google group,  but I couldn't find it.

I don't think so. Egon has taking over CML schema and it may be a good point to reconfigure. You can always post to the Blue Obelisk list 

This is tremendous. Projects like these have a start-stop-start history and by persevering we are rapidly building Open Chemical software that is better than anything else. 

Thanks so much 

--
You received this message because you are subscribed to the Google Groups "Quixote project on QC databases" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quixote-qcdb...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Peter Murray-Rust
Reader in Molecular Informatics
Unilever Centre, Dep. Of Chemistry
University of Cambridge
CB2 1EW, UK
+44-1223-763069

[Jens]

(My original reply yesterday was rejected by the mailing list as I replied from the wrong address so I'm sending this again from the correct email).

Hi Oliver,

Good to hear that you're getting involved!

As Peter said the list has been quiet for a while, but I think most of us are still interested in seeing this area move forward.

I know that Marcus Hanwell at Kitware has been working in this area recently within the openchemistry project (http://www.openchemistry.org/), which you should have a look at if you're not already aware of it.

As to your questions regarding the code, it's a long time since I've looked at any of it (and I've just failed to get it to build on my laptop at first attempt which shows I'm a bit rusty) but I made some notes a while back on the structure of the compchem-specific parts of chempound which you can find here:

http://quixote.wikispot.org/Chempound#head-5c7f4ffb44aa6f11d7d77c3cfd015a76c94a9461

Hopefully they're still relevant and should point you at the bits of chempound that build up the RDF.

Best wishes,

Jens

[Oliver Stueker]

Hi everyone,

Thanks to hints of Peter and Jens I am now working on a nicer command line interface for running the legacy -> CML -> RDF conversion. (using the jumbo-converters GaussianLog2CompchemConverter )

For that I'm planning to include some other features like generating a 2D depiction using the CDK.

Unfortunately I'm stuck right now as the CDK (1.4.18) can't read the CML/compchem files (returns no molecules)

and I can't find a function in Jumbo or the Jumbo-Converters to read the CML file back into a  CMLMolecule

which I then could convert to a CDK IMolecule with one of the CDKUtils classes in jumbo-converters.

Can someone point me into the right direction?

Is there somewhere a place where I can browse the API of Jumbo, Jumbo-converters, cmlxom and other related packages? (generating the javadoc of the jumbo-converters myself with mvn javadoc:javadoc fails with an error (below)).

Thanks,

Oliver
-- 
Oliver Stueker, Dr. rer. nat.
Postdoctoral Fellow, Poirier Lab
Department of Chemistry, 
Memorial University of Newfoundland
St. John's, Canada

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.9.1:javadoc (default-cli) on project jumbo-converters-core: An error has occurred in JavaDocs report generation:

[ERROR] Exit code: 1 - /Users/stuekero/workspace_quixote/jumbo-converters/jumbo-converters-core/src/main/java/org/xmlcml/cml/converters/cmllite/CMLLiteHelper.java:125: error: unmappable character for encoding UTF-8

[ERROR] value = value.replaceAll("[&^%$�\"!(),;'/#~=+*{}\\[\\]]", "_");

[ERROR] ^

--

[Peter Murray-Rust]

On Thu, Apr 10, 2014 at 3:42 PM, Oliver Stueker <ostu...@gmail.com> wrote:

Hi everyone,

Thanks to hints of Peter and Jens I am now working on a nicer command line interface for running the legacy -> CML -> RDF conversion. (using the jumbo-converters GaussianLog2CompchemConverter )

Great,

we try to convert the Compchem output into structured CML which is conform ant to the Compchem convention. This arranges the molecules and annotates them so they are roughly independent of the generating program. Then a compchem reader will read them using a standard XSLT. It's a little while since I did this - it's in the compchem templates for various programs but an early stage.

For that I'm planning to include some other features like generating a 2D depiction using the CDK.

Great 

Unfortunately I'm stuck right now as the CDK (1.4.18) can't read the CML/compchem files (returns no molecules)

and I can't find a function in Jumbo or the Jumbo-Converters to read the CML file back into a  CMLMolecule

which I then could convert to a CDK IMolecule with one of the CDKUtils classes in jumbo-converters.

Can someone point me into the right direction?

Is there somewhere a place where I can browse the API of Jumbo, Jumbo-converters, cmlxom and other related packages? (generating the javadoc of the jumbo-converters myself with mvn javadoc:javadoc fails with an error (below)).

Thanks,

Oliver
-- 
Oliver Stueker, Dr. rer. nat.
Postdoctoral Fellow, Poirier Lab
Department of Chemistry, 
Memorial University of Newfoundland
St. John's, Canada

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.9.1:javadoc (default-cli) on project jumbo-converters-core: An error has occurred in JavaDocs report generation:

[ERROR] Exit code: 1 - /Users/stuekero/workspace_quixote/jumbo-converters/jumbo-converters-core/src/main/java/org/xmlcml/cml/converters/cmllite/CMLLiteHelper.java:125: error: unmappable character for encoding UTF-8

[ERROR] value = value.replaceAll("[&^%$�\"!(),;'/#~=+*{}\\[\\]]", "_");

[ERROR] ^

If we can work out what this character is we can replace it by its Unicode point. It'a probably something like an em-dash The simplest thing is to remove it and it's still likely to work. That's what I have just done. 

On Wed, Apr 2, 2014 at 9:22 AM, Jens <linu...@gmail.com> wrote:

(My original reply yesterday was rejected by the mailing list as I replied from the wrong address so I'm sending this again from the correct email).

Hi Oliver,

Good to hear that you're getting involved!

As Peter said the list has been quiet for a while, but I think most of us are still interested in seeing this area move forward.

I know that Marcus Hanwell at Kitware has been working in this area recently within the openchemistry project (http://www.openchemistry.org/), which you should have a look at if you're not already aware of it.

As to your questions regarding the code, it's a long time since I've looked at any of it (and I've just failed to get it to build on my laptop at first attempt which shows I'm a bit rusty) but I made some notes a while back on the structure of the compchem-specific parts of chempound which you can find here:

http://quixote.wikispot.org/Chempound#head-5c7f4ffb44aa6f11d7d77c3cfd015a76c94a9461

Hopefully they're still relevant and should point you at the bits of chempound that build up the RDF.

Best wishes,

Jens

--
You received this message because you are subscribed to the Google Groups "Quixote project on QC databases" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quixote-qcdb...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Quixote project on QC databases" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quixote-qcdb...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.