Fwd: MIAPA ETL update

[Elliott Hauser]

Dear NeXML and MIAPA communities,

Please find my weekly update below.

Additional updates:

• Rutger Vos will be joining the weekly project call tomorrow at 9:30 EDT.  I look forward to the benefit of his NeXML knowledge.
• With Rutger and Phillippe Rocca-Serra of ISAtools on the line, I hope to make concrete progress on what the logical mapping of NeXML to ISAtab should look like.  This is the dreaded double moving target: deciding what from NeXML belongs in ISAtab while deciding what ISA configuration is appropriate for NeXML.  The goal is to have a configuration/mapping ready for community input as soon as possible and iterate rapidly

As always, any input is welcome.

Elliott

Google Folder:

http://bit.ly/ehmiapafolder

Github:

http://bit.ly/ehmiapacode

________________________

Completed:

• Re-orgnaized entire project into .xsl files corresponding to NeXML elements, all imported into a master one
• More robust tabular output now, especially for <<otu>> elements
• Realized that tab delimited format requires de-normalizing to a certain extent
• i.e.I think  I have to repeat group metadata for each member of a group within ISAtab
• But the ISA configuration will determine this, so I'm going to shift to tht kinda.
• Now construing each <<meta>> element as a 'cell' in a table
• Able to correctly label column headers & align data

Not Completed:

• Verify ISAtab files.  As noted before, verification of ISAtab output requires a custom configuration.
• Integrate with ontologies.  This isn't planned, but I'm keeping an eye on it because I might need to.  Likely, though, ISA's functionality will be sufficient

Next Steps:

• (!) Update project plan & documentation (behind)
• Create simple shell script to pass variables to Saxon (almost done)
• Create simple ISA configuration for NeXML (in progress)
• Meeting with Rutger Vos this week or next RE NeXML content (email out)
• Continue to refine/tweak XSLT
• (next week or after) Simple jar interface combining Saxon and my XSLT.
• (next week or after) draft NeXML-MIAPA checklist for community approval
• (next week or after) incorporate draft NeXML-MIAPA checklist into ISA configuration

Basically, I've been learning about ISAtab and NeXML quickly to the point where the way I had the work broken up before doesn't work.  Also there are dependencies on things like the ISA configuration that I'm now factoring in to my schedule  I didn't get a chance today to update that plan, but it's first on my list tomorrow.  I also got clear guidance from Hilmar last week to make sure that I do simple prototypes early on to feel out any problems ahead of time.  Hence the shell script I wrote today & the planned Java.

Comments, questions and suggestions welcome.

E