Loading XML file into semantic xml taking hours

[Tim Smith]

Hi,

I am attempting to open an xml file in TBC 6.4 beta on Windows 10.  The file is 840,000 rows and ~50MB on disk.  -Xmx is set to 28GB.  

TBC has been chewing CPU for over three hours.  Memory consumption fluctuates from ~5GB to ~8GB (up and down) so I don't think it is memory bound.

Is there anyway to tell if it's making progress?  Is this file too large to load?

Thanks,

Tim

[Rob Atkinson]

I'm not sure about whether it will work or not, but a warning that AFAIK if you leave an xml file in an open project in a workspace it is parsed on TBC startup - and this can take a long time too. It may only apply to XSD files, life is too short to check  this

[Tim Smith]

Thanks for the heads up Rob.  I'll remove it from the workspace before I restart TBC.  For now, it gets to run overnight.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/be68ffef-b8aa-4f74-ac9d-d86addd5150eo%40googlegroups.com.

[Holger Knublauch]

Yeah in general I wouldn't be surprised if a file of that size doesn't work well with semantic xml. Lots of layers of indirection and local memory. It would be best handled with some streaming parser, which we probably don't have integrated well enough. Are there any other tools you could use to preprocess the XML? For example define an XSLT to convert to RDF/XML?

Holger

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/CAF0WbnLreJOVe18Sr3U8Okx_7sj6ynak0kJ1HFqVORV7yLimpA%40mail.gmail.com.

[Tim Smith]

Hi Holger,

Well, it did finish after about 4+ hours.  I will certainly look at other tools.  Haskell was recommended and it certainly looks promising.  In the meantime, I saved the triples into an RDF file so I can analyze them tomorrow.  

If this works, I may need to do it on a regular basis.  The file is an xml dump of a ladder logic program.  I'm hoping to extract the data and alarm flows into and out of the PLC that runs the program and populate EDG.  This would be a big step towards automating the modeling of our OT layer.  I may also be able to capture and model the control loops.

Tim

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/010967b3-a2b0-0318-4d0b-62342dfe08d1%40topquadrant.com.