Ideas on how to parse Diviac dive profile?

[cnjdk]

Dear community

As per another post, I am desparately trying to salvage my diving data that was first trusted to Scubaearth.com and next to Diviac, https://logbook.diviac.com/home/. Even paid for an enterprise subscription but the service is showing signs of being in a death struggle (right now you cannot see details for your dives, the SSL certificate expired and it took weeks for it to be renewed. The uploader has not been updated for months and a lot more). So I am trying to migrate the data into Subversion. In that way I have some control of them. 

It turns out, Diviac saves dive profile data in a native format. I was able to export some of it before the service entered its current state of paralysis. It looks like this:

[

[0, 1.5, 20.0, ["surface"], null],

[10, 2.94, 20.0, [], null],

[20, 4.53, 20.0, [], null], 

[30, 7.27, 20.0, [], null], 

[40, 10.24, 20.0, [], null], 

[50, 12.56, 20.0, [], null], 

....

]

First param is <number of seconds into dive>, second is <depth in metres> and third is <temperature in degrees C>, fourth is some kind of event like "surface". Fifth, I do not know, and I do not need, it think.

I think it should be easy to translate that format into the XML format used by Subsurface:

  <depth max='3.62 m' mean='1.909 m' />

  <temperature air='20.0 C' water='19.0 C' />

  <extradata key='Serial' value='72870300' />

  <event time='0:00 min' type='25' flags='2' name='gaschange' cylinder='1' o2='32.0%' />

  <event time='0:00 min' type='9' name='surface' />

  <event time='1:07 min' type='9' name='surface' />

  <event time='2:23 min' type='8' name='bookmark' />

  <event time='3:59 min' type='9' name='surface' />

  <sample time='0:00 min' depth='1.52 m' />

  <sample time='0:10 min' depth='2.93 m' temp='20.0 C' />

  <sample time='0:20 min' depth='3.55 m' />

....

The question is this: Does anyone have a suggestion on what parser to use to interpret the format with the many square brackets used by Diviac? I could probably write a parser in Perl using Regular Expressions but if it is a well-known format then there is no point in spending a full day writing and debugging such a parser. 

Yours

Claus, Denmark

[Robert C. Helling]

Am Montag, 22. Juni 2020 03:59:52 UTC+2 schrieb cnjdk:

The question is this: Does anyone have a suggestion on what parser to use to interpret the format with the many square brackets used by Diviac? I could probably write a parser in Perl using Regular Expressions but if it is a well-known format then there is no point in spending a full day writing and debugging such a parser. 

Yours

Claus, Denmark

Claus,

this looks like JSON, so a few lines of perl should be sufficient (see attachment).

Best

Robert

 

[cnjdk]

My goodness, is that how JSON looks. Thanks a lot for pointing me in the right direction and also for the sample Perl script, Robert. 

Thanks a lot.

Yours

Claus

[Peter Clark]

Is there a final version of the script you used, and what were the exact steps you took?  I only just today installed Subsurface for the first time and would like to migrate my dives over from Diviac.  While I am using Windows, I luckily do have access to a Linux machine that I do by pure chance use to run some VERY basic Perl on.  My coding knowledge is all basic and extremely outdated, but at least what little coding I do is in Perl and PHP.

I have the CSV file from Diviac where the JSON part is just a single column.  Do I feed the entire CSV into a script, or just the JSON part?  If just the JSON part, how do I then put it back together with the rest?

[Robert C. Helling]

The script I posted above assumes just JSON as input. I you want you can email your diviac file and I'll take a look if there is any additional data that can be extracted.

Best 

Robert (hel...@atdotde.de)

[Peter Clark]

I just emailed the file.  Worst case I can just remove the JSON parts and run those through your script.  I just wanted to figure out the planned workflow.  Is the idea of the script that it then creates a separate XML file for each dive, or are they all still in one XML file?  I'm just not familiar with the specific formats wanted for import.  One can easily move other data over to an XML file manually if that is needed.

I also would love to see a clearer set of instructions for anyone who follows after me.  Diviac still has some users like myself, but if things go the way I expect, I easily see PADI trying to force them off by diminishing performance.

Thank you!

-Peter

[Robert C. Helling]

It shouldn't be hard to extend the script so it parses more of the fields of the CSV and produces an XML that Subsurface can digest. Given that I am a bit busy at the moment, this might take a few days. Will keep you updated.

Best

Robert

[Peter Clark]

No rush at all.  It will probably be Summer next time I get a chance to go diving, so I'm in no hurry at all.  I'm very interested in looking at the code once its down.  I'm a novice at a lot of this so will love seeing how it is done.

Thank you!

[Robert C. Helling]

Here is an extended version of the script that I used to parse Peter's CSV file. 

[Thomas O.]

Hi, I tried to run that script on my mac (afer installing Text::CSV). However I am still getting an error. Would you mind helping me out?

T-MacBook-Pro:~ thomas$ perl /Volumes/home/Backups/diviac/diviac.pl 

<divelog program='Diviac' version='42'>

<dives>

EHR - getline_hr () called before column_names () at /Volumes/home/Backups/diviac/diviac.pl line 22