Support for Gamrin Descent MK1

[Andrew Trevor-Jones]

I am hoping support for Garmin Descent MK1 is in the works.

[Adric Norris]

On Tue, Feb 6, 2018 at 8:54 PM, Andrew Trevor-Jones <atj777...@gmail.com> wrote:

I am hoping support for Garmin Descent MK1 is in the works.

I know that a colleague of mine is working on this, although I'm unsure how close it is to a usable state. I'll point him to this discussion, and see if he's willing to make a public comment.

--

"In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move." -Douglas Adams

[Steve Perry]

I am too!

I haven't bought a Descent MK1 yet but am very close. I'm a little hesitant because this is Garmin's first entry into dive computers.  Does anyone have any feedback?

[Adric Norris]

I like mine. While it's currently a bit rough around the edges, the device is pretty capable and has a lot of out-of-water utility... unlike any other dive computer that I've encountered. One thing I do wish it had is air integration, although that omission isn't too surprising for an initial product from a company new to the dive industry. Mine has been very reliable, and I think there's enough potential to be a real game changer a bit further down the line. Feel free to email me directly if you have any specific questions.

Full disclosure: I'm currently a beta tester for the Descent (neither employed nor paid by Garmin), and operating under a NDA. While I'm happy to discuss what I can, there are some topics -- such as commentary and/or speculation concerning future direction, unreleased firmware, etc. -- which I'll have to respectfully avoid.

On Sat, Jul 7, 2018 at 5:37 PM, Steve Perry <wald...@gmail.com> wrote:

I am too!

I haven't bought a Descent MK1 yet but am very close. I'm a little hesitant because this is Garmin's first entry into dive computers.  Does anyone have any feedback?

--
You received this message because you are subscribed to the Google Groups "Subsurface Divelog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to subsurface-divelog+unsub...@googlegroups.com.
To post to this group, send email to subsurface-divelog@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/subsurface-divelog/aac0b284-5117-438e-9369-d9b4297e20b9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Linus Torvalds]

On Tue, Jul 10, 2018 at 12:03 PM Adric Norris <landsta...@gmail.com> wrote:
>
> I like mine. While it's currently a bit rough around the edges, the device is pretty capable and has a lot of out-of-water utility...

The one thing I really like about it is the on-surface GPS capability
for entry/exit point. It's the one remaining feature I notice I'd
really like to become standard.

> One thing I do wish it had is air integration

Yeah, that kills it for me as a primary dive computer, but as a
secondary I guess it could be good. And as you say, for a
first-generation thing it's not surprising.

> Full disclosure: I'm currently a beta tester for the Descent (neither
> employed nor paid by Garmin), and operating under a NDA. While I'm
> happy to discuss what I can, there are some topics -- such as
> commentary and/or speculation concerning future direction, unreleased
> firmware, etc. -- which I'll have to respectfully avoid.

Since you're a beta tester, presumably you have some contact at Garmin.

Could you at least ask around if there is any possibility for
technical information on the download protocol? I realize it's a long
shot, but I'd be willing to use my own money to buy the thing (for
that GPS thing if nothing else), but I'd want to feel like there is
*some* chance of actually supporting it if I do.

And manufacturers *can* make it really hard to download from them. So
if this is a "no, we see this as a way to sell our own garmin connect
dive log software" dive computer, I don't think I'll bother even
looking at it.

Linus

[Adric Norris]

On Tue, Jul 10, 2018 at 2:18 PM, Linus Torvalds <torv...@linux-foundation.org> wrote:

Since you're a beta tester, presumably you have some contact at Garmin.

Could you at least ask around if there is any possibility for
technical information on the download protocol? I realize it's a long
shot, but I'd be willing to use my own money to buy the thing (for
that GPS thing if nothing else), but I'd want to feel like there is
*some* chance of actually supporting it if I do.

There isn't really a download protocol per se... the Descent simply presents itself as a USB mass storage device, and the activity logs (dives being one of several possibilities) will then be found in the Garmin/Activity subdirectory. The file naming format is simply "YYYY-MM-DD-HH-MM-SS.fit".

I'm not sure if the dive-related fields have been documented outside of Garmin yet, but will be happy to inquire. I do know someone who has been working on parsing out this information... he's unfortunately been very time constrained recently, but we'll be taking a class together in two weeks so I might be able to get some information then.

[Linus Torvalds]

On Tue, Jul 10, 2018 at 1:28 PM Adric Norris <landsta...@gmail.com> wrote:
>
> There isn't really a download protocol per se... the Descent simply presents itself as a USB mass storage device, and the activity logs (dives being one of several possibilities) will then be found in the Garmin/Activity subdirectory. The file naming format is simply "YYYY-MM-DD-HH-MM-SS.fit".

That may be very useful for the USB case, but there's definitely a
download protocol for bluetooth, since that seems to be how most
people actually interact with Garmin sport watches..

> I'm not sure if the dive-related fields have been documented outside of Garmin yet, but will be happy to inquire. I do know someone who has been working on parsing out this information... he's unfortunately been very time constrained recently, but we'll be taking a class together in two weeks so I might be able to get some information then.

Mind sending me an example file? Is it some nice almost-legible text
format, or just a raw database dump?

Linus

[Adric Norris]

On Tue, Jul 10, 2018 at 3:34 PM, Linus Torvalds <torv...@linux-foundation.org> wrote:

That may be very useful for the USB case, but there's definitely a
download protocol for bluetooth, since that seems to be how most
people actually interact with Garmin sport watches..

You're right, of course. I was thinking of how my friend was processing the data, and should have stepped back to view the bigger picture. I'll see what I can find out.

 

Mind sending me an example file? Is it some nice almost-legible text
format, or just a raw database dump?

I've attached a sample file. It appears to be some sort of binary format.

$ file 2018-01-06-11-47-38.fit
2018-01-06-11-47-38.fit: FIT Map data, unit id 84050944, serial 620756993, Mon May 16 04:25:43 2089, manufacturer 46015, product 65332, type 255

[Linus Torvalds]

On Tue, Jul 10, 2018 at 3:55 PM Adric Norris <landsta...@gmail.com> wrote:
>
> I've attached a sample file. It appears to be some sort of binary format.
>
> $ file 2018-01-06-11-47-38.fit
> 2018-01-06-11-47-38.fit: FIT Map data, unit id 84050944, serial 620756993, Mon May 16 04:25:43 2089, manufacturer 46015, product 65332, type 255

Yeah. At least it doesn't look encrypted or otherwise actively made
hard to parse.

It does look like there are some resources for FIT file formats:

https://wiki.openstreetmap.org/wiki/FIT
https://www.thisisant.com/resources/fit

and even some parser project:

https://github.com/mrihtar/Garmin-FIT/blob/master/README.md

and maybe that would "just work". Things like depth samples wouldn't
be *that* different from elevation samples when running or biking,
so..

But I do suspect it would take somebody pretty motivated and dedicated
to integrate it with subsurface (either through libdivecomputer, or
more likely as a "import" at least when it comes to the USB filesystem
interface files).

Linus

[Wojciech Więckowski]

Parsers for FIT format are ready to use. Format is very well documented and field names are self-descriptive. Root source for formats is here (FIT SDK):

https://www.thisisant.com/resources/fit

Amount of data recorded during activity is quite impressive. Current FIT SDK contains most of the records and fields used by Garmin in MK1 to record diving-related activities:

DC settings - units, time zone, hw/sw versions, deco model, ...

Gas settings - cylinders list with HE/O2 fractions

Dive settings - type (air/multi-gas/gauge), model conservativism, ppO2 limits (criitical/deco/warning), residual N2 and CNS, water type and density, safe stop settings

User profile - body data, gender, age, ...

Dive start and end records - GPS long / lat, air pressure, temp, heart rate, ...

Dive summary - typical data set, sample below

Diving record (1s resolution):

name: record

mesg_num: 20

    absolute_pressure: 166501 [Pa]

    altitude: 3981

    cns_load: 29 [percent]

    depth: 6.364 [m]

    distance: 0.0 [m]

    enhanced_altitude: 296.20000000000005 [m]

    heart_rate: 61 [bpm]

    n2_load: 98 [percent]

    ndl_time: 0 [s]

    next_stop_depth: 3.0 [m]

    next_stop_time: 30 [s]

    temperature: 15 [C]

    time_to_surface: 49 [s]

    timestamp: 2018-04-11 10:09:17

User profile

name: user_profile

mesg_num: 3

    activity_class: 50

    depth_setting: metric

    dist_setting: metric

    dive_count: 4

    elev_setting: metric

    gender: male

    height: 1.7 [m]

    height_setting: metric

    hr_setting: max

    language: polish

    position_setting: degree_minute_second

    resting_heart_rate: 0 [bpm]

    sleep_time: 23:00:00

    speed_setting: metric

    temperature_setting: metric

    user_running_step_length: 0.0 [m]

    user_walking_step_length: 0.0 [m]

    wake_time: 06:00:00

    weight: 86.0 [kg]

    weight_setting: metric

Dive summary:

name: dive_summary

mesg_num: 268

    avg_depth: 24.867 [m]

    bottom_time: 5114.713 [s]

    dive_number: 5

    end_cns: 70 [percent]

    end_n2: 79 [percent]

    max_depth: 68.884 [m]

    o2_toxicity: 104 [OTUs]

    reference_index: 0

    reference_mesg: session

    start_cns: 0 [percent]

    start_n2: 4 [percent]

    surface_interval: 82759 [s]

    timestamp: 2018-04-11 10:22:14

To access the data from fit files I'm using python-fitparse library with field mappings profile generated from the most recent FIT SDK headers.

https://github.com/dtcooper/python-fitparse

I'm attaching also Adric's fit file dumped to text format.

[Dirk Hohndel]

On Aug 6, 2018, at 6:31 PM, Wojciech Więckowski <xpl...@gmail.com> wrote:

Parsers for FIT format are ready to use. Format is very well documented and field names are self-descriptive. Root source for formats is here (FIT SDK):

https://www.thisisant.com/resources/fit

The license for the information and tools could possibly be problematic. I'm talking to Garmin about that.

I should also receive a loaner Descent watch - which should help me get motivated to work on an import model.

Amount of data recorded during activity is quite impressive. Current FIT SDK contains most of the records and fields used by Garmin in MK1 to record diving-related activities:

DC settings - units, time zone, hw/sw versions, deco model, ...

Gas settings - cylinders list with HE/O2 fractions

Dive settings - type (air/multi-gas/gauge), model conservativism, ppO2 limits (criitical/deco/warning), residual N2 and CNS, water type and density, safe stop settings

User profile - body data, gender, age, ...

Dive start and end records - GPS long / lat, air pressure, temp, heart rate, ...

Dive summary - typical data set, sample below

There are also quite a few fields in the Descent Fit files that don't appear to be documented (at least not in the version of the SDK that I downloaded a week or two ago).

To access the data from fit files I'm using python-fitparse library with field mappings profile generated from the most recent FIT SDK headers.

https://github.com/dtcooper/python-fitparse

Yuck, Python.

I meant to say, "oh, how interesting"...

My preference would be a direct importer, without any extra tools involved, but I'll have to play with all this a bit more to figure out if that is a reasonable path forward or if we use some sort of helper process.

Obviously, any help with this is appreciated.

/D

[Wojciech Więckowski]

As I'm too dumb to take part in Subsurface development but quite interested in getting data from MK1 to my existing logs I had two options: wait for someone to implement support for this computer or provide some kind of prosthesis.

Preparing attached piece of code took me last two nights, spent mostly on studying Subsurface xml format and mapping it to parsed FIT file content. As tomorrow I'm going to start diving at Red Sea so I wanted to have at least some working code for deeper profile analysis. 

Generally it opens existing subsurface log and adds dive and computer data imported from FIT file. It can be also used for (quite raw) records dumping. 

MK1 introduces itself as fenix5x - don't be confused.

Fortunately it seems that Garmin specific (undocumented) records don't contain any interesting data (at least at first glance). Maybe they are intended for their internal use. 

I had no time to provide GPS (semicircles) conversion and some kind of automatic creation of dive sites.

For now time zone conversion is absent and converter assumes that source data uses metric units. I have no access to sample imperial FIT profile to properly implement units conversion.

Now I’m going to collect some data by myself (upcoming vacation is a good occasion) to inspect some MK1 features like: gas switching, alerts, markers, surfacing events, repetitive dives...

XML parser I used has no pretty-print feature so output file is hard to read until it is opened and saved from Subsurface. 

Code is undocumented and its quality is poor (two nights work) but it works for me…

As parser requires python-fitparse with recent profile mappings I forked original repo just to update single file:

It can be cloned from here: https://github.com/xplwowi/python-fitparse

I’m attaching also quite complex source FIT file used in my tests.

-- 

Wojtek

[Wojciech Więckowski]

One more note. I didn't chech original procedures used in Subsurface for UIDs creation and temporary created my own version.

-- 

Wojtek

[Wojciech Więckowski]

And just to prove that import works with FIT file I shared earlier.

[Sébastien BOCK]

Thanks to Wojciech Więckowski's update on the python-fitparse project, i developed a small script to convert Gamin Descent MK1's FIT files to a more universal format: UDDF (http://uddf.org/)

UDDF is supported by almost all the Dive Log softwares including subsurface.

I tested it on a couple of dives, including Gauge Dive and it seems to work pretty well.

You can find the details here:

https://github.com/SPD13/Fit2UDDF

Honestly, I think Garmin should include the ability to export to UDDF as a standard...

[Dirk Hohndel]

That's really cool. We are working on a way to be able to import directly from the Descent, but this is certainly a way to get started in the meantime.

I'll definitely have to try this with the dives on my Descent and compare the result with the download from other, supported dive computers I had with me on the same dives...

/D

--
From my phone

[Dirk Hohndel]

Gave it a quick try with a file of mine - the result looks great, except that apparently it didn't find any GPS data in any of my dive files - even though the Descent reported storing the GPS locations. Looking at the same dives in the Garmin Connect app, they do appear to have GPS info there.

I played with the underlying parsing lib from https://github.com/xplwowi/python-fitparse and that doesn't appear to find GPS info in my .FIT files, either, so I'm guessing the problem isn't in your UDDF conversion.

Anyway, great job and thanks for posting this here!

/D

[Lionel Wolovitz]

Hi Dirk

I'm working on a .fit to .ssrf converter using the FitSDKRelease_20.72.00 C tools. Python certainly is a lot less verbose and easier - tempting... I am extracting the start GPS location but the exit location is not currently supported in the SDK. The data is there (obviously as it shows on Garmin Connect) and also using the java FitCSVTool to convert to .csv. Working on adding the required missing field information to config.csv. No problem extracting usual dive information, including stop_depth, stop_time, TTS, NDL.

Do you have any dives with gas switches? I'd be interested to see what events are produced as I doubt these are currently supported in the SDK. There are a number of events in logs that are not supported that I have not worked out yet.

Thanks

Lionel

[Sébastien BOCK]

The only GPS info I have found in the Fit file is start_position_lat and start_position_long and i put them in the node divesite=>site=>geography as recommended in the UDDF definition.

Not sure why Subsurface is not finding them...

[Sébastien BOCK]

may have to do with the unit conversion

this is what I have:

<geography><latitude>445670659</latitude><longitude>-1455372858</longitude></geography>

Where it's supposed to be real numbers.

[Lionel Wolovitz]

Looks like you need to convert from semi-circles to degrees:

 degrees = semicircles * ( 180 / 2^31 )

See https://docs.microsoft.com/en-us/previous-versions/windows/embedded/cc510650(v=msdn.10)

[Sébastien BOCK]

Thanks, you are right, i now have the right coordinates but subsurface is still not reading them.

I pushed the fix

[Miika Turkia]

send me a sample file and I'll take a look at Subsurface import

[Wojciech Więckowski]

Exit coordinates are semicircles coded and stored in msg #18 (session), in fields 38 and 39.

[Linus Torvalds]

On Tue, Aug 28, 2018 at 7:45 AM Wojciech Więckowski <xpl...@gmail.com> wrote:
>
> Exit coordinates are semicircles coded and stored in msg #18 (session), in fields 38 and 39.

Where do you find these things? The SDK doesn't seem to have any sane overview.

For example, I can find that yes, msg #18 is "session", but then I
find things like fields 3 and 4 for "start_position_lat" and
"start_position_long". I don't find 38 and 39.

Linus

[Lionel Wolovitz]

Thanks, now extracting start and end co-ords. BUT had to hack these in to the auto-generated files as SDK does not have support for these and the auto-gen supplied just seems to be to configurae what is supported, not add new fields. Clearly Garmin started using these new ones, but the SDK has not caught up. Need the tool that auto generates from Profile.xlsx. I'll ask the folks at thisisant. I assume you have the same issue for python-fitparse. No comments on using spreadsheets to configure software...

[Dirk Hohndel]

The complexity of this format and the design decisions behind the SDK are... interesting.

I appreciate the incredible breadth of devices that Garmin supports with this format and their desire to keep things compatible and to constantly expand the format. But yeah, deriving an SDK from an .xlsx file is... something.

It would be great if you could document the fields that you use in your tool - as I mentioned we want to integrate download from the Descent into Subsurface, so it acts like any fully supported dive computer and doesn't rely on a secondary app to convert the .FIT files. And any information that can make that internal parsing (as part of libdivecomputer) easier, would of course be useful to accelerate this work. To be clear - this will be C code that is NOT derived from the SDK itself, just from the information about the different fields.

Thanks

/D

[Lionel Wolovitz]

I intend to share what I have done when got it all working. But it is ugly. The only new data that would be useful to add to subsurface that I have come across so far is start and end location. I intend just adding these into notes as well as calculating the distance between the two. My thoughts are that location should be for the dive site (that does not change, e.g. wreck or reef), whereas entry and exit do. So a dive has three co-ords. The dive location would generally be curated by hand (but a good automated starting point could be the start / entry co-ords). 

Could be painful to develop your own fit parser in C from scratch...

[Linus Torvalds]

On Tue, Aug 28, 2018 at 11:02 AM Lionel Wolovitz
<lionel....@gmail.com> wrote:
>
> Could be painful to develop your own fit parser in C from scratch...

I'm working on exactly that for libdivecomputer.

I'm actually starting to get some data, and have a structure for the
parser definition that seems not entirely unreasonable. Fingers
crossed.

But I'm missing a lot of the message ID definitions (and I haven't
gotten to a lot of the field numbers within them).

Linus

[Wojciech Więckowski]

I just returned from vacation where I had possibility to supplement some missing events in my FIT files. I mean – forcing the alarms, violating MODs, breaking ceilings, etc. Nothing harmful of course...

I dived during the day and trying to find some time to code and analyze FIT files at night. The effect is quite obvious - a lot of new information gathered and a lot of chaotic code written.

I wanted to hardly refactor the script before showing it in public. Especially because current idea (collect the data and update Subsurface log on the fly) is stupid and causes a lot of conditional code. Additionally this is only temporary solution, created for my own needs, before support for Garmin is ready in Subsurface.

Today’s activity in this thread convinced me to share the code as is. Some developers may found interesting conditions in function “message_processor” as it contains all known for me, reverse engineered Garmin records and fields I needed to use. 

But after all - my script imports data from multiple fit files at once, creates new Subsurface logs or updates existing, supports sites, computers, cylinders creation, consolidates sites using defined radius around GPS coordinates, can include apnea dives, filters out dives shorter than defined time, lists FIT files summaries (to easy pick interesting ones), contains a lot of already recognized events…

Some extra data I had no chance to treat other way is imported to Subsurface Extra Info tab. The most important one for me is gradient factor setting that can be set per-dive in Garmin but only globally in Subsurface.

FIT is flexible and has consistent API. Unfortunately Garmin writes there a lot of data that is not documented. For instance, the last problem I had to resolve was time offset [s] stored in ‘device_settings’ message. It is required to calculate local time as internally Garmin uses UTC time. It turned out that it’s signed integer stored in uint32 field using “two's complement” format. I had to do a lot of tests before I recognized it.

Once (not for the current version) I tested the script also under Windows using python 3.7 and it worked fine. Among others, for this purpose there are a few changes in the code to substitute missing shell-level file wildcards expansion when called from shells as crippled as CMD.

If someone’s interested, please look at this repo:

https://github.com/xplwowi/fit2subs

You will find there also postulated source FIT files with GPS and diving activities inside. 

[Lionel Wolovitz]

Embarrassed look on face.

Forgot the company we're in. :)

[Lionel Wolovitz]

Hero.

Many thanks for sharing and your not inconsiderable efforts. I look forward to looking through what you have uncovered.

[Wojciech Więckowski]

Just a part of raw dump from my real diving activity

source fit:  https://github.com/xplwowi/fit2subs/blob/master/fit-files/2018-08-14-09-50-38.fit

dump: https://github.com/xplwowi/fit2subs/blob/master/dump-files/2018-08-14-09-50-38.txt

Description is missing in SDK docs but both fields are always present in logs. It's Garmin...

    total_work: None [J]

    training_stress_score: None [tss]

    trigger: activity_end

    unknown_106: 0

    unknown_107: None

    unknown_108: None

    unknown_109: None

    unknown_110: 1 mieszanka

    unknown_138: (0, 0)

    unknown_150: 28

    unknown_153: None

    unknown_154: None

    unknown_33: None

    unknown_38: 320217439

    unknown_39: 405641522

    unknown_78: None

    unknown_79: None

    unknown_80: None

    unknown_81: 0

 

[Linus Torvalds]

On Tue, Aug 28, 2018 at 12:17 PM Wojciech Więckowski <xpl...@gmail.com> wrote:
>
> If someone’s interested, please look at this repo:
>
> https://github.com/xplwowi/fit2subs
>
> You will find there also postulated source FIT files with GPS and diving activities inside.

Thanks, I'm definitely interested, because I'm getting to the point
where my FIT parser skeleton code is ready to actually maybe become
more than a skeleton.

And you shamed me into actually pushing my (INCOMPLETE!) work out too.

It's the 'garmin-descent' branch of

https://github.com/torvalds/libdc-for-dirk.git

and the way it currently works is that you build this version of
libdivecomputer together with the current git version of subsurface,
and you can download directly from your Garmin Descent.

And by "download directly" I mean "not really download at all, but
test". Because right now it doesn't actually fill any dive information
with the data, it just prints it out for my debugging purposes.

So I just capture 'stderr', and see things like this:

FILE_serial: ecee9feb
FILE_creation_time: 35de2636
FILE/7: ffffffff
FILE_manufacturer: 1
FILE_product: a2c
FILE/5: ffff
FILE_file_type: 4
FILE_CREATOR/0: 0104
FILE_CREATOR/1: ff
EVENT/253: 35de2636
EVENT/3: 00000000
EVENT/15: ffffffff
EVENT/0: 00
EVENT/1: 00
EVENT/4: 00
DEVICE_INFO/253: 35de2636
DEVICE_INFO/3: ecee9feb
DEVICE_INFO/7: ffffffff
DEVICE_INFO/8: ffffffff
...

where things like "FILE_serial" means that it actually has figured out
that it's the serial number field (although I have no idea what the
rule is for turning the value into a real Garmin serial number), but
something like "DEVICE_INFO/3" just means that I haven't even filled
in the data for field 3 for the DEVICE_INFO msg.

The parser is probably broken, and currently it doesn't even try to
handle the compressed record format because I've not actually seen one
yet.

And as mentioned, it doesn't actually fill any dive data at all, so
just do a "download" and then "cancel" after you see the dive list.
Don't actually accept it, you'll just get empty data. It's the debug
printout that is the interesting part right now.

I'll take a look at your fit files and your record numbers at some
point, but I need to take a break from looking at FIT files for now.

Linus

[Linus Torvalds]

On Tue, Aug 28, 2018 at 12:46 PM Wojciech Więckowski <xpl...@gmail.com> wrote:
>
> Description is missing in SDK docs but both fields are always present in logs. It's Garmin...

Heh. It sounds like you've worked with the format before. I never
have, so didn't know what to expect.

I've spent a few days trying to really figure out what the rules for
the format are, since I wanted to get this merged natively with
libdivecomputer, and so perl doesn't really work (and the SDK seems to
have some truly disgusting "generate crazy C" module that I refuse to
even look at, but found the header file more convenient than most of
the other sources of actual message and field numbering data).

Linus

[Wojciech Więckowski]

W dniu wtorek, 28 sierpnia 2018 21:56:40 UTC+2 użytkownik Linus Torvalds napisał:

On Tue, Aug 28, 2018 at 12:46 PM Wojciech Więckowski <xpl...@gmail.com> wrote:
>
> Description is missing in SDK docs but both fields are always present in logs. It's Garmin...

Heh. It sounds like you've worked with the format before. I never
have, so didn't know what to expect.

Nope, never touched this. Even MK1 I own three months lost its virginity two weeks ago.

But after all it's very good piece of HW. 

I found that SDK docs often referes to Profile.xlsx. I suppose that SDK headers generation is automated by parsing Profile's underlying XML.

[Linus Torvalds]

On Tue, Aug 28, 2018 at 12:52 PM Linus Torvalds <torv...@linux-foundation.org> wrote:
>
> It's the 'garmin-descent' branch of
>
>     https://github.com/torvalds/libdc-for-dirk.git
>
> and the way it currently works is that you build this version of
> libdivecomputer together with the current git version of subsurface,
> and you can download directly from your Garmin Descent.

Now the downloading actually works, and does something.

Wojciech, this is what your FIT data looks like when the current branch downloads it directly.

  

(although I don't know if images will actually show up correctly on the google groups thing).

I don't import everything - things like gas mixes and changes (and any events) are currently just ignored. 

But the basics certainly work.

                           Linus

[Wojciech Więckowski]

Wow - I'm impressed. I just strarted testing dctool compiled from your branch. In comment for commit modifying common.c I informed you that:
src/libdivecomputer.symbols has no symbol dc_usb_storage_open added.
It's not exported by libdivecomputer and linking dctools fails with undefined reference... error.

I'm reading your commits messages and observing code changes.
There are a lot of GPS coordinates stored in LAP records but I see no reason to support them as whole dive summary is stored in SESSION record. Laps are created after surfacing, spending there less than 1 minute (time can be defined) and immersing again. Laps don't affect sample timers, what is obvious as they are related to Garmin's epoch.

/WW

[Linus Torvalds]

On Wed, Aug 29, 2018 at 7:06 PM Wojciech Więckowski <xpl...@gmail.com> wrote:
>
> Wow - I'm impressed. I just strarted testing dctool compiled from your branch.

Ahh. I haven't used dctool in a long while. It's of dubious use, but I
guess for testing it's fine, and it's easier to build than subsurface
is.

> In comment for commit modifying common.c I informed you that:

Ok, you seem to have done it on the github comment system, so I missed it.

I don't actually use github that way - I have all notifications turned
off, because they are way too noisy for me. So I basically use github
as a hosting place, without ever looking at the other things github
offers.

And yeah, I have no idea how src/libdivecomputer.symbols is supposed
to work, I think it's some odd Windows thing, so I missed it entirely.

Is that the only symbol that needs adding?

> I'm reading your commits messages and observing code changes.

I'm done for a while. It's not perfect, but it is "good enough" for
me, and I'm traveling a bit the rest of the week.

So I might get back to it to finish up some odds and ends on the
weekend, but for now it is what it is. Comments welcome, but it's
probably best to just email me directly than to use the github comment
system. Or comment on github, but send me an email telling me to go
look.

> There are a lot of GPS coordinates stored in LAP records but I see no
> reason to support them as whole dive summary is stored in SESSION
> record. Laps are created after surfacing, spending there less than 1
> minute (time can be defined) and immersing again. Laps don't affect
> sample timers, what is obvious as they are related to Garmin's epoch.

As you'll find out, I wrote the parser basically by just looking at
what fields I could find in your and Dirk's FIT files, and then
looking at patterns.

I didn't even filter by activity, which is why it currently happily
creates a "dive" even from non-dive activity. It won't have any actual
*dive* data, but it can have random other data - like GPS points.

So there's definitely a fair amount of "cleanup and polishing" for
this to be a *good* Garmin downloader, but it seems to mostly work
already.

I'll be diving for a few days in a week or two, and then I'll actually
get to test the Garmin Descent myself. Right now I've just been
working off other peoples data.

Linus

[Dirk Hohndel]

A first test version that includes support for downloading (via USB, on a computer) from the Garmin Descent is available.

There are a few shortcomings, still, but I think it's reasonably functional already. The two biggest issues are

a) no events are downloaded - this is simply code missing in our parsing algorithm and should be added reasonably soon.

b) the downloader right now offers ALL activities, it doesn't filter for dive activities. So unless you remember when your dives started, you may end up with weird super short/shallow "dives" in your dive log. Again, something I expect us to fix pretty soon.

You can find binaries here: https://github.com/Subsurface-divelog/subsurface/releases/tag/continuous

For Windows simply grab the subsurface-4.8.1-...exe - that's an installer

For Mac we don't have a signed DMG as we do for releases, instead there's a Subsurface-4.8.1-...app.zip - which you can download, unpack, and then manually run (your Mac may complain about it being unsigned, depending on your security settings... worst case you may have to run the Contents/MacOS/Subsurface file directly

For Linux there's an AppImage there.

I'd be very interested in feedback how this works out for you.

Thanks

/D

[Linus Torvalds]

On Sat, Sep 1, 2018 at 12:13 PM Dirk Hohndel <di...@hohndel.org> wrote:
>
> I'd be very interested in feedback how this works out for you.

NOTE! If somebody finds oddities, parse errors, has some case that
doesn't work at all, etc, please do send me and the list

- a description of what the expected result is (ie "the Garmin app
says the max depth was 76 ft)

- a description of what subsurface did wrong (ie "subsurface reports
that dive as a 401 mile depth dive")

- the "Activity" file that matches the dive that got mis-parsed. Or
just all your activity files from the dive computer, and I'll figure
it out.

but as Dirk says, the current downloader is known to be incomplete wrt
event information and has that oddity of "all activities are recorded
as dives".

If you *do* have some event you care deeply about, and that you
actually know is there in the dive file, you can also send me that
kind of "this thing XYZ is important to me" information. It's likely
really easy to add certain event parsing, I just got fed up at looking
at FIT files, and my motivation to look at them was pretty low. BNut
if you point out particular events and have an example FIT file for
them, I'll be motivated to add at least those ones.

In a couple of weeks, I should have dive data of my own, and I'll
probably finish this off then.

And yes, while the downloader works, if you use your Garmin for other
things too, you will see silly 0-minute dives etc without a real
profile for things that are supposed to be
running/walking/cycling/whatever activities when you download.

For now, just uncheck them from the dive download list. They'll
usually stand out as being zero minutes long and having a zero depth
(although with swimming, I think you'll actually get a really boring
"dive" profile).

Linus

[Ryan January]

I apologize for not popping in sooner.  As Adric mentioned earlier, I did some preliminary work, but life got in the way and my code was never to a point where I felt comfortable releasing it to the wild.  Where I may be able to help from this point is documentation of the dive specific FIT file parameters.  I do have a contact that has (so far) supplied me with everything I've needed.  

If there's anything specific you continue to need please let me know.  In the meantime, I'll dig up all possible values for the dive specific fields and reply back with an update.

Thank you,

Ryan

[Wojciech Więckowski]

W dniu środa, 5 września 2018 17:59:52 UTC+2 użytkownik Ryan January napisał:

If there's anything specific you continue to need please let me know.  In the meantime, I'll dig up all possible values for the dive specific fields and reply back with an update.

Garmin didn't provide a reliable document describing diving-specific fields and their values. Definitions of some fields can be found in FIT SDK. But many others have already been recognized by analyzing FIT records and comparing them with real activities. They are supported by Subsurface when it's compiled from git sources. 

But there was no way to simulate critical violations (disregarded decompression, exceeded emergency partial pressure, ...). For me, the most interesting are combinations of 'event' and 'data' values stored in 'event' messages. Do you have any information about them?

For example, what is the meaning of data: 18, data: 9, data: 1, data: 5 in event #56? These combinations have been observed in multi-mix deco dives.

/Wojciech 

[Ryan January]

As Adric, I was a beta tester for the Descent and have had a closer relationship to Garmin engineers than most.  You're right their official SDK hasn't been updated in a while.  I was provided a document of all dive specific fields and possible values.  It was internal information which I believe was cleared for external release.  I'm in the process of verifying that now.  Once I get the all clear I'm certain we can get these questions answered. 

[Ryan January]

I just recieved the go-ahead to share the data.  Everything below this point is a direct copy and paste from his info.  

I'm going to use this format in this post:
Message:
number - Field [type] - Notes

At the start of a diving FIT file, you have several messages related to settings. Here are the dive-specific ones.
device_settings:
134 - tap_interface [switch] - On Descent Mk1 this is either AUTO or OFF

user_profile:
47 - depth_setting [display_measure] - Feet or Meters for depth
49 - dive_count [uint32] - Increments after every non-apnea dive. Connect can update this in the settings file. Not used on-device.

dive_settings (258):
0 - name [string] - Unused except in dive plans
1 - model [tissue_model_type] - Always 0 for Buhlmann ZHL-16C
2 - gf_low [uint8] - 0 to 100
3 - gf_high [uint8] - 0 to 100
4 - water_type [water_type] - One of fresh (0), salt (1), or custom (3). 2 is en13319 which is unused.
5 - water_density [float32] - If water_type is custom, this will be the density. Fresh is usually 1000, salt is usually 1025
6 - po2_warn [uint8] - PO2 * 100, so typically 140 to 160. When the PO2 starts blinking yellow
7 - po2_critical [uint8] - See above; value when PO2 blinks red and you get a popup
8 - po2_deco [uint8] - See above; PO2 limited used for choosing which gas to suggest
9 - safety_stop_enabled [bool] - Used in conjunction with safety_stop_time below
10 - bottom_depth [float32] - Unused except in dive plans
11 - bottom_time [uint32] - Unused except in dive plans
12 - apnea_countdown_enabled [bool] - This and apnea_countdown_time are the "Apnea Surface Alert" setting
13 - apnea_countdown_time [uint32]
14 - backlight_mode [dive_backlight_mode] - 0 is "At Depth" and 1 is "Always On"
15 - backlight_brightness [uint8] - 0 to 100
16 - backlight_timeout [uint8] - seconds; 0 is no timeout
17 - repeat_dive_interval [uint16] - seconds between surfacing and when the watch stops and saves your dive. Must be at least 20.
18 - safety_stop_time [uint16] - seconds; 180 or 300 are acceptable values
19 - heart_rate_source_type [source_type] - For now all you need to know is source_type_local means WHR and source_type_antplus means strap data or off. (We're reusing existing infrastructure here which is why this is complex.)
20 - [uint8] - device type depending on heart_rate_source_type (ignorable for now)

dive_gas (259) x6 - Index 0 is bottom gas:
0 - helium_content [uint8] - percent
1 - oxygen_content [uint8] - percent
2 - status [dive_gas_status] - one of disabled (0), enabled (1), backup (2)

dive_alarm (262) x? - User-set alarms:
0 - depth [uint32] - mm
1 - time [sint32] - s
2 - enabled [bool]
3 - alarm_type [dive_alarm_type] - depth (0) or time (1); use the appropriate field above depending on this value
4 - sound [enum] - off (0), tone (1), vib (2), or tone+vib (3)
5 - dive_types [array of sub_sport] - list of dive types this alarm applies to

Whew! Record messages come next. They only have a few extra fields.
record:
91 - absolute_pressure [uint32] - Pascals
92 - depth [uint32] - mm
93 - next_stop_depth [uint32] - mm; if you have a ceiling
94 - next_stop_time [uint32] - s; if you ahve a ceiling
95 - time_to_surface [uint32] - s
96 - ndl_time [uint32] - s; if you don't have a ceiling
97 - cns_load [uint8] - percent
98 - n2_load [uint16] - percent; represents all inert gas, not just N2

Throughout the dive, you'll see events. Here's the dive-specific ones.
event:
0 - event [event] - 56 is dive_alert, 57 is dive_gas_switched
1 - ignorable
2 - ignorable
3 - data [uint32] - for dive_gas_switched, this is the index of the gas that we switched to
for dive_alert this is a bunch of random things, though they generally correspond to the popups we show:
ndl_reached - 0
gas_switch_prompted - 1
near_surface - 2
approaching_ndl - 3
po2_warn - 4
po2_crit_high - 5
po2_crit_low - 6
time_alert - 7
depth_alert - 8
deco_ceiling_broken - 9
deco_complete - 10
safety_stop_broken - 11
safety_stop_complete - 12
cns_warning - 13
cns_critical - 14
otu_warning - 15
otu_critical - 16
ascent_critical - 17
alert_dismissed_by_key - 18
alert_dismissed_by_timeout - 19
battery_low - 20
battery_critical - 21
safety_stop_started - 22
approaching_first_deco_stop - 23

At the end of the dive or the end of a lap (Apnea and Apnea Hunt), we write a dive_summary message.
dive_summary:
0 - reference_mesg [mesg_num] - One of either 19 (laps) or 18 (sessions)
1 - reference_index [message_index] - The index of the corresponding lap or session
2 - avg_depth [uint32] - mm
3 - max_depth [uint32] - mm; minimum 0
4 - surface_interval [uint32] - s; time since end of last dive
5 - start_cns [uint8] - percent
6 - end_cns [uint8] - percent
7 - start_n2 [uint16] - percent; N2 and He
8 - end_n2 [uint16] - percent; N2 and He
9 - o2_toxicity [uint16] - OTUs from this dive
10 - dive_number [uint32] - Increments with every non-apnea dive; Connect can set the next dive's number via the dive_count user profile setting
11 - bottom_time [uint32] - s; Currently only important for apnea laps

There were a few notes that I put in my internal changelog for 4.00 that didn’t make it to the public one. Those are:

• FIT files will now be correctly marked as coming from Descent (2859) instead of a fenix 5X
  
• Travel gas is indicated in the dive_settings message as field 21. It is a FIT_MESSAGE_INDEX and is the index of the selected dive_gas message.
  
• If it is unset/invalid, the travel gas is the bottom gas (i.e. index 0).
• Previously, gas dives' bottom and total times were the same. Now bottom time excludes the end dive delay.
  

 

[Wojciech Więckowski]

czw., 6 wrz 2018 o 01:04 Ryan January <rjan...@gmail.com> napisał(a):

I just recieved the go-ahead to share the data.  Everything below this point is a direct copy and paste from his info.  

Perfect! Thank you!

Hm, it means that some events recorded during deep deco dive were critical... Fortunately computer was tested by the guy using CCR and all gas changes were virtual and in most cases forced by MK1 alarm states.

/WW

[Linus Torvalds]

On Wed, Sep 5, 2018 at 4:04 PM Ryan January <rjan...@gmail.com> wrote:
>
> I just recieved the go-ahead to share the data. Everything below this point is a direct copy and paste from his info.

Thanks. I've updated our parser, I think we do pretty much everything
now. Not that we necessarily care about all the things, but it looks
like we are fairly complete.

We could still generate some extra data (surface interval, OTUs etc),
so it's not like there aren't things we couldn't add to the parser.

Dirk - I've pushed out the changes to update the event descriptions
and parse the dive setting fields. It all seems sane with your and
Wojciech's dives

Linus

[Andrew Trevor-Jones]

I feel stupid asking this question...

How does one download dives from the Mk1?

I've upgraded to 4.8.2 on my Mac Mini (10.12.6) and Garmin doesn't show up in the list of Vendors under Import from dive computer

[Dirk Hohndel]

That sounds like a problem with my build environment…

Indeed, the wrong version of libdiveconputer was linked into the official installer.

I have updated the 4.8.2 DMG, please re-download :-(

/D

[Andrew Trevor-Jones]

Great, thanks.  It now comes up.

I like how you have the Entry and Exit points in the Extra Info.  Is there a way to copy the GPS data to the Clipboard?

[Andrew Trevor-Jones]

Would  it be possible to also add .FIT files to Import log files as a valid format?  The parsing should be the same and it makes it easier if you have already downloaded the .FIT file from Connect.

[Dirk Hohndel]

In a way it is.
Simply create a folder structure that the downloader understands:

<some path>/Garmin/Activities/<FIT files>

Then go to Download from dive computer, enter <some path>, and you should be able to import your dives. Which seems a lot easier than painstakingly use the import interface for every FIT file that you have...

/D

[Dirk Hohndel]

Turns out this is wrong as a good friend pointed out to me privately...

It's actually "<some path>/Garmin/Activitiy/<FIT files>" and the naming of the FIT files themselves currently matters, too.
We expect exactly 23-character filenames like

2018-08-20-10-23-30.fit

because that's the format the Garmin gives. That's a limitation that would be quite easy to remove.

It *should* work on Windows too. But it's possible that the usual "/" vs "\" confusion might have broken something - we have one report that this doesn't work as expected right now.

/D

[Andrew Trevor-Jones]

Ah... that explains why I couldn't get it to work.

After renaming the folder to Activity and renaming the FIT files it worked.  I could even get it to work using a symbolic link (as I already have the FIT files in another location).

It would be great if the filename limitation was removed.

[Linus Torvalds]

On Fri, Sep 21, 2018 at 4:26 AM Andrew Trevor-Jones
<atj777...@gmail.com> wrote:
>
> Ah... that explains why I couldn't get it to work.
>
> After renaming the folder to Activity and renaming the FIT files it worked. I could even get it to work using a symbolic link (as I already have the FIT files in another location).
>
> It would be great if the filename limitation was removed.

So it wouldn't be hard to relax the filename length limitation, but
right now we actually use the filename as the "fingerprint" for
libdivecomputer too, which means that it is what is used to determine
"you have already seen this fit activity" too.

That means that the original filename actually kind of matters.

Just out of curiosity: why did you rename the files when you archived
them? Because if what happened was something trivial like "I archived
them to a FAT filesystem that has a 8.3 name format", but the
filenames are still unique, we could _trivially_ just say "ok, the
filename is still the fingerprint, and it is _up_to_ 23 characvters in
length".

Something like the attached (UNTESTED!) patch should do it..

Linus

[Dirk Hohndel]

IIRC the problem is that if you end up downloading your FIT files from
Garmin’s Connect service, they come back with a different file name
that’s this random number that apparently Garmin uses to keep track
of the files (since different users could conceivably have Activities
with the exact same filename)

Whether that makes sense is a completely different question, but from
our perspective I think it means that we should allow completely different
filenames.

Question (since I haven’t thought this through)… what’s the consequence
of saying “if the filenames don’t fit the pattern, assume they are new” and
then offer all of them as available to import? I think this would work as
worst case it would show the user dives that are already in their dive list
and we should be doing the right thing if they get re-imported. But it would
trivially address this problem, right?

/D

[Linus Torvalds]

On Fri, Sep 21, 2018 at 9:55 AM Dirk Hohndel <di...@hohndel.org> wrote:
>
> Question (since I haven’t thought this through)… what’s the consequence
> of saying “if the filenames don’t fit the pattern, assume they are new” and
> then offer all of them as available to import?

There's no downside to that, it just makes the download (very
slightly) slower when we end up reading and parsing files that don't
actually get used.

With the garmin filesystem setup, the "slightly slower" isn't even
really an issue, since the accesses are so fast. The real advantage of
the fingerprint code is for things where the actual data transfer is
slow.

And it sounds like the garmin names are still unique, and are shorter
than 23 bytes? If so, the patch I posted should just
DoTheRightThing(tm).

Except since I didn't actually test it, maybe I screwed something up
despite how trivially obvious it all looks.

Linus

[Andrew Trevor-Jones]

I don't actually rename them as such.

Rather than have to connect my Mk1 to USB I sync the watch with Garmin Connect Mobile.  This puts the diving activity in the Garmin Connect cloud.  I then download the file as a .ZIP file from Garmin Connect web (.ZIP is the only option).  The filename of the .ZIP file (and the .FIT file contained within) is the activity number on Garmin Connect.

If I download this Dive Activity:

https://connect.garmin.com/modern/activity/3030954326

I get 3030954326.zip which contains 3030954326.fit

I guess Garmin want to be unique across the cloud as two people could have an Activity of 2018-09-21-10-23-36.fit

It will still be a fingerprint and even more unique than the YYYY-MM-DD-HH-MM-SS as it is on the watch.

[Linus Torvalds]

On Fri, Sep 21, 2018 at 1:27 PM Andrew Trevor-Jones
<atj777...@gmail.com> wrote:
>
> It will still be a fingerprint and even more unique than the YYYY-MM-DD-HH-MM-SS as it is on the watch.

Yes, that looks all fine.

I tested the garmin import with a renamed file and it seemed to work
fine, so I committed and pushed out the patch I posted.

Linus

[pprimoz]

Is it importing of Apnea and Apnea Hunt dives from Descent MK1 also implemented?

Subsurface 4.8.2 on my Windows 10 does not recognize these dives nor for direct import from Descent MK1 and nor as "<some path> / Garmin / Activitiy / <FIT files>" - I always get the message: Failed to parse "E:/ Garmin/Activitiy /2018-08-31-10-33-26.fit"

[Dirk Hohndel]

Can you send us sample .fit files so we can analyse them?

No need to post them to the group, just send to me directly

/D

--
From my phone

[pprimoz]

Sent to your eMail!

Dne četrtek, 04. oktober 2018 13.45.25 UTC+2 je oseba Dirk napisala:

[John Dayton]

I'm having a similar issue in thet I'm using Subsurface 4.8.3 and trying to import fit files from a "<some path> / Garmin / Activitiy / <FIT files>"  location and all I'm getting is a "Connecting to Dive computer" message that never goes away. Ideas?

Fit file is https://connect.garmin.com/modern/activity/3107130822

[pprimoz]

At the moment I am using subsurface-4.8.3-39-g432a32475604 (Dirk sent me a link to Windows version) which is working perfectly on both my Win7 and Win10 PCs.

I connected my Descent MK1 to PC via USB and downloaded all my dives (FIT files) from Garmin/Activity folder of my MK1 to my PC (in my case g:/Garmin/Activity). Than I started Subsurface, selected Import, Import from dive computer, Garmin, Descent MK1 and G:/ (Device or mount point). After clicking to Download I saw all the dives. The last step was to click OK and all my dives were imported into Subsurface.

[John Dayton]

That's exactly what I'm doing as well but it hangs on the connecting to computer, tried another Desktop with Windows 10 last night with the same results. Think I'll dig out my older Win 7 laptop today and see if I get a different result. Thanks for the input.

[Dirk Hohndel]

>
> That's exactly what I'm doing as well but it hangs on the connecting to computer, tried another Desktop with Windows 10 last night with the same results. Think I'll dig out my older Win 7 laptop today and see if I get a different result. Thanks for the input.
>

Can you be more specific what you mean by that?
When you connect the Descent Mk1 to your computer, does it show up as a "disk drive" with its own drive letter in file explorer?
If not, then I'd suspect an issue with your charging cradle.
If it does (so you can browse the files with file explorer), then can you explain what exactly "hangs"?
Normally I would expect the download to be extremely fast, near instantaneous once you enter the correct drive letter.

/D

[pprimoz]

Are you shore that

1 you are using Subsurface versions 4.8.3-39-g432a32475604 or 4.8.3-46-g0a24b6b48652 (and not 4.8.3!)?

2. your FIT files with recorded dives are in the  root/Garmin/Activity folder (and not deeper in the folders structure!)?

3. you selected Import from Dive computer (and not import  log files!) and Garmin as Vendor and Descent MK1 as Dive computer?

4. that as Device or mount point you entered only one letter (C, D, E, ..) : and / - in my case G:/ (like drive letter:/ and not drive letter:/../..!)

Try also to inserts a check mark before Force download all dives

Primoz

[John Dayton]

Yes to all but the first question. I could not locate the 2 builds you suggested but downloaded 4.8.3-57 off github and now they import just like they should. Thanks for the help.

[pprimoz]

http://subsurface-divelog.org/downloads/test/