Kplex to send AIS data over network to Furuno plotter

[Brett Marl]

Just getting started on a project using a RPi and dAISy hat to harvest AIS data. I have the RPi setup as an Access Point and kplex wrapping the serial input and exposing a TCP server. So far it's awesome - and I'm able to successfully connect an iPad with Navionics to the kplex server on the AP and suck in AIS data.

My next hope is to spit out AIS/NMEA sentences using kplex on eth0 of the pi which would be on the same network as a Furuno NavNet VX2 plotter. Hopefully emulating what the expensive Furuno (FA30) AIS receiver does. 

From poking around the interwebs looking at some people that have posted some wireshark dumps of the FA-30 traffic - it seems the UDP datagrams have some Furuno specific pre-amble in them (null terminated) before the AIS ASCII payload. 

Thought I'd ask here to see if anyone has had success using kplex to spit out regular AIS sentences in UDP and have a Furuno plotter read them, or if I need to kind of reverse-filter them an add the magic preamble somehow.

Oh - and thanks for making kplex. It's awesome, powerful and a necessary tool in the chain of DIY AIS :) 

[Brett Marl]

A little update on this, and perhaps a more focused question for kplex gurus..

I spent some more time with the networks sniffs and have determined that the UDP packets sent out by the Furuno FA-30 contains 8 preamble bytes before the NMEA sentence (eg: !AIVDM...) the bytes are a little tricksy as they contain some ascii nulls:

01 CD 00 01 00 00 00 00

The data is mostly static except for the second byte (0xCD) which is a sequence counter that is incremented for every UDP message that is sent out. Once it hits 0xFF it wraps around to 0x00. 

I'm guessing this was Furunos own version of TAG Blocks before is got standardized? 

Anyway... I think what I need to do is to have kplex PREFIX this sequence to every UDP output and also manage that sequence counter. I don't see any obvious way to do this using the filtering feature. 

I'd be really surprised if I was the first person to try and feed AIS data into a Furuno plotter with kplex, so am hoping this is solved somehow. Or maybe I'm over thinking it - and the plotter can handle UDP packets with and without the magic preamble. I guess I'll try non-prefixed messages first and see if anything shows up that'll be faster than trying to remember how to write in C and putting a pull request into Github ;) 

[Keith Young]

The nulls in that Furuno preamble used to be a problem for OpenCPN before they changed the parsing rules.  I've never had anyone raise the lack of preamble as a problem before, nor have I seen it as an issue on the OpenCPN forum (ie AIS being output through an OCPN data connection not being recognised) and but that's not to say the vx2 (or similar) doesn't need it.  As you say, I don't think this is a use case for the new external filter feature and as you may have guessed, the most obvious place (without looking in too much detail) might be as an alternative to the TAG code in output.

I'd say give just feeding non-prefixed AIS a go.  If that works we're done.  If not...if you can dig up more info on the navnet format (including what ports are used for what) that would be awesome.  I'd be happy to look at adding this as a feature if we had a good handle on what the rest of what we were adding meant.

[Brett Marl]

I was able to confirm today that the Furuno Navnet NX2 with latest firmware [if you can call 2012 "latest" ;)] does indeed require the "magic preamble" in order to accept incoming AIS sentences. I tried without it and nothing showed up on the plotter.  When I added the pre-amble - targets showed up and updated correctly. I wrote a little node script to listen to the kplex TCP server and spit out the UDP packets myself with the preamble including the 2nd-byte counter. 

The Furuno FA-30 broadcasts these packets to 172.31.255.255 on port 10021. I used the same broadcast address and port number. 

As I mentioned above, I'm not confident of the 8-byte block format - other than the sequence-counter in byte two - but it appears to be static and not a function of the NMEA payload. I did ask for the format on Furuno's forum but the response was pretty much "you need to be a dealer, and buy our SDK - then we can tell you that". Sadly I'm just a dude with a boat full of old Furuno gear - so don't think I can get access to the SDK.

I did have a good thread with an engineer from Yacht Devices about if their N2K->Ethernet converter supported N2K born AIS into Furuno plotters and they did not know about the magic preamble nor have any dealer relationship with Furuno.

Having said all that - I'm not sure it's a reasonable ask to add this to kplex without more understanding from Furuno of the byte sequence. I think my use-case is so specific that it's probably easier for me to just read the serial data myself from dASIy in node and spit out the UDP packets with the preamble.

[Keith Young]

Thanks for your understanding.  I'm not averse to putting vendor-specific stuff into kplex: I don't know if anyone ever used the Navico gofree stuff I put in but it's there and should save people digging the dhcp-assigned addres out of their plotter.  As you suggested, I'm less keen to put in a hack where I don't know what it means.  The actual implementation should be reasonably straight forward at first glance: put in an if clause in write_udp() (udp.c)  to replace the optional TAG block writing with the preamble and have a thread-local counter to increment the sequence number.  If I get time in the next couple of days I may try a proof of concept.  I'll also try a couple of forum posts to see if anyone knows more about the navnet format: keep me posted if you find anything

It's a shame vendors aren't more open.  Most of us would probably consider that an incentive to buy but maybe the boat hackers are an insignificant minority.

I tried to get Navico's SDK (to write to GoFree rather than just read) which supposedly they provide free on request, but they just never replied to my requests (3 over the course of 2 years).  I suspect that's more incompetence than deliberate business strategy

[Keith Young]

So no reply from Furuno to my query I sent them last week.  Forum posts seem to have zero replies.  So I launched into coding and suddenly realised that simply incrementing a counter with each packet might not be the right thing to do.  Brett was looking at AIS messages but is the counter incremented for each sentence, for each AIS sentence or for each sentence with any given formatter?  Or is it the case that it's only AIS which is sent out on that port (I recall magnet uses several ports for different purposes, although I don't know what).  Forgive me for backing out on this with "need more information" and if we find out the format I'll be happy to include it.

Again, this isn't overly complex to implement if it's just a per-packet-sent-out-by-one-interface counter.  It's just an alternate if clause which instead of adding in a TAG block adds in the header with the counter being stored in an extension to the per-interface if_udp structure.

[Brett Marl]

The only thing I have to go off is this attached PCAP file that was posted on a forum in 2014 with some kind of AIS receiver that is working with his NavNet. I assumed at first it was an FA-30, but after re-reviewing the wireshark I think it's actually the plotter echoing AIS data it's getting via serial 0183 onto the network port, either way - it is still in Furuno-flavor. 

The PCAP was downloaded from https://www.cruisersforum.com/forums/f134/opencpn-and-furuno-navnet3d-119421.html (See post #8)

I see two devices in the traces:

- 172.31.254.3 (Radar)

- 172.31.100.254 (I'm calling this "AIS emitter", but as above I now think it's the plotter)

The Radar broadcasts on port 10034.

The "AIS emitter" is broadcasting on many ports, I assume as different categories of data have different update rates?

10038 | GPMWV

10036 | HCHDG

10021 | [AIVDM] | [GPDPT, GPZDA, GPGG, GPGS]

Both devices also broadcast on port 10010 but these packets are weird - they have the preamble with a zero byte counter and then mostly binary data after. There is no ASCII NMEA payload that I see on this port, so I'm ignoring those.

The sequence counter in the Furuno preamble seems to be bound to an (IP-source, Dst-Port) pair and in incremented per UDP packet emitted regardless of how many NMEA sentences are inside the UDP payload.

Evidence for this:

- There are many cases where multiple GPS sentences are packed into a single UDP payload on port 10021 and the counter is incremented just once per payload. 

- There is only one cases I see where AIS message are packed into a single UDP payload - this is packet #783 where there are two AIVDM's and the counter is again only incremented once. 

- Port 10021 is either emitting payloads with both AIS or GPS data and the counter is incremented regardless of type. 

Also - across all of these packets (all ports and both sources) the rest of the Furuno preamble bytes remains constant. 

-- 

FWIW, OpenCPN has a flag you can set in it's config (EnableUDPNullHeader=1) that will allow ingestion of Furuno null messages. I was looking at what they did when that is set:

At https://github.com/OpenCPN/OpenCPN/blob/49a95f0b9b1c85ffc86e2b0c2d202bc7cfed3555/src/datastream.cpp line 394 it reads:

else{
// XXX FIXME: is it reliable?
// copy all received bytes
// there's 0 in furuno UDP tags before NMEA sentences.
m_sock_buffer.append(&data.front(), count);
}

so I'm not entirely sure they are confident of their fix either ;) 

Here's the script I'm using to pad the output of dAISy and it seems to be working so far with my temporary setup.

https://github.com/brettmarl/dAISyFuruno/blob/main/UDPEmitter.js

I'm going to wire the Pi in to my big AIS antenna and get a more permanent setup on the boat, so I can observe it's behavior for a few weeks and will report back with my findings. 

On Tue, Feb 2, 2021 at 7:21 AM Keith Young <strip...@gmail.com> wrote:

So no reply from Furuno to my query I sent them last week.  Forum posts seem to have zero replies.  So I launched into coding and suddenly realised that simply incrementing a counter with each packet might not be the right thing to do.  Brett was looking at AIS messages but is the counter incremented for each sentence, for each AIS sentence or for each sentence with any given formatter?  Or is it the case that it's only AIS which is sent out on that port (I recall magnet uses several ports for different purposes, although I don't know what).  Forgive me for backing out on this with "need more information" and if we find out the format I'll be happy to include it.

Again, this isn't overly complex to implement if it's just a per-packet-sent-out-by-one-interface counter.  It's just an alternate if clause which instead of adding in a TAG block adds in the header with the counter being stored in an extension to the per-interface if_udp structure.

--
You received this message because you are subscribed to a topic in the Google Groups "kplex" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kplex/laHZu7RKRV0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kplex+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kplex/95399ff9-be63-402d-af0f-4d3d98da1eden%40googlegroups.com.

[Keith Young]

The only thing I have to go off is this attached PCAP file that was posted on a forum in 2014

There's something remarkably familiar about one of those posters...;-) 

Yes I remember that.

FWIW no reply from Furuno and no reply to forum posts about the format.

Having multiple sentences in a single packet means the preamble couldn't work as a drop-in replacement for the TAG additions thanks to the "coalesce" option which bundles multiple TAG block + Sentences together in a single packet (whereas for this we'd want to write one furuno preamble for each packet).  Having sad that, thanks to this thread I've noticed some issues with that coalesce processing which I'm now fixing...

I may revisit this (especially if we get more info) but for now your solution looks good.

[Keith Young]

A reply from Furuno this morning.  I've restated the problem and how information on the format might benefit Furuno owners in interconnecting their equipment.  We'll see.  I was briefly wondering if this was actually a standard like IEC61162-450.  Whilst I have no visibility on IEC61162 (it's a "pay" standard so I probably couldn't use it even if I bought it) from what I've read it's a similar concept but multicast based whereas this is broadcast

[Brett Marl]

Update on my end - Adrian from WegMatt (dAISy maker) has a spare FA30 he’s sending me. I will get it hooked up and get a true FA30 sniff (vs the plotter-relayed one) and muck with settings to see if I can get the preamble to change at all. 

On Mon, Feb 8, 2021 at 9:21 AM Keith Young <strip...@gmail.com> wrote:

A reply from Furuno this morning.  I've restated the problem and how information on the format might benefit Furuno owners in interconnecting their equipment.  We'll see.  I was briefly wondering if this was actually a standard like IEC61162-450.  Whilst I have no visibility on IEC61162 (it's a "pay" standard so I probably couldn't use it even if I bought it) from what I've read it's a similar concept but multicast based whereas this is broadcast

--

You received this message because you are subscribed to a topic in the Google Groups "kplex" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kplex/laHZu7RKRV0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kplex+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kplex/7c6fcccf-44e8-4c0c-af86-03a066dfeac7n%40googlegroups.com.

[Brett Marl]

I was able to hook up an actual FA-30 and attached is the PCAP - there is a bunch of noise in it coming from other apps on the laptop but you can filter the FA-30 with ip.addr == 172.31.24.2. 

Some observations:

- The proprietary furuno format does seem to be there (as anticipated) and no setting I mucked with in the WebUI made it go away.

- It is actually broadcasting on two ports 10021 and 10033. Both AIVDO and AIVDM seem to go on both ports (but they are not a duplicate of each other)

- I noticed in the FA-30 settings it seems to be able to receive on two different channels. RX1/2 set to 2087/2088.  I wondered if it uses one UDP port for each channel or just randomly picks the port. I tried receiving on just one channel - but saw AIVDMs on both ports still. So I don't see any obvious order to it. 

- No settings I mucked with changed the seeming static bytes in the preamble payload.

- As observed previously the counter is bound to a specific port, i.e. each port is maintaining its own independent sequence counter.

Anyway, I'm pretty confident now that my Pi project is a good clone of the network output of an FA-30.  if you end up adding a flag to kplex or not, hopefully this thread serves as a good tombstone for some other poor soul that might be trying to do similar in the future ;)

Cheers,

-Brett

[Keith Young]

Thanks for this Brett.  Some further observations based on what you sent me:

- It is actually broadcasting on two ports 10021 and 10033. Both AIVDO and AIVDM seem to go on both ports (but they are not a duplicate of each other)

Mostly the same AIS data but not transmitted  as identical packets.  proprietary $PFEC,idfnc sentences are transmitted on 100021 and 10033, but $PFEC,AIcmd is 10033 only and the one AIALR sentence is 10021 only.  You also have 10010 which is broadcasting a mostly identical packet consisting of the same header as the other ports (0x01, one byte counter, then 0x00 0x01 0x00 0x00 0x 00 0x00) followed by 0x00 0x01 0x00 0x14 0x00 0x01 0x00 0x00 "AIS0" 0x00 0x00 0x00 0x00 0x01 0x00 0x00 0x00

- I noticed in the FA-30 settings it seems to be able to receive on two different channels. RX1/2 set to 2087/2088.  I wondered if it uses one UDP port for each channel or just randomly picks the port. I tried receiving on just one channel - but saw AIVDMs on both ports still. So I don't see any obvious order to it. 

2087 is 161.097MHz (channel A), 2088 is 162.025 MHz (channel B) and yes, we see both A and B sentences on both ports

Anyway, I'm pretty confident now that my Pi project is a good clone of the network output of an FA-30.  if you end up adding a flag to kplex or not, hopefully this thread serves as a good tombstone for some other poor soul that might be trying to do similar in the future ;)

I will stick revisiting this on queue of things to do and up the priority if the nice man from Furuno gets back to me 

[Brett Marl]

a couple more captures with more activity (for the archives). 

--

You received this message because you are subscribed to a topic in the Google Groups "kplex" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kplex/laHZu7RKRV0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kplex+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kplex/8bd5d46c-0a9c-45a4-9aef-c67cf3481538n%40googlegroups.com.