Debate on how to deal with multiple gnss sources.

[Brian Scally]

First off I want to be VERY CLEAR,  this is not a complaint.  I am asking /making suggestions.
I just finished a big project and so now have some time to really work on some stuff that interests me.. 

I am going to break my questions/solutions out into one per email.
--
QUESTION

Why is there only one /navigation/gnss branch?

My vessel has 6 GNSS sources.  4 on NMEA2k and 2 on ethernet outputting in 0183 format.
If you watch the GNSS data it swirl back and forth as each talker speaks on the networks.

I did fix it the problem with some gratuitous use of filters and some code.
BUT
I am not the only one more than one GNSS receiver that are not physically co-located.
I don't think I should have had to add 'custom' code for this problem.
 
So here  is my implementation method.

--
First some conceptual language based on my understanding of the data.

1- The Signal K data is arranged in a set of branches.
2- At the ends of the branches are petiole nodes and leaves.  The leaves are 'devices'.  The petiole nodes are the attachment point of the leaves to the structure.  Each leaf has one or more attributes. These attributes are the outside surface of the data structure.
3- Devices can be logical or physical.

--
I shall start at  /vessels/<RegExp>/navigation as the node point for my discussion

What I have moved to implement is..

The /gnss tree now includes additional petiole nodes 1 through # ( 6 in my case )
Each node is populated with the structure identified in the 1.5 specification and contains data from a physical GPS device.

The /gnss node is populated with data fused from each of the leaves.

The /gnss data that has an aggregate position calculated with a Kalmen filter which takes input from the distributed set of sense leaves after applying their stated positional offsets. .

The implementation is ugly and based on hard coded filters and some node red code that loops at update rate - about 40Hz.  But the result is a stable and highly accurate position.
Clearly if too many GNSS sources get added this would present a high processing load.

BUT the solution looks BUTT ugly.  And ugly is bad..  how to improve this?

Clearly I could write a plugin that provides a more elegant filter and calculation solution.

But, is this the correct way to approach the problem?

What other suggestions do people have and how have you all dealt with multiple sources feeding data?

Brian

[Thomas Sarlandie]

Brian,

If you look at the SignalK Delta spec, you will see that all updates have a source. The way your scenario would work is that multiple sources would send boatXXX/navigation/gnss branches and your Kalman filter "module"/"plugin" would collect them all and emit a new update with a new source specific to this filter.

How these updates are compiled into one Full Model is specific to the application. I think what the SignalK server does is keep the most recent one and keep track of the source which sent the data.

I am not sure whether the server comes built-in with a way to change this. That is a great question.

What you describe is another solution to that problem but I think it was decided early on that sources would be the way to handle multiple instruments sending the same data. The logic being that there is only one physical property being measured.

However, you will find that the way tanks are handled is very similar to what you describe. Each tank is an independent node with its own properties.

So I guess the way to handle your scenario is pretty clear in SignalK Delta model, but not so in SignalK full.

thomas

--

You received this message because you are subscribed to the Google Groups "Signal K" group.

To unsubscribe from this group and stop receiving emails from it, send an email to signalk+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/signalk/CACboiC91Ym43vpMaiB1XurD-%3D3%2B4stPn6onJS5%3D0v5XcvkJ8ng%40mail.gmail.com.

[Brian Scally]

Thomas

I totally agree with your statement and the signalk original hypothesis.

An example, it is possible to have data enter the server from multiple paths from the same sensor.  

I have this case where a N2K tank Sensor is reported directly off the can network and the data is reported by the Victron Gx.  Here having the data in one end node makes sence.

My inelegantly made point is that multiple GNSS recievers can not occupy the same space.

The physical sepration requirements for good reception mean they are outside sensing MOE.  They are measuring different things and so should be treated as multiple Leaves.

Mine are located at the corners of a 5m x 14m triangle.  So they are  clearly not the same reading. is the bow North or South of the stern ?  You read to know to properly fuse all the data.

I considered the approach of taking all the readings, and just adding back another one with a new "filtered" source ID. 

But as this and all the other gnss recievers are not generating the same  data it should  not be stored in the same leaf.

Thankyou for helping me refining my point.

Brian (Mub)

On Wed, Mar 17, 2021, 9:53 PM Thomas Sarlandie <tho...@sarlandie.net> wrote:

Brian

To view this discussion on the web visit https://groups.google.com/d/msgid/signalk/e4a6607e-7845-4a17-bc20-ad148261c462%40www.fastmail.com.

[Thomas Sarlandie]

Yes I think you have convinced me.

For boats where we have multiple GNSS sources deliberately placed in different point of the boat it would be good to have separate leaves (like we do for tanks or engines) and include some additional properties such as offset from a reference (bow, center, stern).

I am not sure this scenario was considered. You might want to open an issue for this and start brainstorming how we would change the schema to support it.

Maybe a good start point would be how does NMEA2000 handle it? 

thomas

To view this discussion on the web visit https://groups.google.com/d/msgid/signalk/CACboiC9_DtPhPhuTvEqj-46_P-2ODw1G3O9BhwNTsUqSxSUs%3DA%40mail.gmail.com.

[Brian Scally]

NMEA doesn't handeI it.  Each devices is set up to 1 gps source or picks it's own.

funny story of a seagull an autopilot and a pot of coffee.. Me motoring on go to Lat/long.  Seagull Lands on gps antennae.. Autopilot gets bum reading and goes to safety mode.. coffee every where.

I would not call NMEA a benchmark in this case.

will open a case with suggestion of how to handel.

Brian

To view this discussion on the web visit https://groups.google.com/d/msgid/signalk/edd81d97-0826-4c7b-bd44-85f776a52628%40www.fastmail.com.

[Teppo Kurki]

 

What you describe is another solution to that problem but I think it was decided early on that sources would be the way to handle multiple instruments sending the same data. The logic being that there is only one physical property being measured.

However, you will find that the way tanks are handled is very similar to what you describe. Each tank is an independent node with its own properties.

So I guess the way to handle your scenario is pretty clear in SignalK Delta model, but not so in SignalK full.

Mostly what Thomas described, except for the fact that the data from multiple sources are all available and can cohabit the same same leaf, under a property calles values.

You can simulated having multiple sources by running the server with bin/signalk-server -s settings/multiple-sources.json and there are a few tests that document how these are handled. This simulation results in a full structure like

"navigation": {
	"speedThroughWater": {
	"meta": {
	"units": "m/s",
	"description": "Vessel speed through the water"
	},
	"value": 3.143256351847165,
	"$source": "nmea2.II",
	"timestamp": "2021-03-18T06:20:24.191Z",
	"values": {
	"nmea1.II": {
	"value": 3.143256351847165,
	"sentence": "VHW",
	"timestamp": "2021-03-18T06:20:23.989Z"
	},
	"nmea2.II": {
	"value": 3.143256351847165,
	"timestamp": "2021-03-18T06:20:24.191Z",
	"sentence": "VHW"
	},
	"n2k1.115": {
	"value": 3.21,
	"timestamp": "2014-08-15T19:00:01.086Z",
	"pgn": 128259
	},
	"n2k2.115": {
	"value": 3.21,
	"timestamp": "2014-08-15T19:00:01.086Z",
	"pgn": 128259
	}
	},
	"sentence": "VHW"
	},

where you can see data from multiple sources present and addressable via http request path.

The $source value and the prop names in values are pointers to the /sources structure where you would store the specifics of the sensor, like its positioning.

If we start from the problem that you are facing and not from the hierarchical, single world view: if I understand what you are trying to do is observe data from multiple GNSS receivers and perform data fusion, for which you need their position relative to the boat?

I would do that this way:

- store the sensor positions under /sources

- connect to the server and subscribe to navigation.position for "self"

- whenever you see a new source in the data retrieve its position

- happily do your sensor fusion magic

You really don't need or want to care about  how the data is presented in the full model, except for the position data.

Here we have AGAIN fallen to the problems that we end up with when we look at a single hierarchical model. You can argue for different viewpoints on how things should be stored. It has proven very, very unfruitful over the years and we get into endless debates. I consider the single hierarchical data model (as opposed to different ways to query the data) a mistake in the SK 1.x design.

I urge everybody to think about things in terms of a stream of updates and accessing static data, like sensor positioning, synchronously.

As for the same data entering the server via multiple paths vs the sources being actually different: the server can not tell that they are the same data or not. How could it? The server needs to handle these in a generic way. How does it make the distinction that these are two separate sources? That they are the same source, or actually two receivers, 20 cm from each other - or 50 meters apart?

And just to be clear: I am not saying that the way things work today is the absolute optimum - partly because I don't think there is such a thing. The design choices we made were the best we could come up with. Some day we will have Signal K 2.0 or Next Gen or whatever, but for SK 1.x this is the way things work, for better or worse. You are very welcome to join working on the next version.

PS. Starting a discussion with the word Debate is kinda passive aggressive don't you think?

[Brian Scally]

Teppo

THANK YOU for the language comments.  Sorry my work keyword language crept in.
We use -
 Debate ~ I'm doing this ... challenge my ideas, I need to be sure I am doing it right.
 Discuss ~ Is this crazy?  I want to do this.
 Help ~ I want to do a thing but I have no idea how.
I am not at work so the language needs to change.

Second THANK YOU
For typing out the long and full reply.  I know these things take time to enter even if this is known content to you.

Yes you are correct in what I am trying to do, but I think that correction of my base view is more fundamentally important.

--
If I am understanding your words properly - ( the light bulb moment -much more thanks )- I will restate them here.

_A Postulate_
- We will never be able to build a tree structure that allows the representation of any of the systems where it is deployed, sure some sort of fit. But not really.
- So how do we represent the data?
- The data is simple and common, BUT the implementations are so vastly diverse and varied they cannot possibly be represented in a static tree.

That says that my paradigm for the data was reversed.
 -  What I am thinking of as paths of keys should be viewed as data-value descriptors.  Rather than the other way around .  The value is NOT a descriptor of the keys.
       This descriptor allows us to filter the data flow into the correct 'processing' pipeline.
  
This infers that the origin/primary access of the data is time - and as long as we are all good with back processing UTC to TAI there is no problem there for any data fusion.

The server preserves last state information of the sensors it sees.
  This is so that sensors with small delta value/delta time can have meaningful data when a sink subscribes.

  An example is a full water tank that is not being used.  If it was a pure delta view you would never know how much water was in the tank till there was a draw.
  One should be aware of this as things like an engine temperature sensor that may go offline when the engine is switched off.  The engine temperature will represent the last temperature of the engine till that data times out of the server.  But a simple check of reading time will resolve this.

The last light bulb moment:
  One should subscribe, then de-subscribe when one has finished consuming deltas.
  This may be for a single delta or years worth of deltas it doesn't matter.
  But the simple GET value method is orthogonal to the data model.
 

--
For my application:working with the new paradigm.
I'm thinking it consists of two processes.

1-
subscribe to the 'source delta flow'.
Recognise the source objects and add suitable physical descriptor information.

For sensor fusion this would be  a set installation physical descriptor fields, but this data might include an HRN and a few other pieces of information.

2-
subscribe to the 'values delta flow'
Filtering for the necessary information ( this may be in part of the subscribe )
Using the physical information from the 'sources delta flow' it would do some filtering and present the result back to the values delta flow.
It would appear in the 'sources delta flow' as a data provider.
--

Paradigm Future.

If we view the keys as descriptors of the values which are a temporal sequence.

Then values can be described with more than one key - they have many attributes like source and time, so this is only a small extension.

For example:

ON - can be described by //vessel/self/electrical/switches/bank/0/1/state AND //vessel/self/navigation/lights/anchor/state

This requires a user config, but we went and did the wiring.. and wrote the control code .. So maybe it gets added as part of the user description fields in the source?
I haven't got a model of how to add that.

Brian

--
You received this message because you are subscribed to the Google Groups "Signal K" group.
To unsubscribe from this group and stop receiving emails from it, send an email to signalk+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/signalk/CAOyOCECNEu2JxNifER-2moZJoP-DrLJoZ_%3D95f6HK%3DSoKUkaGw%40mail.gmail.com.