SBAS related GPS timing jitter using an ublox M8T

[Thorsten]

Hi all,

by accident I came across some GPS timing jitter. This jitter - when the GPS is not constantly reporting at 200ms anymore - starts when the GPS status switches from 3 to 4.

I have tested it against JamInd, NoisePerMS, nSats, HDOP etc. but there is no relation. The only thing that has a delayed influence is the flight path. During the beginning of the second mission segment the status changes to 4 and the jitter starts. So the pitch angle seems to have some effect on the GPS signal reception - but only in the beginning of the flight.

Below you find a screenshot of some analysis in Excel (the xlsx file is attached as well).

The GPS Timing Offset is the difference in ms between TimeUS and GMS of two successive GPS entries, i.e. when there relative TimeUS and there relative GMS diverge.

There are some other interesting things to see like some drift and some strange systematic pattern in the jitter but in the first place, based on my experience with GPS timing jitter so far (visible twitches), I am wondering if anyone has an idea what might be going on. It seems to be more a M8T problem instead of an APM code problem. But what are the consequences - not using SBAS?

The related parameters I am using are:

GPS_MIN_DGPS = 100

GPS_SBAS_MODE = 1

GPS_MIN_ELEV = 10

GPS_INJECT_TO = 127

GPS_RAW_DATA = 5

GPS_GNSS_MODE = 67 

Has setting GPS_SBAS_MODE = 0 any influence on RAW data logging?

Thanks a lot in advance and best regards,

Thorsten

[Michael du Breuil]

I haven't had a chance to review the excel file yet, but it's not something I've noticed before or would have expected.

All I would have expected getting SBAS to have done (out side of position refinements) is provide you with one extra satellite when recording raw data.

[Jonathan Challinger]

One diagnostic action that may be useful would be to change the fix frequency to 10hz and 2hz and see what happens then. Also, maybe there are ublox messages relating to CPU load.

--
You received this message because you are subscribed to the Google Groups "drones-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drones-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thorsten]

Michael,

if RTK (DGPS) is used with a base station which records SBAS as well will there be any differences if SBAS is used on the Rover or not? I'm new to RTKLIB but I'll try to make some comparison where I exclude some sat systems (if possible).

Jonathan,

can you specify what fix frequency you refer to? I'd be happy to run some tests.

I'll check all other UBX messages again tomorrow.

I found a similar event in another log:

It's a little vague but there seems to be a relation to pitch. So rising the min elevation to 15° might help. 

I also had other strange event: After powering the copter I waited until GPS status was 4 (fast flashing green light). Then I armed and directly after take off the copter drifted away about 1.5 m and immediately returned back to its original position. I forgot about that event but found it scanning though some logs. This is the scaring result:

A complete loss of the GPS data. But within a few seconds all was fine again. If I remember correctly I armed immediately after GPS status reached 4. I also remember a similar event with a small position change when GPS status changed from 3 to 4 - but with an M8N. I'll check if I can find that log.

[Gregor Meyer]

Hi all,

First I must say I am no developer but I was the one who had a crash with the M8N discussed in this thread:

https://groups.google.com/forum/#!topic/drones-discuss/shCILZE_Nk0

In the past I saw several position issues, with my copter in disarmed state(position was about 10 to 50 m away from the real position, HDOP always below 1). The only thing I could do ist to power off the copter (and GPS) to solve this.

I am using the config file of Marco found one Github for the M8N.

After the crash I also looked into the GPS config and found a few things that can lead into problems from my personal experience:

MIN_SV Elevation: I think the minimum should be 10 maybe 15 would give a better stability.

USE SBAS correction data: This switch causes position problems with all auf my devices. Maybe you give this switch a try..(I think so SBAS is nearly disabled but if it give better results ...)

SBAS SYSTEM: on default its configured to OTHER with a huge number of satelites numbers. This does not work for me (Germany). Only if I enable EGNOS for Europe I got a DGPS fix.

SBAS fix time (I don´t know the real name at the moment). the value 0 enables searching for an unlimited time which is the best if you wan´t to be sure that it is used.

On other thing I saw is that flying only in low heights could cause jerky movements. If I fly in height about 20 m and then going low 1-2 m I did not get this movments (I ALWAYS wait several minutes after powering to get good GPS fix mostly I did not arm before HDOP is 0.8)

Maybe you are also interested in my log from the crash, so I attached it.

PS: sorry for my poor english ;-(

Cheers Gregor

[Thorsten]

Hi Gregor and all,

I had a look at your log. The GPS timing jitter continuously increased shortly before the crash. I all cases I can't find anything in the logs which seems related to an increasing jitter. So I tend towards a ublox issue.

Maybe, it is not a bad idea to report a jitter (especially if > 100ms) to the GCSs the same way as the new EKF and the vibration reports.

Cheers,

Thorsten

[Thorsten]

What is also strange is the systematic pattern in the jitter. Interestingly in your log there is period of relatively random pattern around the fist min of your flight. Maybe it is related to velocity/flight characteristics. I'll check that...

[Thorsten]

Hi all, 

ok, so the systematic vs. random pattern in jitter is related to NLon (number of long running main loops). NLon is a function of workload, which can be seen by it's relation to GCrs/flight dynamics (see screenshots below). This might be normal but it's interesting anyway especially, because I was not expecting to see something like this that clear in my log because it was not that windy and it was a survey mission at 4m/s.

But I am much more concerned about the negative influence of SBAS on the jitter which I can find in the logs. The flights were conducted around 37° latitude. I'll make some tests at 52° latitude to see if there is any difference regarding the SBAS reception. 

Any suggestion is highly appreciated. 

Best regards,

Thorsten 

[Robert Lefebvre]

Do I understand correctly that you are seeing consistent long loops?  Is this on AVR or STM32 system?  I wouldn't think we should be having lots of long loops on STM32?

[Thorsten]

STM32. On this one I am using an AUAV-X2. Not sure what Gregor is using but it must be a *hawk as well.

I never had a close look at PM.NLon so far, because I thought the STM32 has enough power. But the relation to flight characteristics is obvious on both systems - and it has an influence on the reported system time.

[Randy Mackay]

 

     We had a massive improvement in the CPU utilisation in Copter-3.3 vs 3.2.1 but there are still some mysterious events that are causing occasional long cycles.  Nobody’s really looked into it in depth but we suspect it’s I/O.  My personal guess is that it’s the logging to the dataflash but that’s an evidence-free opinion.

 

-Randy

[Thorsten]

I have a very fast SD card on board and there is no missing data (as far as I can tell).

Looking at the screenshots above, there is a clear relation to flight maneuvers. The I/O traffic should be the same - or? It is a simple survey mission and on each turn the NLon rises and wind direction also has an influence (or flying backward vs. forward, but the CG should be pretty good on this setup). Hence, I suspect some processing taking longer as expected in these cases. 

[Jonathan Challinger]

This is certainly not an APM problem, it has to be a Ublox issue. Take a look in AP_GPS_UBlox where we configure the sample rate. The GPS may not have enough CPU to compute a fix before it needs to start preparing the next one, hence the 100ms jitter.

[Thorsten]

I also think so - especially for the larger offsets. I already contacted ublox.

The NLon vs. flight characteristics is another story.

[Thorsten]

What would be the consequences for flight characteristics, EKF, etc. if the update rate would be reduced from 5Hz to 2.5Hz in APM? This should give the ublox more time to prepare a fix.

[Michael du Breuil]

It's possible, that the CPU load of applying the SBAS corrections would be to much, although I would be somewhat surprised if it couldn't apply one. I'm curious if you'll see the same behavior if you step down from a multi gnss solution to a single GNSS solution (GPS_GNSS_MODE 3 for example (only GPS and SBAS no GLONASS)) working on a single GNSS solution is easier then a multi gnss solution.

With regards to some of the earlier comments:

The biggest advantage of SBAS on the Rover with RTKLib is that it provides you with another satellite source to use for ranging corrections, as well as usually having a high elevation. The additional benefit is that it directly corresponds to a better solution in flight/a more accurate GPS position in flight. So from this perspective I try to use it as much as possible.

Raising MIN_SV elevation can be a very good thing, but there are tradeoffs with it. The higher the elevation mask the better signal you can usually expect to see, but at the same time you have less satellites feeding into the solution, and for some people raising the elevation mask reduces the number of satellites they have by to much to be able to fly successfully. (There is of course an argument about flying with poor satellite geometry or not)

Gregor, I need to rework the way SBAS is configured to allow you to configure the usage of EGNOS/other SBAS satellites from the ublox driver. Unfortunately I haven't yet found a good scheme that is both easy to present to the user and easy to implement. I'm currently leaning towards treating SBAS systems as a bitmask and enable all the satellites at once, and simply accept that while a bitmask isn't the most intuitive thing, only a small percentage of people will ever want to change it, and that they would be willing to figure out the correct bitmask.

[Jonathan Challinger]

If you want to reduce the GPS fix frequency you will need to modify the EKF to handle it. There's a constant in the EKF that is set to 200ms.

On Oct 8, 2015 11:51 PM, "Thorsten" <thorsten...@soilution.de> wrote:

What would be the consequences for flight characteristics, EKF, etc. if the update rate would be reduced from 5Hz to 2.5Hz in APM? This should give the ublox more time to prepare a fix.

--

[Thorsten]

Ah, ok!

Any volunteers? ;-)

No honestly, do you think this will have any negative impact on navigation? Maybe more for faster planes than for copters I assume. If not it might really be an option since the positional accuracy of the M8 is much better (at least in many regions of the world) compared to GPS only systems like the 6&7 models. 

Michael, I'll make some tests with different GNSS settings next week.

[Michael du Breuil]

It'd be really bad on navigation having seen the problems with it in other logs. I wouldn't recommend it at all, unless Randy/Tridge or Paul have something to say about having success.

[Michael du Breuil]

Finally got to look through my old flight logs I'm not seeing any correlation between number of satellites, sbas and timing jitter. I have a peak jitter of around 20 ms on my flight logs. I had SBAS almost the whole time with the flight, my settings where almost identical to what you have there except that I had GPS_RAW_DATA 1 (which is log every raw sample), and my minimum elevation wasn't set. This was with a NEO-M8T (http://www.drotek.fr/shop/fr/home/679-module-gps-ublox-neo-m8t-magnetometre-hmc5983-xxl.html) on ArduPlane

GPS_TYPE, 1

GPS_TYPE2, 2

GPS_NAVFILTER, 8

GPS_AUTO_SWITCH, 0

GPS_MIN_DGPS, 100

GPS_SBAS_MODE, 1

GPS_MIN_ELEV, -100

GPS_INJECT_TO, 127

GPS_SBP_LOGMASK, -256

GPS_RAW_DATA, 1

GPS_GNSS_MODE, 67

I also can't see the problem in my Ublox 6 logs.

Thorsten have you found the problem in any of your other flight logs? (I apologize if you've already answered this and I missed it).

On Tuesday, October 6, 2015 at 1:12:34 AM UTC-7, Thorsten wrote:

[Randy Mackay]

     I think slowing down the GPS is probably a last resort so I’d recommend not going down that route immediately.  Instead a good first step might be to look into whether the jitter is coming from the GPS or from the Pixhawk.  Perhaps try connecting the GPS to an APM2 or just a regular PC using uCenter and see if the jitter appears.  Another option (suggested by Michael) is to check if it appears on one of the less CPU intensive frames like Rover or Plane.  If it doesn’t then you can be almost completely sure that it’s a Pixhawk CPU or I/O issue.  If it does appear then it’s more likely that it’s something within the GPS.

                             

-Randy

 

From: drones-...@googlegroups.com [mailto:drones-...@googlegroups.com] On Behalf Of Michael du Breuil
Sent: 9-Oct-15 7:58 PM
To: drones-discuss
Subject: [drones-discuss] Re: SBAS related GPS timing jitter using an ublox M8T

 

It'd be really bad on navigation having seen the problems with it in other logs. I wouldn't recommend it at all, unless Randy/Tridge or Paul have something to say about having success.

On Tuesday, October 6, 2015 at 1:12:34 AM UTC-7, Thorsten wrote:

Hi all,

 

by accident I came across some GPS timing jitter. This jitter - when the GPS is not constantly reporting at 200ms anymore - starts when the GPS status switches from 3 to 4.

I have tested it against JamInd, NoisePerMS, nSats, HDOP etc. but there is no relation. The only thing that has a delayed influence is the flight path. During the beginning of the second mission segment the status changes to 4 and the jitter starts. So the pitch angle seems to have some effect on the GPS signal reception - but only in the beginning of the flight.

Below you find a screenshot of some analysis in Excel (the xlsx file is attached as well).

The GPS Timing Offset is the difference in ms between TimeUS and GMS of two successive GPS entries, i.e. when there relative TimeUS and there relative GMS diverge.

 

 

There are some other interesting things to see like some drift and some strange systematic pattern in the jitter but in the first place, based on my experience with GPS timing jitter so far (visible twitches), I am wondering if anyone has an idea what might be going on. It seems to be more a M8T problem instead of an APM code problem. But what are the consequences - not using SBAS?

 

The related parameters I am using are:

GPS_MIN_DGPS = 100

GPS_SBAS_MODE = 1

GPS_MIN_ELEV = 10

GPS_INJECT_TO = 127

GPS_RAW_DATA = 5

GPS_GNSS_MODE = 67 

 

Has setting GPS_SBAS_MODE = 0 any influence on RAW data logging?

 

Thanks a lot in advance and best regards,

Thorsten

 

 

 

 

 

 

--

[Thorsten]

Hi all,

I have 16 logs from that period - about half are from an M8T the other ones are M8N.

Some things I noticed scanning though all of them:

• in all M8N logs GPS status 4 was on from the beginning. Since the operation scheme was the same it seems the M8N is faster in fixing SBAS
  
• the jitter pattern are different. With the M8N there is a general drift between TimeUS and GMS. I have to check the random pattern in more detail, but it seems to be less depending on flight characteristics.
• what is most interesting is that the increased jitter with SBAS is accompanied by missing samples. I checked this before but made a small mistake...

The copters are pretty identical apart from the flight controller and the GPS (Drotek xxl M8T and AUAV-X2 vs. older Drotek small M8N and DroPix). The SD cards are both Transcend HC I 10. The AUAV-X2/M8T has higher vibes (< 20)  compared to the M8N (< 5). 

The question is what is the cause the GPS or the *hawk. I tend towards the GPS, especially because of the missing samples. Re the drift using M8Ns: I saw this before with original Pixhawks (reported somewhere here in the group).

So I am planning the following tests: 

• with and without of SBAS 
  
• replace the M8T with an M8N on the AUAV-X2 copter (I have both xxl version, so that is no problem)
  

Then the influence of the GPS and SBAS should be more clear. I am happy for any further suggestions for testing.

This is the missing samples related to SBAS on the M8T copter (50 means a missing sample at that time):

This is the jitter behavior on the M8N copter:

[Michael du Breuil]

If you don't mind could I get both those logs? There are a couple things I want to check.

[Thorsten]

Michael, sure.

I'll send you am PM tomorrow.

[Thorsten]

Hi all,

unfortunately I had no time so far to run further tests.

I am still waiting for reply from the u-blox support. However, I opened a discussion in the u-blox forum regarding this issue:

https://forum.u-blox.com/index.php?qa=826&qa_1=timing-issues-m8n-and-m8t-especially-when-switching-sbas-dgps

Best regards,

Thorsten