Magnetic Compass woes

[Darren Reis]

I need help.  I'm out of my area and digging around

The internet is full of users and developers complaining about the compass being wrong on their phone.  I've done a lot of experimenting and here is what i believe:

Android takes care of magnetic compass calibration of Hard Iron effects (biases) under the hood.  The OS is continually looking for axes-calibration motion (such as spinning on all three axes) and I believe it is automatically updating the calibration values.  It is also looking at the metal effects nearby and for this specific lat/long, and trying to compensate for the local area magnetic fields.  From the Developer-level API, I can see the SensorEvents and read the calibrated and uncalibrated magnetic field data.  From this, I verify that the calibrated data is almost always good (gives a correct heading compared to a physical compass I reference). However, on start up, the Galaxy S5 gets the heading wrong.  I notice the uncalibrated values (the hard iron biases), on start up, are all 0.  After doing the spinning, they go back to good values.  But for the initial start up, the compass readings are wrong.

 It is not clear which versions of Android, or which phones have this problem.  In fact, finding which system configurations have the problem is quite a headache - I know I've been Googling for days.  Like I said, its a nightmare to try and Google for this answer and then have to sift through pages and pages of user complaints.

Here is what we know: this problem affects the GS5.  The problem does not affect my Moto X, and it doesn't seem to be a problem on the GS3.


I've used multiple techniques to try and do my own magnetic calibration inside the app I'm writing.  Here are some links for doing magnetic calibration:
10 parameter estimation
4 parameter estimation

What I find is that nothing performs as well as the Android OS bias values [link].  I believe this makes sense because I am statically calibrating the device, in one location, whereas I think the bias values are calibrated online - constantly recomputed based on the local field.  This is great, but like it says in the link, the bias values are only good after update.  I'm in trouble right after start up.  I confirmed this by printing the bias values on my Samsung GS5 and seeing zeros on start up, with real values after I moved the phone around a bit.

My question is this:
Where can I go to find the hardware/software algorithm calculating these bias values?  Somewhere, either in Android, or a lower level, there is some auto-calibration going on to recompute the bias values.  My theory is that at very least it will be insightful to see what they do.  Perhaps I can store some values to carry over between power cycles that will give me a heading at least in the ballpark instead of effectively a random heading.

Thanks for your help!

[Ashutosh Joshi]

Hi, 

This is Ashutosh - and I work for the Android team. Sometimes  I play around with sensors !

You are right. Every Android device that supports a magnetometer keeps calibrating the magnetometer - at least when the magnetometer is active. This is needed because the magnetometer offsets can change depending on the environment that the device finds itself in. The referenecs that you have linked to are quite good and should be adequate to compensate the compass.

But - you should not have to do this. The Android framework should handle this for you. More specifically, every Android device is expected to conform to the some minimum specs. (See section 7.3.2  the Android CDD  for compass @ of http://static.googleusercontent.com/media/source.android.com/en/us/compatibility/android-cdd.pdf)

The bad news is that we cannot share the source code of such calibration on Android devices. These are capabilities that are built by the OEMs (Motorola and Samsung in your case) - or the partners that supply the sensor parts for these devices.  These algorithms are proprietary. While they are based on the work similar to what you linked to, they do add their own secret sauce at time to make the calibration perform better. 

Have you verified that the GS5 loses calibration between reboots ? If so, this would be a bug.  Do let us know.  Ideally - that is all you should have to do, this is not expected behaviour.

However, since you have put in some work already, and you may find this at least interesting, let me tell you about the sensor TYPE_MAGNETIC_FIELD_UNCALIBRATED. You are probably registering for the TYPE_MAGNETIC_FIELD sensor. 

The uncalibrated sensor type is exposed for more sophisticated developers who want to do something interesting with the raw sensor data. It does still require the underlying calibration to occur continuously though. The calibration parameters (biases in this case) are reported in the SensorEvent.

Sensor.TYPE_MAGNETIC_FIELD_UNCALIBRATED:

Similar to TYPE_MAGNETIC_FIELD, but the hard iron calibration is reported separately instead of being included in the measurement. Factory calibration and temperature compensation will still be applied to the "uncalibrated" measurement. Assumptions that the magnetic field is due to the Earth's poles is avoided.

The values array is shown below:

• values[0] = x_uncalib
• values[1] = y_uncalib
• values[2] = z_uncalib
• values[3] = x_bias
• values[4] = y_bias
• values[5] = z_bias

x_uncalib, y_uncalib, z_uncalib are the measured magnetic field in X, Y, Z axes. Soft iron and temperature calibrations are applied. But the hard iron calibration is not applied. The values are in micro-Tesla (uT).

x_bias, y_bias, z_bias give the iron bias estimated in X, Y, Z axes. Each field is a component of the estimated hard iron calibration. The values are in micro-Tesla (uT).

Hard iron - These distortions arise due to the magnetized iron, steel or permanenet magnets on the device. Soft iron - These distortions arise due to the interaction with the earth's magentic field.

You can save the calibration values persistently. The reported values must align with the calibrated sensor type. 

Hope that helps.

[Darren Reis]

Joshi,

Thanks for the response!

I understand the proprietary concerns, that makes sense.  And you're right, I would like the Android framework to take care of the calibrating for me, under the hood.  However, I believe there is a bug on the GS5.  I wrote a simple app that registered and listens to the UNCALIBRATED values, just as you posted.  I notice on startup, the bias values are all 0.  That's why the heading calculated from getOrientation is wrong.

If the GS5 has this bug, how can I be sure it won't be a problem on other phones?  The GS5 is one of the most used phones.  Plus, if you Google "android compass problem" (or similar), there are tons and tons of pages of people saying their heading is wrong.  As a good programmer, I'm paranoid about all components in my system and putting them together.  I can't let my user have a bad heading angle.  What else can I do? 

Thanks for your help, I'm so glad to have someone to call on.
Darren

[Ashutosh Joshi]

The compass is an interesting sensor. Its performance is affected by what is happening in the world around it. We live in a world where there are many sources of error.  Everytime you walk by a metal structure - or an electronic gizmo, the compass is affected.  Everytime your magentic environment changes, (say you walk from your office to inside your car), your calibration goes for a toss. 

Most of the problems that we see on the compass implementations fall into these groups. It is not a bug per se - but it does frustrate the user experience. 

A quick advice would be to look at the accuracy field reported on the sensor events. An accuracy value of 0 indicates that the device should be recalibrated. The underlying implementation may know that its calibration is shot, but it may not have enough data to recalibrate the device (hence the request to move the device in a circle/figure of 8 etc.) . Depending on your app, you may want to either ignore the magnetometer data in such cases - or perhaps prompt them to recalibrate the device. 

Another thing you may want to look at is to use the rotation vector sensor. This sensor can use the gyroscope and can tide over some of the problems that you see. If your app is an app that runs with the screen display on, your additional power hit should be small - and worth the improved accuracy. 

I would also be interested in hearing what thoughts you have on how this can be better handled in Android. 

Thanks for bringing this to our attention. 

-ashutosh

[Darren Reis]

Ashutosh,

I have tried both of your suggestions.  On the S5, on startup, the Uncalibrated Sensor bias values [elements 3-5] all start as 0s, and the accuracy value for the sensor is still 3, denoting these as good readings.  Thus, this is not a fool-proof way to check for the quality of the sensor event.  Without an accuracy warning, I won't know to prompt the user to calibrate.

Similarly, when I use the Rotation Vector sensor, the reported heading is wrong at startup.  Not until I do the figure-8 motion to let the phone recalibrate itself does the value show correctly. 

I am doing all my testing away from metals as best as possible, on a laptop away from things.  I understand the problem of sitting in a car, and I believe that is not the issues I see. 

At this point I'm not sure where to go next.  I have proved to myself there are conditions where I cannot trust android's reported heading.  I have gone back as close to raw sensor values as I can.  You have told me I cannot see the setup of the bias values for proprietary reasons.  That's fine.  But that means I can't find the problem.  I have been independently trying to build my own bias filter, like I mentioned before.  But like we said, this is stuff I think Android should handle for me. 

The worst part of this is I can rerun all of these experiments on my Moto X and I do not run into the same problems.  On start up for my Moto X, the calibration biases are present - not all zeroed like for the GS5.  Thus, through the raw sensor values and the composite sensor readings, I have not been able to repeatably find a condition where my Moto X loses heading calibration. 

Do you have any more suggestions for how I can further debug this issue?  Thanks for your help,
Darren