Thanks for pointing out the discussion about DCM and quaternions on diydrones, and for giving me a good excuse to talk on this subject.
1. The single largest source of attitude estimate error was gyro calibration. This will cause any algorithm, including DCM and quaternion, a great deal of trouble if they do not perform inflight auto-calibration of the gyros. I am not talking about gyro offset, I am talking about gyro gain, which can easily be off by 3%. At 500 degrees/second sustained rotation rate, this produces a 15 degree/sec error in attitude estimation. In the course of my research, I figured out a way to do the inflight auto-calibration.
2. The integrator in the complementary drift compensator creates a great deal of trouble at sustained rotation rates much above 50 degrees/second, and has to be turned off. This is an issue for either DCM or quaternion.
3. DCM can be made more accurate than the usual quaternion algorithm by doing a nonlinear, rather than linear computation. If you look in the rmat.c file of the latest version of MatrixPilot, you will see that there is a term in the update computation that is proportional to the third power of the rotation rate.
4. At high rotation rates, the latency of the magnetometer must be accounted for, otherwise there can be huge attitude estimation error.
Regarding quaternion versus direction cosine matrix, the real question is not which algorithm to use, but whether or not you need to use the linear or nonlinear representation of a rotation in doing the integration step.
Both quaternion and DCM enjoy the property that it is possible to exactly combine two rotations to produce the net result. In the case of DCM, this is done by multiplying two matrices.
For the integration step, in both quaternion and DCM, the update is performed by combining two rotations, in which one of the rotations represents the incremental rotation in one integration time step. In both cases, there are two ways to represent the incremental rotation. One way is an exact method that expresses the rotation in terms of a rotation axis and the trigonometric sine and cosine of the rotation angle. The other method is a linearization of the non-linear, exact, trigonometric expressions in which the cosine is approximated as being equal to 1, and the sine is approximated by the magnitude of the rotation.
In both quaternion and DCM, the linear form of the update will lead to small (much smaller than gyro calibration error) errors at high rotation rates. These errors can be eliminated by using the exact expression for the update rotation, or greatly reduced by using a Taylor's expansion of the trig functions.
In the latest version of MatrixPilot, sin(theta) is approximated as theta-(theta**3)/6, which produces much better accuracy than the linear quaternion update method. (Actually, there is another subtlety involved, which I explain in my report, so the correction factor is not 1/6.)
If you are going to use quaternions at high rotation rates, you should not use the commonly published versions of the update algorithms, which are linear, but rather you should use either the exact non-linear form, or use several terms in the Taylor's expansion.
For more information on the relative merits of matrices and quaternions, I suggest
this book.
The bottom line is that I do not see any clear advantages of DCM or quaternion per se, but you have to address several issues in order to achieve good performance at high rotation rates.
Best regards,
Bill Premerlani