Navigation with OAK-D IMU

[James H Phelan]

Team,

I'm working with the OAK-D on the stingray.

Since in has a built-in IMU, I thought I'd give it a try.

In Rviz2 its orientation is far from that of the robot.  (see attachment 1.)

I went a couple rounds in VSCode with CoPilot/Claude-Sonnet-4.5 which assured me that since the IMU published its orientation, the EKF would figure it all all so don't worry about the Rviz2 orientation.

However in practice, the robot appears disoriented and travels in the opposite direction. (see video attachment 2.)

Has anyone else used the OAK-D IMU?  Does it line up in Rviz?  How did you get it to do so?

Does it behave in action?  Any particular configuration tricks?

James

-- 
James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

[James H Phelan]

attached ekf_odom_params.yaml fyi.

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/75d6568b-6df6-4ad9-b547-8e3c57b247fd%40hal-pc.org.

[Marco Walther]

On 12/21/25 16:05, 'James H Phelan' via HomeBrew Robotics Club wrote:
> attached ekf_odom_params.yaml fyi.

Is the chain of transforms from the Oak-D IMU to the robot 'base frame'
correct? t looks like something believes , it's different from what it
actually is?!

-- Marco

>> hbrobotics/75d6568b-6df6-4ad9-b547-8e3c57b247fd%40hal-pc.org <https://
>> groups.google.com/d/msgid/hbrobotics/75d6568b-6df6-4ad9-
>> b547-8e3c57b247fd%40hal-pc.org?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/

> hbrobotics/9376ab5f-dde6-4038-a523-57cc8200860a%40hal-pc.org <https://
> groups.google.com/d/msgid/hbrobotics/9376ab5f-dde6-4038-
> a523-57cc8200860a%40hal-pc.org?utm_medium=email&utm_source=footer>.

[Sergei Grichine]

James,

I would rotate it in URDF till it aligns with the robot like on the following picture:

https://github.com/slgrobotics/robots_bringup/blob/main/Docs/Sensors/BNO085%20IMU.md#real-time-monitoring-of-imu-orientation-data

BTW, I don't know about Claude, but Chrome "AI Mode" is lightning fast and usually very helpful. ChatGPT.com is very slow, but often a good help too. Both are free.

Best Regards,

-- Sergei

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/13a49a35-8ba3-4d6c-aaf0-dc2f57789041%40gmail.com.

[James H Phelan]

see attached TF frames.  That's what it seems to me, but I don't see were.

James

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

[James H Phelan]

Sergei,

I could rotate the BNO055 axes to fit easily.  The OAK-D is different since it's integral to the camera.

I really don't understand why the OAK-D IMU axes aren't exactly the same as the camera except, perhaps, a final video frame rotation.  

I went through all kinds of hoops with VSCode/Claude trying to orient the IMU axes and it just took me deeper into the weeds.  Claude is usually the best coder and in VSCode has the advantage of being able to see, modify, run and evaluate the output of the code.  I may give another AI a try.

James

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVNKDNRsACLQWpRe%2BjUeD%2BOizoNJaqz%3DdyD%2Bb%3DP6VPKi_Q%40mail.gmail.com.

[Chris Albertson]

On Dec 21, 2025, at 3:43 PM, 'James H Phelan' via HomeBrew Robotics Club <hbrob...@googlegroups.com> wrote:

 with CoPilot/Claude-Sonnet-4.5 which assured me that since the IMU published its orientation, the EKF would figure it all all so don't worry about the Rviz2 orientation.

Don't believe what AI’s tell you without doing your own fact-checking.  My favorite AI story so far is when it gave me a detailed academic-sounding analysis of Beethoven’s 13th symphony.

You are going to have to get the IMU-to-Base transforms right.  EKF is not going to figure out the IMU is mounted upside down and backward all on its own.

This is the best video on the theory of rotation matrices I have seen.   It is part of a “101” intro course on robotics at ASU.  Once you know the theory, the ROS TF system is not so hard

The next video in the series extends the concept to Homogeneous Transformation Matrix and DH parameters.  You will need this later part for any robot with arms and legs, but the basic rotations are needed to move from camera or IMU to world coordinates.   The instructor makes this usually confusing subject really simple.  She’s a professional mechanical engineering instructor, not some random YouTuber; she’s good.

[Michael Wimble]

Having the IMU in the TF tree is good, but that graph doesn’t show you the orientation. Viewing it in rviz2 will show you if the 3 axis are aligned rotation-wise with the base_link frame. I’m not where I can do the experiment myself right now. But, if you need to rotate the OakD’s IMU frame, just add another frame to the URDF whose parent is the OakdD IMU frame, and add the needed rotation to this new, child frame. Then just have the EKF node read from that frame. Or does EKF get configured by topic names? If so, just create a trivial node to read the OakD imu data and republish it with the needed rotation. Personally, I’d add in Sergei’s new BMO sensor (85? 88?) and use that sensor, but the OakD IMU is already there and is already fused well.

[James H Phelan]

Michael,

The EKF just listens to the topic, I think, and, supposedly, that publishes the orientation and it's supposed to figure it all out according to Claude.

The first attached pic in the original message is of the orientation in Rviz.

The EKF .yaml is in the second message.

The AI tried adding links/frames and rotating them but nothing changed in Rviz.

The support for the BNO085 is pending reopening of the Fab Lab after New Years.

But surely (?) someone else has made use of the OAK-D IMU without having to write some convoluted translation node that Claude was attempting?

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

--

You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/55A192D4-B525-4CC9-BB13-2D946E826D84%40gmail.com.

[Marco Walther]

On 12/22/25 09:02, 'James H Phelan' via HomeBrew Robotics Club wrote:
> Michael,
>
> The EKF just listens to the topic, I think, and, supposedly, that
> publishes the orientation and it's supposed to figure it all out
> according to Claude.
>
> The first attached pic in the original message is of the orientation in
> Rviz.
>
> The EKF .yaml is in the second message.
>
> The AI tried adding links/frames and rotating them but nothing changed
> in Rviz.
>
> The support for the BNO085 is pending reopening of the Fab Lab after New
> Years.
>

> But /surely/ (?) someone else has made use of the OAK-D IMU without

> having to write some convoluted translation node that Claude was attempting?

You have seen (and handled) the following note from the docs?
"Note on Extrinsics
If your camera has uncalibrated IMU, a warning will be shown, and IMU
will be published with zero rotation and translation.
You can override this behavior by setting camera.i_tf_imu_from_descr:
true. This will publish default IMU extrinsics from URDF based on camera
model."

https://docs.luxonis.com/software/ros/depthai-ros/driver/

-- Marco

>> hbrobotics/55A192D4-B525-4CC9-BB13-2D946E826D84%40gmail.com <https://
>> groups.google.com/d/msgid/hbrobotics/55A192D4-B525-4CC9-
>> BB13-2D946E826D84%40gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/

> hbrobotics/f619d6eb-1a7a-4007-8c7c-b94999edb63e%40hal-pc.org <https://
> groups.google.com/d/msgid/hbrobotics/f619d6eb-1a7a-4007-8c7c-
> b94999edb63e%40hal-pc.org?utm_medium=email&utm_source=footer>.

[James H Phelan]

Thanks, Marco!

I'll have to dig into this.

James

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

[Sergei Grichine]

James,

Not sure what you meant by "The support for the BNO085 is pending reopening of the Fab Lab after New Years."

- but if you've got the device (https://www.amazon.com/dp/B0CL26J81F), the ROS2 driver and instructions are here: https://github.com/slgrobotics/robots_bringup/blob/main/Docs/Sensors/BNO085%20IMU.md

Best Regards,

-- Sergei

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/b9d66f73-9c90-4329-b5b8-454dea46587d%40hal-pc.org.

[Chris Albertson]

There is a two semester “Intro to robotics” class you can take as Arizona State Uniitverty (ASU) either for credit and get a grade or for free on-line. The instructor is the best I’ve seen.

Here is the main page where they lay out the full two semesters: https://www.robogrok.com/index.html

This is the part about rotation matrix: https://www.robogrok.com/1-1-3_Rotation_Matrices.php

[James H Phelan]

Sergei,

I mean the little piece of plastic that supports the IMU between the wheel axles. 

The screw spacing for the BNO085 is different than the BNO055 so I'm making a new one.

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVMLtRtgSXBB%2BmKanEYkLuKMp%2B6XABEhr%3D57fu%3D3AcxDRA%40mail.gmail.com.