SLAM Camera Module

[Asad Memon]

Hi all,

Been a lurker for a while here but thought I would share my project I have been working on for about last 18+ months. I am building an embedded ultra-efficient camera module that does VIO on-the-chip using mono camera + IMU. 

The output from the board is pose @ 15Hz. Here is a video demo of then using the output pose to generate a map in realtime on host side.

https://youtube.com/shorts/gPK6ioYKE8E

(I hate shorts but youtube isn't letting me upload it as normal video)

[Chris Albertson]

This is good, Can you tell us at least a little bit about the theory of operation? It looks like you are creating a point cloud from video. So it is using a depth from motion algorithm? It seems the IMU allows you to estimate the baseline between frames. I’m guessing. Is there something you could point us to that explains how it works.

One thing, I hope it can do is split the video off so that it could be processed some other way to avoid the need for another camera.

How will these be distributed

[camp .]

> Here is a video demo of then using the output pose to generate a map in realtime on host side.

    What are you using to visualize this?

- Camp

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/db0ed313-0c29-4dc3-bdbd-00669a5656b7n%40googlegroups.com.

[Asad Memon]

> Can you tell us at least a little bit about the theory of operation

You are mostly correct. It is using a global shutter camera + imu. It’s close to classic SLAM methods ie. Frontend detects and tracks features, + imu is used to estimate scale of things and those go into an EKF like estimator.

The module outputs camera+imu streams with timestamps along with pose estimates. 

The mapping here is happening on host side using image stream + pose data to do motion depth analysis. 

The goal of the project is to provide temporal pose and then let you do further things (mapping, loop closure, depth) on host-side as needed. 

> What are you using to visualize this?

This is just pangolin but I have written my own web-based visualizer for general usage. 

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/5038188.55639.1780968099736%40mail.yahoo.com.

[Steve " 'dillo" Okay]

Cool! 

Is it a global shutter ? 

I see a Rockchip on there, which one ? 

Is this going up on Crowd Supply or Tindie or ????

'dillo 

[Stephen Williams]

One thing I need is inside-out tracking, i.e. a pose stream at up to 80fps or faster.  But a lower rate is still good.

In the cases I'm thinking of, a high-quality SLAM + pose would be available as input.  From a robot's head for instance.  Need inside-out tracking pose generation that is relative to that, such as for hands, limbs, etc.  The former can be somewhat expensive & bulky.  The latter needs to be inexpensive, low power, and as small as possible.  This looks like it is close to the latter, while also being a modest quality VSLAM for standalone use.

Stephen

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/CABiSWvNvxED4rARvuiWKRvh7WCed02pLDsK10h_hQDSYV5diQA%40mail.gmail.com.

--

Stephen D. Williams Founder: VolksDroid, Blue Scholar Foundation 650-450-8649 | fax:703-995-0407 | s...@lg.net | https://VolksDroid.org | https://BlueScholar.org | https://sdw.st/in

[Alex Sy]

Looks interesting.  You say that the board returns the image and the IMU info.  Does the board perform any image processing to detect and track features or does that also have to be done on the host PC?  What format is the image coming back, ie H264 or H265 or MJPEG?

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/CABiSWvNvxED4rARvuiWKRvh7WCed02pLDsK10h_hQDSYV5diQA%40mail.gmail.com.

[Asad Memon]

Cool! 
Is it a global shutter ? 
I see a Rockchip on there, which one ? 
Is this going up on Crowd Supply or Tindie or ????

Yes this is OV9281 global shutter. You are right about Rockchip, it's RV1103 which is very low power. 

I have preorder page up at mightycamera.com and I plan to ship them this month!

One thing I need is inside-out tracking, i.e. a pose stream at up to 80fps or faster.  But a lower rate is still good.

This is inside-out tracking but at 15fps. I think what you are describing is similar to Vive's lighthouse, which is pretty popular in VR scene.

Looks interesting.  You say that the board returns the image and the IMU info.  Does the board perform any image processing to detect and track features or does that also have to be done on the host PC?  What format is the image coming back, ie H264 or H265 or MJPEG?

Yup, the board detects and tracks features and returns pose data as well as image and IMU info. The image output is actually in RAW bytes (i.e. Gray8). So it can be consumed immediately without host-side decoding.