Raspi Camera Calibration

[Pito Salas]

I have a little puzzle that I can’t solve so far. 

I am setting up to use opencv Aruco fiducials. For completeness and  in order to track down a bug I am trying to calibrate the raspberry Pi camera. There are many many write-ups and videos on this topic but as usual they are of wildly varying utility. Here’s the puzzle.

The camera is attached to a Raspberry Pi 4. The software is Ubuntu Server 20.04 (shell only!). If relevant I mention that ROS is installed but I believe this is done without ROS.

The calibration tool(s) require a series of images (of a chessboard or other targets) to be collected which are then fed into the algorithm to compute the camera intrinsic parameters and distortion coefficients. The tool(s) that I have found mostly guide you through the collection of photos and tell you if you need more etc. However they want to display the image as part of the algorithm. As the camera is attached to the Pi and the Pi is shell only, there’s no way to display the image there (I believe).

So I am stuck with the very clumsy way of using raspistill to blindly collect photos and then run a script which silently tries to process the images.

Any suggestions? Also critical is it to calibrate?

Pito

[Chris Albertson]

> On Feb 19, 2024, at 8:33 AM, Pito Salas <rps...@brandeis.edu> wrote:

> The tool(s) that I have found mostly guide you through the collection of photos and tell you if you need more etc. However they want to display the image as part of the algorithm. As the camera is attached to the Pi and the Pi is shell only, there’s no way to display the image there (I believe).
>
> So I am stuck with the very clumsy way of using raspistill to blindly collect photos and then run a script which silently tries to process the images.

Yes, Use X11 Forwarding. This will let the Pi4 run graphical software but. withthe GUI and mouse on your Linux based PC or on a Mac.

Use the “-Y” option when you log into the Pi. Like this

ssh -Y chris@RPI3HOSTNAME

Before you do this you will need to install an X-server on the computer you are sitting in front of. Also you need to disable All of its security options by using the command “xhost +”

I think the man pages for ssh and ghost explain better.

This is different from screen sharing. With screen sharing like VNC the Pi’s screen is displayed. But with X11 forwarding the app on the Pi4 opens a window on your PC just like it would on its own screen, it is had a screen. So you can run just one GOU app without installing the entire Ubuntu desktop environment.

[Mark Rose]

Hi Pito,

I had good luck with Aruco markers with one of my robots. I did do the calibration (the camera model is here: https://github.com/merose/tabletop/blob/master/config/raspicam_640x480.yaml), but noticed that the distortion found was slight. In particular, the center calculated was within a few pixels of the image center. Also, the Aruco markers detection works quite well even with the marker at an oblique angle, so you may not need a very accurate matrix. (Would probably make more difference if doing stereo processing, rather than just Aruco markers – though even there, I've found that it's sometimes better to generate the disparity map from the unrectified images and then adjust the distances afterward, especially with fisheye lenses. Though the rectification is supposed to put pixels on epipolar lines, I've found that the pixel distortion caused by rectification can mess up the disparity calculation.)

You might try with the model I linked to above and see if you get decent results. Or even skip rectification altogether. The RaspiCam does not have significant distortion that will mess up Aruco detection.

Mark

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CAMpcSKQJ_w7DHzLuzDYm0ddg4QJ0yAT72ZAMB27Fi%3DMagHQWuQ%40mail.gmail.com.

--

Mark Rose
mark...@acm.org
925.922.7263

[Pito Salas]

Chris,

Thanks a lot for this tip! I didn’t know about it and it’s just what I needed!

Pito

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/0DB513A5-F56E-4E4E-9FCD-68F424483D8A%40gmail.com.

[Pito Salas]

Mark,

Thanks a lot for this! It led me in the right direction, how to hand-make a camera matrix based on the resolution of my camera. So that got me unstuck. Next I will try Chris’ way of seeing the image from the PI using X11.

By the way, one tip that had me stuck. If you subscribe to the standard /cam/camera_info message you will get the matrix in the message, as a 9 component 1-d vector. You need to convert it to a 3x3 matrix otherwise when you pass it to the cv.aruco function you will get an inscrutable error.

By the way #2, for better or worse, ChatGPT is my constant companion but in that case it led me astray!

Thanks all!

Pito

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CAFAtoD08Ov_WQkqr2f%3DjrVa0ouyfd9JvM0tyoqmpnFFypmn3Zw%40mail.gmail.com.

[Chris Albertson]

Decoupling the physical display unit from the computer where the app is running was the entire point of X11.      This means you can run on any computer on your network and sit down at any desk in your office building and the user experience is the same.  There is no “remote access” because X11 is always client/server based, even if the client and server are running on the same machine. 

This is how we all worked in the 1980’s.  No, it was actually even better.  The user account data was on a file server.    And then came PC’s and lowered everyone’s expectations.   The primitive PC forces you to do everything (execution, data and display) on the same physical computer.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CA%2B%2B7fQxSDyzLw9pOQ1GNVHE%3DJrVFK6Ha2n2YjC7Tod%3D%2Bt0d_JQ%40mail.gmail.com.

[Marco Walther]

On 2/20/24 10:00, Chris Albertson wrote:
> Decoupling the physical display unit from the computer where the app is
> running was the entire point of X11.      This means you can run on any
> computer on your network and sit down at any desk in your office
> building and the user experience is the same.  There is no “remote

> access” because X11 is *always* client/server based, even if the client

> and server are running on the same machine.

Unfortunately, things are moving away from that somewhat with the GL,
Wayland and friends:-( 'The need for speed':-( But simple X11 apps still
work that way.

-- Marco

>
> This is how we all worked in the 1980’s.  No, it was actually even
> better.  The user account data was on a file server.    And then came
> PC’s and lowered everyone’s expectations.   The primitive PC forces you
> to do everything (execution, data and display) on the same physical
> computer.
>
>
>
>> On Feb 20, 2024, at 8:33 AM, Pito Salas <pito...@gmail.com> wrote:
>>
>> Chris,
>>
>> Thanks a lot for this tip! I didn’t know about it and it’s just what I
>> needed!
>>
>> Pito
>>
>> On Feb 19, 2024 at 2:19:21 PM, Chris Albertson

>> <alberts...@gmail.com <mailto:alberts...@gmail.com>> wrote:
>>>
>>>
>>>> On Feb 19, 2024, at 8:33 AM, Pito Salas <rps...@brandeis.edu

>>> <mailto:hbrobotics%2Bunsu...@googlegroups.com>.

>>> To view this discussion on the web visit

>>> https://groups.google.com/d/msgid/hbrobotics/0DB513A5-F56E-4E4E-9FCD-68F424483D8A%40gmail.com <https://groups.google.com/d/msgid/hbrobotics/0DB513A5-F56E-4E4E-9FCD-68F424483D8A%40gmail.com>.

>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "HomeBrew Robotics Club" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to hbrobotics+...@googlegroups.com

>> <mailto:hbrobotics+...@googlegroups.com>.

>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/hbrobotics/CA%2B%2B7fQxSDyzLw9pOQ1GNVHE%3DJrVFK6Ha2n2YjC7Tod%3D%2Bt0d_JQ%40mail.gmail.com <https://groups.google.com/d/msgid/hbrobotics/CA%2B%2B7fQxSDyzLw9pOQ1GNVHE%3DJrVFK6Ha2n2YjC7Tod%3D%2Bt0d_JQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hbrobotics/239F6BCF-F314-4AC4-B7BF-428D3249FE14%40gmail.com <https://groups.google.com/d/msgid/hbrobotics/239F6BCF-F314-4AC4-B7BF-428D3249FE14%40gmail.com?utm_medium=email&utm_source=footer>.

[Pito Salas]

Chris,

Re X11 forwarding on Mac. I am not getting to work. Maybe you see what’s wrong?

• Running XQuartz on Mac. Used settings to  turn off security
• 
  

• In Xquartz, I open a terminal window
• Ssh -X pi...@192.168.5.18 (also tried -Y)

• In that terminal window (now showing prompt for raspi)
• raspistill -o Desktop/image.jpg
  
• I see the expected 5 second delay and the image is saved but no window opened in XQuartz.

See anything?

Thanks,

Pito

[Sergei Grichine]

Pito, as a test, try "echo $DISPLAY" in that ssh window. If it is not set - "export DISPLAY=<your Mac addr or name.local here>:0.0" and then "xclock"

You should see the XClock app on the X11 emulator. If not - there's a problem with X11 emulator authentication or something.

There should be a "brute force" way to just start your favorite Desktop, when DISPLAY is already set correctly. Maybe somewhere in init or other settings, and definitely from the ssh command line.

I personally use VcXsrv on my Windows 10 machine, but not for a full Desktop - only to see specific clients' window (Gazebo, running on a Ubuntu I5 machine). Works very well.

There's a "Open session via XDMCP" option when running VcXsrv - if I remember correctly, that relates to Desktops etc.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CA%2B%2B7fQxSxGg%2BLfKnyxAcAwdNBkEqO84Kg4EONhMNG7cPOvwPEA%40mail.gmail.com.

--

Best Regards,

-- Sergei

[Chris Albertson]

Maybe try checking the second box in that box. It says “when disabled, connections for networked clients are not allowed”. I think you want it enabled. Otherwise Sergei already wrote what I would have.

[Pito Salas]

Hi Survey and Chris,

It worked: a combination of using a hard IP address, and not going through the VPN. One of those two did it. It worked with the second checkbox set or unset.

Thanks!

Pito

On Feb 22, 2024 at 12:01:30 PM, Chris Albertson <alberts...@gmail.com> wrote:

Maybe try checking the second box in that box.  It says “when disabled, connections for networked clients are not allowed”.  I think you want it enabled.   Otherwise Sergei already wrote what I would have.

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/E614DFDE-6473-4648-A1A7-996BCB0E32CD%40gmail.com.