My collected wisdom about cameras and raspberry pi 5 and ubuntu

[Pito Salas]

I wrote down everything I thought I understood about packages and dependencies after troubleshooting my simple Raspi 5 + Pi Camera 2.1 + Ubuntu 24.04 setup. I asked claude to organize what I had figured out, which is I am sure incomplete and incorrect in places.

Maybe this is useful and maybe you will send me some corrections.

- Pito


# Claude's contribution

## The Camera Stack on Raspberry Pi + Ubuntu

### Layer 1: Hardware and Firmware

At the lowest level, the Raspberry Pi has a dedicated **Image Signal Processor (ISP)** built into its GPU. The ISP handles raw sensor data processing—demosaicing, white balance, auto-exposure, and noise reduction. Libraries that interface with this hardware (like `libisp`) translate between raw sensor output and usable image data.

### Layer 2: libcamera (The Core Library)

`libcamera` is a modern Linux camera stack that replaces older V4L2-based approaches. It provides:
- A unified API for camera access
- Pipeline handling for the ISP
- Support for complex camera configurations

Raspberry Pi maintains their own fork of libcamera with Pi-specific optimizations and tuning files for their camera sensors.

### Layer 3: libcamera-apps / rpicam-apps

These are command-line applications built on libcamera:
- `libcamera-hello` / `rpicam-hello` — camera preview
- `libcamera-still` / `rpicam-still` — capture stills
- `libcamera-vid` / `rpicam-vid` — capture video

**Why two names?** Raspberry Pi renamed these tools from `libcamera-*` to `rpicam-*` (late 2023) to distinguish their Pi-specific implementations from generic libcamera tools. On many systems, the old names exist as symlinks to the new ones.

### Layer 4: ROS2 Camera Nodes

Packages like `camera-ros` sit on top of libcamera and publish camera data to ROS2 topics. They link against the libcamera libraries at compile time.

---

## Linux Utilities for Library Management

### ldd — List Dynamic Dependencies

Shows which shared libraries (`.so` files) an executable needs at runtime:
```bash
ldd /usr/bin/rpicam-hello
```
Useful for debugging "library not found" errors or version mismatches.

### ldconfig — Update Library Cache

Linux caches the locations of shared libraries in `/etc/ld.so.cache`. After installing libraries from source, run:
```bash
sudo ldconfig
```
This refreshes the cache so programs can find the new libraries.

---

## The Binary Compatibility Problem

When you install ROS2 packages via apt (`apt install ros-jazzy-camera-ros`), you get pre-compiled binaries. These binaries were compiled against whatever version of libcamera existed when the package was built.

**The problem:** If you later compile and install a newer libcamera from source, the apt-installed ROS packages may be incompatible—they expect the old library's symbols and behavior.

**The solution:** Clone the ROS package source into your workspace (`ros2_ws/src/`) and rebuild with `colcon build`. This compiles the package against your current libcamera installation, ensuring compatibility.

---

## Version Differences: Ubuntu 22 vs 24

(To be documented: specific differences in camera support between Ubuntu versions)

[Marco Walther]

On 1/28/26 08:45, Pito Salas wrote:
> I wrote down everything I thought I understood about packages and dependencies after troubleshooting my simple Raspi 5 + Pi Camera 2.1 + Ubuntu 24.04 setup. I asked claude to organize what I had figured out, which is I am sure incomplete and incorrect in places.
>
> Maybe this is useful and maybe you will send me some corrections.

Some truth, some half-truth and some questionable conclusions;-)

>
> - Pito
>
>
> # Claude's contribution
>
> ## The Camera Stack on Raspberry Pi + Ubuntu
>
> ### Layer 1: Hardware and Firmware
>
> At the lowest level, the Raspberry Pi has a dedicated **Image Signal Processor (ISP)** built into its GPU. The ISP handles raw sensor data processing—demosaicing, white balance, auto-exposure, and noise reduction. Libraries that interface with this hardware (like `libisp`) translate between raw sensor output and usable image data.
>
> ### Layer 2: libcamera (The Core Library)
>
> `libcamera` is a modern Linux camera stack that replaces older V4L2-based approaches. It provides:
> - A unified API for camera access
> - Pipeline handling for the ISP
> - Support for complex camera configurations
>
> Raspberry Pi maintains their own fork of libcamera with Pi-specific optimizations and tuning files for their camera sensors.
>
> ### Layer 3: libcamera-apps / rpicam-apps
>
> These are command-line applications built on libcamera:
> - `libcamera-hello` / `rpicam-hello` — camera preview
> - `libcamera-still` / `rpicam-still` — capture stills
> - `libcamera-vid` / `rpicam-vid` — capture video
>
> **Why two names?** Raspberry Pi renamed these tools from `libcamera-*` to `rpicam-*` (late 2023) to distinguish their Pi-specific implementations from generic libcamera tools. On many systems, the old names exist as symlinks to the new ones.

I don't think, those tools exist outside of the RasPi environment. They
are more a re-implementation of the original RasPi camera apps on-top of
libcamera.

>
> ### Layer 4: ROS2 Camera Nodes
>
> Packages like `camera-ros` sit on top of libcamera and publish camera data to ROS2 topics. They link against the libcamera libraries at compile time.

Kind of;-) At compile time, the camera-ros is built against an API
version and that and the name of the shared library is recorded in the
binary. The actual/fitting library is located at run-time.

>
> ---
>
> ## Linux Utilities for Library Management
>
> ### ldd — List Dynamic Dependencies
>
> Shows which shared libraries (`.so` files) an executable needs at runtime:
> ```bash
> ldd /usr/bin/rpicam-hello
> ```
> Useful for debugging "library not found" errors or version mismatches.

`ldd` shows the dependency tree based on what is recorded in the ELF
file header[s]. But some programs do their own dynamic loading as well
and ldd would not be able to help with that:-( I believe around
libcamera, the libpisp might be loading specific parts for different
cameras.

>
> ### ldconfig — Update Library Cache
>
> Linux caches the locations of shared libraries in `/etc/ld.so.cache`. After installing libraries from source, run:
> ```bash
> sudo ldconfig
> ```
> This refreshes the cache so programs can find the new libraries.

Normally, you should not have to do that. Well built packages or
'make-files' should re-run that command on 'install' when needed.

>
> ---
>
> ## The Binary Compatibility Problem
>
> When you install ROS2 packages via apt (`apt install ros-jazzy-camera-ros`), you get pre-compiled binaries. These binaries were compiled against whatever version of libcamera existed when the package was built.

Against an API version and library name.

>
> **The problem:** If you later compile and install a newer libcamera from source, the apt-installed ROS packages may be incompatible—they expect the old library's symbols and behavior.

That's true in the most general case. But the shared library system with
it's naming conventions and recording of required API versions should
prevent most surprises.

What you more frequently run into are search/path issues between the
system libraries and your custom built ones.

The 'ros-jazzy-camera-ros' package depends on the 'ros-jazzy-libcamera'
package, so all those dependencies will be installed (and potentially)
updated. On the RasPi, there is also the system 'libcamera' package,
which adds more version confusion:-(

When you build libcamera from source, by default, it would install into
/usr/local/..., adding potentially a third version to the mix. You could
bend the environment to find your versions, but that's usually error
prone in the long run.

You could force the build to override one of the packaged versions, but
that's not a good idea. You should never override 'package-managed'
locations with 'manual-build' things. The next update of that package
will break things!

>
> **The solution:** Clone the ROS package source into your workspace (`ros2_ws/src/`) and rebuild with `colcon build`. This compiles the package against your current libcamera installation, ensuring compatibility.

That will most likely work but it gets cumbersome when you have to do
this for a couple of packages. And now you have 'one more' package to
keep up-to-date.

That's why I started building those libcamera and related packages and
to publish them. They follow all the normal packaging standards and just
replace the 'Ubuntu-company-supplied' packages with newer versions. The
versioning makes sure, only packages with support for the latest PiCam's
get updated;-)

My packages also satisfy the 'ros-jazzy-libcamera' dependency with a
newer version. So, the normal 'ros-jazzy-camera-ros' package should be
happy.

In the end, with my packages, you should have one libcamera.so and
everything should use it;-)

> ---
>
> ## Version Differences: Ubuntu 22 vs 24
>
> (To be documented: specific differences in camera support between Ubuntu versions)
>

Not sure, if Ubuntu 22 was completely on libcamera yet. I know, I built
versions for it at some point.

Have fun;-)
-- Marco

[Sergei Grichine]

I couldn't resist feeding Claude's wisdom to ChatGPT.com - let the robots do criticism in the XXI century.

Here is the result:

• https://chatgpt.com/s/t_697a4c323560819187e5220b9870b14c  - criticism
• https://chatgpt.com/s/t_697a4d585e788191ab4052aff273c2f4 - a cleaned-up final version suitable for docs
• https://chatgpt.com/s/t_697a4e7dcfb881918a3758fc7c945524 - a diagram-style explanation (pipeline view)
• https://chatgpt.com/s/t_697a530ef6488191a25bf54696619309 - a ROS-focused version explaining where image_transport, compressed, etc. fit in.

I hope this helps.

Best Regards,

-- Sergei

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/94DA30C4-8AFA-4C2B-9FF4-AC9BDF35AFA9%40gmail.com.

[Pito Salas]

THanks! To clarify, Claude didn’t write it in a vacuum. I gave it a list of what I thought I had learned and understood and asked it to organize and correct it as needed.

With all due respect to ChatGPT, I thought the human critique from Marco was more, shall we say, “human”.

My goal was to assist my future self when next tackling this kind of problem. The trick in combining these three versions together is to not loose the specific “facts” in the florid Claude/Chat language. I started with text I wrote, I gave it to claude to organize, Sergei gave it to Chat to correct and improve. If I give all of this back to claude I fear that all the good stuff will be watered down and my original goal will be subverted. SO I will do the coating by hand, the old fashioned way!

Thanks all!

Best,

Pito

Boston Robot Hackers &&
Comp. Sci Faculty, Brandeis University (Emeritus)

> To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPCvLb4eb2suzq1hHh8S78Bhvp-2-R1dDE3YvAN2LEYNQ%40mail.gmail.com.