PiCam Video on ROS2

[camp .]

    We talked about this at the ROS Discussion Group. I'm running a PiCam on RaspiPi4 using the v412 camera driver*. We can get video through rqt_image_view on the Pi but not the remote workstation (we can see the nodes, topics, etc., on the workstation). The goal is to get video through rviz2 on the workstation. We also see the "hz" rate of /image_raw at 32.354, and the video** runs on the Pi. On the workstation, the average rate is 0.018 (earlier, it was ~2.84), and no video. So there is some bottleneck between the Pi and PC. Multicast has been enabled. I have no problem mapping and navigating with this workstation and other ROS2 Raspberry Pi 4's.
 

    Interesting sidenote: I can run rviz2 on the Pi with Ubuntu 20.04 and ROS Foxy. I say that because I don't believe the desktop showed up until I installed ROS2, and with ROS1, you were never able to run rviz on the Pi. Something to do with the Pi's graphics capabilities (someone, please correct me if I'm wrong). Ubuntu 20.04 is server only (if I'm not mistaken). Regardless, rviz2 runs, but it crashes... and I couldn't get video through rviz2 locally on the Pi while running. So there's that.

 

Is there another driver I should try to run a PiCam on Ubuntu 20.04 and view through rviz2?

 
Thanks,
Camp
 
* https://chuckmails.medium.com/enable-pi-camera-with-raspberry-pi4-ubuntu-20-10-327208312f6e
 
** ros2 run rqt_image_view rqt_image_view

[Steve " 'dillo" Okay]

You can run rviz under ROS 1 on a Pi3. I did this with somebody at Circuit Launch on their "Leo" Rover(leorover.tech) just last week.

The Pi3 and Pi4 allow you to configure how much RAM the OpenGL driver can use, so that might be one thing to look at.

The more likely issue is using "ssh -X"  vs. just plain ssh for getting into the Pi remotely and then a properly configured config under ~/.ssh/config or /etc/ssh/sshd_config that allows X11 forwarding to the remote client(i.e. the laptop/desktop you're logging in from).

Typically for a regular Ubuntu(or other Linux) install you would need to do this yourself, on both the robot and laptop/desktop.

Since we were able to run rviz out-of-the-box on the LeoTech rover just by running "ssh -X", this makes me think that either recent versions of raspi-config make the necessary changes as part of enabling ssh OR the LeoTech folks enable X-forwarding as part of their own factory install process.

'dillo

[camp .]

> The more likely issue is using "ssh -X" vs. just plain ssh...
 
Alright, "ssh -X" made a difference. Thanks! I launch the v412 camera driver on the Pi and ssh -X to the Pi from the workstation and run "rqt_image_view" from the ssh -X session. I am now getting video on the workstation. Yay!
 
I try and launch "rqt_image_view" with plain "ssh" it's a non-starter (could not connect to display).
 
I am still working on getting the video on rviz2. I can see the /image_raw being published. No errors or anything being reported in rviz2. Still, a big difference in the rate at which /image_raw is published "ros2 topic hz /image_raw." The average frame rate is 9.950 from the Pi and 0.015 on the workstation. This is while video images are running on the workstation.
 
Another funny experience that could be a clue: the video slowed down and eventually stopped when I tried to monitor the messages on the workstation (ros2 topic hz /image_raw). When I killed that process, the video came back.
 
- Camp

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/c5ef6059-0421-41c8-82fd-a544b14806dbn%40googlegroups.com.

[Kevin Chow]

"and with ROS1, you were never able to run rviz on the Pi"

Nope, I've been able to run rviz on a Raspberry Pi (Raspberry Pi 4) just fine.

"Ubuntu 20.04 is server only"

Well... there's Ubuntu Mate, which I have used: https://ubuntu-mate.org/download/

[camp .]

> Nope, I've been able to run rviz on a Raspberry Pi (Raspberry Pi 4) just fine.

 

Okay, I tried it with Kinetic. Perhaps the problem was I didn't build rviz from source. Also, rviz2 was still crashing badly when I ran it on the Pi. Works fine on the workstation. Perhaps the same thing... perhaps I need to build from source.

 

> Well... there's Ubuntu Mate, which I have used: https://ubuntu-mate.org/download/

 

Have you used Ubuntu Mate 20.04 with ROS2? I believe it's "server" too (no graphical desktop).

 

Thanks,

Camp

https://groups.google.com/d/msgid/hbrobotics/c0c4eae5-a843-431f-b699-6768f88459afn%40googlegroups.com

.

[Sergei Grichine]

If I am not mistaken, installing a package with X and Desktop dependencies (rviz?) on a server Ubuntu would pull all the system GUI stuff and effectively turn the server into a desktop. True?

Best Regards, 
-- Sergei Grichine 

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/1764649155.2625637.1627966131044%40mail.yahoo.com.

[Alan Federman]

OK hmm  t hink........

 

1 working on robot

2 not working on workstation

 

3 best guess: "What we have here, is a failure to communicate."

 

I think this is a network issue. RC.local isn't working on your network for some reason.  To test don't use hostnames, use IP numbers. Make sure you can both list and echo topics on the remote workstation.

 

In ros1 on robot:

 

export ROS_IP=10.0.0.59

export ROS_MASTER_URI=http://10.0.0.59:11311

 

On workstation the master uri is the robot's, but

 

ROS_IP is the station's

 

IMHO this solves 90% of my ROS communication issues. I can tell people this until I'm blue in the face, and they just don't listen. It's easy enough to try it and see.   Maybe some networking guru can explain it, but I think it has to do with leases and DHCP. On my crappy Xfinity router sometimes IP numbers get swapped around, and on some Ubuntu distros hostnames don't resolve or there is some other funky environment problem.

--

[Marco Walther]

On 8/3/21 7:11 AM, Sergei Grichine wrote:
> If I am not mistaken, installing a package with X and Desktop
> dependencies (rviz?) on a server Ubuntu would pull all the system GUI
> stuff and effectively turn the server into a desktop. True?

Two things.
* rviz (and friends) would just pull in the libraries, not the 'desktop
environment itself.
* As long the packages are available, you can install a [graphical]
desktop into a server. I normally prefer that way because the 'desktop
distributions' pull in way more stuff, than I need or want.

Have fun;-)
-- Marco

>
> Best Regards,
> -- Sergei Grichine
>
> On Mon, Aug 2, 2021, 11:48 PM camp . <ca...@camppeavy.com
> <mailto:ca...@camppeavy.com>> wrote:
>
> > Nope, I've been able to run rviz on a Raspberry Pi (Raspberry Pi
> 4) just fine.
> Okay, I tried it with Kinetic. Perhaps the problem was I didn't
> build rviz from source. Also, rviz2 was still crashing badly when I
> ran it on the Pi. Works fine on the workstation. Perhaps the same
> thing... perhaps I need to build from source.
>

> > Well... there's Ubuntu Mate, which I have used: https://ubuntu-mate.org/download/ <https://ubuntu-mate.org/download/>

>
> Have you used Ubuntu Mate 20.04 with ROS2? I believe it's "server"
> too (no graphical desktop).
> Thanks,
> Camp
>
>
> On Monday, August 2, 2021, 9:49:08 PM EDT, Kevin Chow

> <kvnc...@gmail.com <mailto:kvnc...@gmail.com>> wrote:
>
>
> "and with ROS1, you were never able to run rviz on the Pi"
> Nope, I've been able to run rviz on a Raspberry Pi (Raspberry Pi 4)
> just fine.
>
> "Ubuntu 20.04 is server only"
> Well... there's Ubuntu Mate, which I have used:

> https://ubuntu-mate.org/download/ <https://ubuntu-mate.org/download/>

>
> On Monday, August 2, 2021 at 4:05:53 PM UTC-7 ca...@camppeavy.com

> <https://groups.google.com/d/msgid/hbrobotics/c5ef6059-0421-41c8-82fd-a544b14806dbn%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hbrobotics/c0c4eae5-a843-431f-b699-6768f88459afn%40googlegroups.com

> <https://groups.google.com/d/msgid/hbrobotics/c0c4eae5-a843-431f-b699-6768f88459afn%40googlegroups.com?utm_medium=email&utm_source=footer>

> .
>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hbrobotics/1764649155.2625637.1627966131044%40mail.yahoo.com
> <https://groups.google.com/d/msgid/hbrobotics/1764649155.2625637.1627966131044%40mail.yahoo.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPUMQqbpE7F0u0C_4DJzxWh%2BScc_kYZHfcMjWuohpS_pg%40mail.gmail.com
> <https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPUMQqbpE7F0u0C_4DJzxWh%2BScc_kYZHfcMjWuohpS_pg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Kevin Chow]

I used Noetic on mine (because I was using Ubuntu 20.04).  And I installed it directly on my Pi (ie. I didn't use a Docker container).  And I also didn't build rviz from source... in fact, a lot of the stuff you shouldn't need to build from source to be able to run it.  You can literally just apt-get most of the basic ROS stuff.  And I never actually used ROS 2 before... I'm still doing all my stuff in ROS 1... I guess I'm just a bit too used to using ROS 1.

Also... are you trying to VNC into your Pi using a virtual display in RealVNC to view rviz while the Pi is headless? If so... that may be the cause of you not being able to view rviz.  How do I know this? Well... it's because I tried doing that before.  When I first tried out running rviz on my Raspberry Pi while I had a monitor plugged in, it was fine.  However, if I booted the Pi without connecting it to a monitor and tried to VNC into it with a virtual display setup in RealVNC and run rviz, it couldn't open it because it didn't like the virtual display.  To resolve this issue... I either had to have a monitor connected to my Raspberry Pi or get a dummy dongle (like this: https://www.amazon.com/gp/product/B06XT1Z9TF) to plug into the HDMI port (using a HDMI to micro HDMI converter cable as well if you're using a Pi 4).  Or the third option is to use a virtual display that was done through another VNC thing that wasn't RealVNC (which I didn't do because I was too lazy).

[camp .]

> If I am not mistaken, installing a package with X and Desktop dependencies (rviz?) on a server Ubuntu would pull all the system GUI stuff and effectively turn the server into a desktop. True?

 

    I installed Ubuntu 20.04 and of course, it had no desktop as it is  a "server." Then I installed ROS2 and a minimum blue desktop appeared with an "Activities" drop-down list with "Help" and "Show Applications" as the options. Perhaps as Marco suggests... it's just the libraries and not the desktop environment itself.

 

- Camp

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPUMQqbpE7F0u0C_4DJzxWh%2BScc_kYZHfcMjWuohpS_pg%40mail.gmail.com.

[Kevin Chow]

Ubuntu Mate includes a GUI desktop.  In fact, Mate is the name of the GUI desktop.

[camp .]

> In fact, Mate is the name of the GUI desktop.

 

Got it! - cp

https://groups.google.com/d/msgid/hbrobotics/7cc7b984-c143-4d6b-9cab-a7e807d1d203n%40googlegroups.com

.

[Steve " 'dillo" Okay]

If you can see the topics on a remote system and can subscribe but not get any data, (like /camera/images/raw or /camera/rgb_images, etc), you might not have all the necessary message types or translator packages installed on both systems. Check to see, for example, if you have ros-$DISTRO-image-transport-plugins installed on all systems, you're running on if you're having trouble seeing images on a remote system.

You don't have to have a desktop GUI installed on your robot, but you do have to have the whole message and data transport chain set up on both sides.

This seems kind of obvious, but I'm saying it anyway because it's bit me in the butt more than once.

HTH,

'dillo

On Monday, August 2, 2021 at 9:48:55 PM UTC-7 ca...@camppeavy.com wrote:

[Steve " 'dillo" Okay]

You can install x11-common and OpenGL w/ out bringing in the whole of the Ubuntu desktop.

You have to pick your way around it, but w/ a little rocket surgery, it's possible.

The reason ORF and other ROS install HOWTOs tell you to just install ros-<distro>-desktop-full as a default, even on your 'bot, is so you don't HAVE to be that kind of an expert.

'dillo

[camp .]

    We did achieve another breakthrough at the ROS discussion group with the help of Michael Wimble. '"ssh -YC" to the Pi from the workstation we were able to get /image_raw video through rviz2 on the workstation by changing the global frame from "odom" to "camera". Changing global frame from "odom" to "camera" when launching rviz2 from the workstation, however, still could not get the video. Using "ros2 topic hz /image_raw" the average frame rate on the Pi was ~30.00 whereas the workstation showed a fraction of a frame per second ~0.015.

 

Thanks,

Camp

https://groups.google.com/d/msgid/hbrobotics/04f07010-3089-4b5e-bd64-693ad3cc554bn%40googlegroups.com

.