v4l2-ctl -d /dev/video0 -p 5

[camp .]

    Shout-out to Michael Wimble, Scott Horton, and Alan Federman of the ROS Discussion Group. I'm trying to run a ROS2 setup with two Raspberry Pi's and a workstation. One Pi is driving the robot (a Neato Botvac; mapping and navigating) the other Pi runs a PiCam, all visualized on the workstation. The problem was every time I'd try and view the camera, in addition to mapping and navigating, rviz would crash. Using some neat network monitoring tools (System Monitor and BpyTOP), we confirmed the problem was wifi network throughput. That is, with v4l2 (Video for Linux2) running at 30fps, the network traffic averaged around 2.2MiB/s. Changing the frame rate to 5fps* (acceptable for remote viewing) reduced wifi network traffic to 500KiB/s, and rviz no longer crashes.

 

Thanks,

Camp

 

* v4l2-ctl -d /dev/video0 -p 5

[Sergei Grichine]

Camp,

I am a bit confused here. The 2.2 mebibyte per second (roughly 20 megabits per second) should be easy for WiFi nowadays, especially in the vicinity of the router. Is there anything in your WiFi setup that impacts throughput? Did you have a chance to measure the actual raw performance of your WiFi? What is the best tool to use for that, in your experience?

Here are some numbers for comparison: https://www.duckware.com/tech/wifi-in-the-us.html

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/452122370.2480727.1630635910622%40mail.yahoo.com.

--

Best Regards,

-- Sergei

[camp .]

Sergei, first off, the webpage you linked is awesome! https://www.duckware.com/tech/wifi-in-the-us.html My network connection is 2.4GHz. And the robot and workstation are in the next room; approximately 50' away. As mentioned we used "System Monitor" and "BpyTOP" to look at the wifi traffic. I'll put your question to the list: What is the best tool to measure raw wifi performance? Is there a way to configure the above tools to see maximum throughput and while I'm at it tips and suggestions to enhance wifi performance? Our experience was with the PiCam at 30fps the wifi traffic was up to 2.2MiB/s and rviz would crash. When we lowered the frame rate to 5fps wifi traffic dropped to 500KiB/s and rviz no longer crashed.

 

Thanks,

Camp

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPfG7f_P84jEe%2B_SefRe824A4UbJHPNdDZpzuihkk%3DSYA%40mail.gmail.com.

[Chris Albertson]

It seems that maybe the bottle neck might not be WiFi bandwidth but rather maybe rviz or the networking stack.    Did you look at CPU utilization?   I can also imagine some kind of buffer overflow caused by a software bug,  The to-be-processed data overflows a buffer.   

In any case, 5 FPS might be all you need.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/727488630.2938098.1630780408166%40mail.yahoo.com.

--

Chris Albertson
Redondo Beach, California

[Scott Horton]

Hi Camp,

As I recall, we found that when monitoring the network usage on the PI side (publisher) it was using around 90Mbits/s when video capture was enabled on the PI.  As a test, the video capture frame rate was reduced as you mentioned.  With that change the laptop side was able to subscribe and see video displayed by rqt while not disrupting rviz2.  (I don't recall if we measured the PI side network usage after that, which we should have...These on-call debugging sessions seem to be a bit organic/chaotic at times. <g>  )

Further, it was noticed that the non-compressed image topic was being selected for rqt instead of compressed.  Selecting the compressed topic dropped the network usage on the subscriber side even more (as expected).

I would say the moral of the story is:  always used a compressed image transport and configure the frame rate to a rate that is sufficient for the application (which also lowers the video processing overhead on the publisher side which is especially important when compression is enabled).

Scott 

[camp .]

I've mounted the camera 📷  back on the robot 🤖. Will get back with more measurements. 

- Camp

Sent from Yahoo Mail on Android

Scott 

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/hbrobotics/1ad6f728-d767-4c82-8d47-a78a07989f0cn%40googlegroups.com

.

[Steve " 'dillo" Okay]

You can use ntop and iperf to monitor traffic flows and bandwidth on your Pi(s).

You might also want to install usbtop if you've got the Pis connected via Ethernet(yes, I did just say that). On the Pi3, the Ethernet port does not have its own controller but lives on the USB2 Bus.

With enough traffic going over the wired connection, it's possible to stomp your entire USB bus along w/ saturating the network.

This has WAY more effects on overall system performance on a practical level than one might actually think(Go ahead, ask me how I know this).

'dillo

[Wayne C. Gramlich]

'dillo:

I'll take the bait. How do you know this?

Regards,

-Wayne

On 9/7/21 6:28 AM, Steve " 'dillo" Okay wrote:
> You can use ntop and iperf to monitor traffic flows and bandwidth on your Pi(s).
> You might also want to install usbtop if you've got the Pis connected via Ethernet(yes, I did just say that). On the Pi3, the Ethernet port does not have its own controller but lives on the USB2 Bus.
> With enough traffic going over the wired connection, it's possible to stomp your entire USB bus along w/ saturating the network.
> This has WAY more effects on overall system performance on a practical level than one might actually think(Go ahead, ask me how I know this).
>
> 'dillo
> On Saturday, September 4, 2021 at 11:34:21 AM UTC-7 ca...@camppeavy.com wrote:
>

> Sergei, first off, the webpage you linked is awesome! https://www.duckware.com/tech/wifi-in-the-us.html <https://www.duckware.com/tech/wifi-in-the-us.html> My network connection is 2.4GHz. And the robot and workstation are in the next room; approximately 50' away. As mentioned we used "System Monitor" and "BpyTOP" to look at the wifi traffic. I'll put your question to the list: What is the best tool to measure raw wifi performance? Is there a way to configure the above tools to see maximum throughput and while I'm at it tips and suggestions to enhance wifi performance? Our experience was with the PiCam at 30fps the wifi traffic was up to 2.2MiB/s and rviz would crash. When we lowered the frame rate to 5fps wifi traffic dropped to 500KiB/s and rviz no longer crashed.

> Thanks,
> Camp
>
> On Saturday, September 4, 2021, 08:56:33 AM EDT, Sergei Grichine <vital...@gmail.com> wrote:
>
>
> Camp,
>
> I am a bit confused here. The 2.2 mebibyte per second (roughly 20 megabits per second) should be easy for WiFi nowadays, especially in the vicinity of the router. Is there anything in your WiFi setup that impacts throughput? Did you have a chance to measure the actual raw performance of your WiFi? What is the best tool to use for that, in your experience?
>

> Here are some numbers for comparison: https://www.duckware.com/tech/wifi-in-the-us.html <https://www.duckware.com/tech/wifi-in-the-us.html>

>
> On Thu, Sep 2, 2021 at 9:25 PM camp . <ca...@camppeavy.com> wrote:
>
>     Shout-out to Michael Wimble, Scott Horton, and Alan Federman of the ROS Discussion Group. I'm trying to run a ROS2 setup with two Raspberry Pi's and a workstation. One Pi is driving the robot (a Neato Botvac; mapping and navigating) the other Pi runs a PiCam, all visualized on the workstation. The problem was every time I'd try and view the camera, in addition to mapping and navigating, rviz would crash. Using some neat network monitoring tools (System Monitor and BpyTOP), we confirmed the problem was wifi network throughput. That is, with v4l2 (Video for Linux2) running at 30fps, the network traffic averaged around 2.2MiB/s. Changing the frame rate to 5fps* (acceptable for remote viewing) reduced wifi network traffic to 500KiB/s, and rviz no longer crashes.
>
> Thanks,
> Camp
> * v4l2-ctl -d /dev/video0 -p 5
>
> --
> You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/452122370.2480727.1630635910622%40mail.yahoo.com <https://groups.google.com/d/msgid/hbrobotics/452122370.2480727.1630635910622%40mail.yahoo.com?utm_medium=email&utm_source=footer>.

>
>
>
> --
> Best Regards,
> -- Sergei
>
> --
> You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPfG7f_P84jEe%2B_SefRe824A4UbJHPNdDZpzuihkk%3DSYA%40mail.gmail.com <https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVPfG7f_P84jEe%2B_SefRe824A4UbJHPNdDZpzuihkk%3DSYA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com <mailto:hbrobotics+...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/71b91d32-4ac0-4e6d-8536-fcb28ef116a3n%40googlegroups.com <https://groups.google.com/d/msgid/hbrobotics/71b91d32-4ac0-4e6d-8536-fcb28ef116a3n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Alan Federman]

just a note, on the raspberry pi, mmal Raspicam support has been dropped going forward. This means mmal in Raspberry Pi releases for ARM64 may not work in in ROS. I recently was able to make the ubiquity Robotics raspicam_node in ROS Noetic, and it appears to work for rqt image view, but I haven't fully tested if mmal is working. I am not sure if that could be an issue in camera driver dependant AI code like Tensor Flow OpenCV or Yolo.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/71b91d32-4ac0-4e6d-8536-fcb28ef116a3n%40googlegroups.com.

[camp .]

    I was going to start a new thread as I've found another problem, but I'll jump in since you're hinting at it here. I'm getting a very satisfactory viewing experience at 5 frames-per-second, which solved the wifi network issue (camera crashing rviz). Ideally, I'd be viewing the images through rviz, but it's on a different Global Fixed Frame than the map ("camera" versus "odom" or "map"), which requires some URDF work (I think). In the meantime, I'm happily viewing rqt_image on the workstation with my rviz map in another window. Regardless, when I change the Global Fixed Frame on rviz to "camera," I can see the video, but it is extremely slow and not useful. I thought /image_raw (I've tried compressed too) was the same message as rqt_image. In other words, rqt_image is fine, but when I view the /image_raw message through rviz, it sucks. What would be different about rviz's subscription to the /image_raw topic than rqt_image? Below, find a screenshot of the rviz map, rqt_image screen, and teleop keyboard. Again, I'm okay with this but I should be able to view the /image_raw through rviz. Reliability Policy on rviz for "Image" is set for "Best Effort."

 
Botvac with Camera
New photo by Camp Peavy
 
Thanks,
Camp

New photo by Camp Peavy

https://groups.google.com/d/msgid/hbrobotics/267829756.289493.1631031617357%40connect.xfinity.com

.

[Steve " 'dillo" Okay]

On Tuesday, September 7, 2021 at 7:03:29 AM UTC-7 Wayne Gramlich wrote:

'dillo:

I'll take the bait. How do you know this?

I had my OAK-D depth camera connected to the Jetson Nano, which itself was plugged into the same switch as the Pi3, all on the rover.

My plan was to use the Nano to host the cameras and run the vision stack and the Pi3 to run the ROS master, talk to the various Arduinos, GPS, etc.

Which seems sensible until you actually fire everything up and watch the USB-based NIC on the Pi eat all the bandwidth on the Pi's USB2 bus from talking to the Nano.

To the point where even the joystick has trouble getting messages through to the teleop_twist_joy node.

The little wireless joystick dongle kept blinking like it was resetting or trying to establish a connection and the rover would just stop dead every few seconds.

Then the image/depth data has to get onto the Pi's Wifi and stream to my laptop.  Looking at usbtop vs. iperf it's clear that the network AND USB are being saturated by this traffic.

One other interesting point I noticed is that the /camera topics are showing up fine in Rviz or Foxglove on my laptop, but I'm getting "unable to subscribe" errors for the other topics from the GPS, sonars, battery monitors,etc.

and these are all plugged into USB on the Pi3.

I could go in and look at the OAK-D parameters on the launch file or not run YOLO or adjust frame rates & resolution, etc.  or I could move on.

Since I've already got the Jetson on the 'bot and its doing much of the heavy lifting anyway, I'm just going to move on.

 This particular Pi3 is still running Kinetic(I'm pretty sure it's one of the ones from Service Robot Labs) so I'm seeing this as the final kick in the pants I need to migrate to a supported version of ROS on

more capable hardware.

'dillo

 

[Wayne C. Gramlich]

'dillo:

Thanks for the explanation. I figured it would be something like you described.

Robots are a complex trade-off space -- wireless bandwidth, computational power,
real power consumption, battery weight to power everything, robot endurance before
needing to feed itself etc.

We come a long way since Basic Stamps, PIC's, AVR's, etc. Heck, we even have SBC's
these days, rather than a repurposed Linksys Router. I'm looking forward to
the new toys coming on the horizon.

Regards,

-Wayne

> --
> You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/328500e9-44b1-47d4-b78b-e0bc340c07cbn%40googlegroups.com <https://groups.google.com/d/msgid/hbrobotics/328500e9-44b1-47d4-b78b-e0bc340c07cbn%40googlegroups.com?utm_medium=email&utm_source=footer>.