A write up of my Can Do Challenge behavior tree

[Michael Wimble]

For your amusement, I have a PDF document attached that attempts to describe how my first attempt at a fetch-me-a-bear behavior tree works on my robot. Remember that this was working in Gazebo simulation, which was discussed in an earlier e-mail.

I don't know if I'll have my robot actually fetching a beer in time for the challenge. Over the last few days, I've struggled with the end-to-end task of uploading images to RoboFlow, getting its help to tag the 20 or so images, getting the Rapid model to actually recognize the cans in the images and draw bounding boxes, then realizing that the Rapid approach didn't meet my needs as it requires an Internet connection to their servers.

Then I struggled to find out how to actually generate a YOLOv8 model with weights. Then it was a bitch to get that to work. With Claude Sonnet's help, I got it to eventually work with both a USB camera and a Pi camera, all on a Pi 5 with an AI hat. After multiple attempts, of course, to get the color to come out in RGB instead of BGR or something else.

I got that all to work, though the model was, understandably, not impressive as it trained on only 13 test images, had 3 valid images, and 2 test images. It was, after all, just a quick attempt to see if I could get the bear to dance at all--to get the end-to-end process of creating a YOLO model and running it on the Raspberry Pi 5. Still, performance was perfectly acceptable, and I think I know how to repeat the whole process again when I get more like 70 to 100 images to train with.

Also, for your amusement, the Pi began crashing. It turned out that using a USB-C cable from my 5-volt supply didn't supply enough current. I had to provide a parallel pair of power wires to the 40-pin connector to get everything to work.

There also was the problem that none of this worked unless I paid $100 to RoboFlow for a 30-day subscription. Without that, I could only run the model using their server. And the Python code to run YOLO and OpenCV wanted a different version of OpenCV that I used for other parts of my system. I'm still not sure if the other parts still run, but we will see soon.

If the object recognizer works, I'm near the point where I can run the behavior tree on the real robot, replacing my simulator object recognizer with a real one. Then there's the issue of replacing my "testicle twister" gripper with something capable of actually grabbing a can of Coke.