Face tracking bot

[Austin Keller]

Hi all! Thanks for the warm welcome today! I'm excited to find a community nearby working on these sorts of things.

This is the build I'm following as my starting point:

• https://towardsdatascience.com/real-time-object-tracking-with-tensorflow-raspberry-pi-and-pan-tilt-hat-2aeaef47e134

That work builds off of this, swapping out the object detection algorithm with Tensorflow models instead of OpenCV:

• https://www.pyimagesearch.com/2019/04/01/pan-tilt-face-tracking-with-a-raspberry-pi-and-opencv/

The only deviation I had to make to get things working was rotating the camera by 180 degrees. I also added some standoffs and an enclosure (modified Cana Kit box for Raspberry Pi) for more stability.

Things I've found

• The PID can often get into an erratic mode where it's oscillating back and forth across your face. It might be improved with some tuning.
• If no object is detected, the camera will drift until it hits the end range of the servos, at which point it gives some errors and won't be able to recover without restarting the process. This is the most annoying issue and prevents it from being a totally standalone interactive thing. I haven't dug into it yet since I'll be replacing that driver with my own. Maybe I'll come back to it at some point.
• Object detection works surprisingly well in low light.
• Face detection is ~1.3 FPS on CPU vs 8FPS on TPU. The CPU model is slow enough that it would require tuning the PID parameters if someone wanted to get that working.

Next Steps

The PanTilt shield I'm using provides a python library for controlling the servos. I'll be replacing that whole stack of motors and controls with two Nema 17s and a CNC shield on an Arduino. I've worked with Arduino a bit before but GRBL is new, so I'll be reaching out for help on that I'm sure!