yolo

[A J]

Hey is anybody using the newer Yolo on Rasp 5 ?

Have seen one blog where they use RPi5 and get

less than 1s classification.

https://docs.ultralytics.com/models/yolov10/

[Nathaniel Lewis]

I would highly recommend checking out the NCNN framework if you want to run a model directly on the Pi.

https://docs.ultralytics.com/integrations/ncnn/#why-should-i-use-ncnn-for-my-mobile-ai-applications

https://www.seeedstudio.com/blog/2023/09/28/raspberry-pi-5-vs-pi-4-ai-performance-cpu-benchmark-how-much-leap-forward/

- Nathan

--

You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/22c8017a-be69-4f58-aa35-2fdda0467bb9n%40googlegroups.com.

[Chris Albertson]

I was able to process realtime video on a Pi4 and the Corel TPU with tensorflow lite with weights reduced to 8-bits.  So that would be 1/30th of a second.   You will get dramatically better results from Yolo if you apply some transfer learning.   The stock yolo is trained on objects your robot will never see, like starfish, whales and guns.   This setup gave the same perfromance is my Xeon-based PC with a low-end Nvidia GPU but in a much smaller and lower cost package.

The other problem with Yolo is poor accuracy.  90% or whatever sounds good untill you have to deal with hundreds of wrong answers.   So what I implemented is a filter, I said the object must be seem in X times in Y video frames.   Basically the detected object are placed in a database and if they we don’t accept them is “real” untill they seem to exist for some short period of time.   This improves the results and only adds a lag of a 1/3rd or half second.

[Chris Albertson]

This might be a beeter verion of the TPU if you have a Pi5 with M.2 and “m” key.  It is half the price but it does use the only m.2 which might be better used by an SSD.

On Jul 15, 2024, at 11:32 PM, Chris Albertson <alberts...@gmail.com> wrote:

I was able to process realtime video on a Pi4 and the Corel TPU with tensorflow lite with weights reduced to 8-bits.  So that would be 1/30th of a second.   You will get dramatically better results from Yolo if you apply some transfer learning.   The stock yolo is trained on objects your robot will never see, like starfish, whales and guns.   This setup gave the same perfromance is my Xeon-based PC with a low-end Nvidia GPU but in a much smaller and lower cost package.

The other problem with Yolo is poor accuracy.  90% or whatever sounds good untill you have to deal with hundreds of wrong answers.   So what I implemented is a filter, I said the object must be seem in X times in Y video frames.   Basically the detected object are placed in a database and if they we don’t accept them is “real” untill they seem to exist for some short period of time.   This improves the results and only adds a lag of a 1/3rd or half second.

[Nathaniel Lewis]

If you want to add a hardware TPU/NPU to the Pi 5, there's the coral as Chris Albertson linked, but there is now also an official Pi 5 AI kit that uses a Hailo-8L, which is some 3x faster than the coral based products (and far newer).

https://www.raspberrypi.com/products/ai-kit/

https://www.jeffgeerling.com/blog/2024/testing-raspberry-pis-ai-kit-13-tops-70

- Nathan

--

You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hbrobotics/ADE14FC0-855D-43EA-826D-C1A11016B63B%40gmail.com.

[Chris Albertson]

On Jul 16, 2024, at 12:56 AM, Nathaniel Lewis <rob...@nrlewis.dev> wrote:

If you want to add a hardware TPU/NPU to the Pi 5, there's the coral as Chris Albertson linked, but there is now also an official Pi 5 AI kit that uses a Hailo-8L, which is some 3x faster than the coral based products (and far newer).

Thanks for poiinting this out.

This device looks good, except for one thing, the reviewer was only able to run the included demo software on it.  He did not talk about running Pytorch or TensorFlow.  Not saying it can’t be done just that I did not see any mention of software other then the included demos.    I tried Google and pretty much the talk on the forums is that running anything other then the included software is not easy.      That said, not many people want to run their own software and the demo stuff does Computer Vision related tasks which might be all you need. 

Corel is actualy Google and it is well suported by Google’s TensorFlow Lite.     

One of the things to look at in an of these is the amount of on-device RAM.  All of these are literally a few thousand times too small to run even a stripped-down LLM and are best used for computer vision.  So the demo software might be all you want.   

The USB verion of the Corel TPU did have one other nice feature, because it is USB I could used it on my larger Xeon-based Linux PC.  Doing development work on a Pi is just too slow,  Then after stuff works move it over to the Pi.    But maybe the Pi5 with 8GB RAM is faster.     Then again you can only have one M.2 device on the Pi5, either one of the AI devices or an SSD.

In any case for serious work on either of these devives you need a larger system with a larger GPU for development although ypu could always just use a cloud server.   

 

https://www.raspberrypi.com/products/ai-kit/

https://www.jeffgeerling.com/blog/2024/testing-raspberry-pis-ai-kit-13-tops-70

- Nathan

On Mon, Jul 15, 2024, at 11:43 PM, Chris Albertson wrote:

This might be a beeter verion of the TPU if you have a Pi5 with M.2 and “m” key.  It is half the price but it does use the only m.2 which might be better used by an SSD.

[Nathaniel Lewis]

Hi Chris,

I'm still waiting on mine to be delivered, so I can't vouch from personal experience yet :)

My understanding is that it uses a similar workflow as other TPU/NPU hardware uses - namely, you take whatever framework you wish to use (Tensorflow, pytorch, etc.), export as an ONNX model, and then use that hardware device's compiler to take the ONNX model to something device specific.

The hardware vendor's page specificly mentions support for Tensorflow, pytorch, and ONNX.

The major reason I wanted to point this out, is that now that this is an official Pi AI kit, any tutorials, guides, examples, etc. that the Raspberry Pi foundation creates moving forward will almost certainly support this device. The major selling point of the whole Pi ecosystem is that anything from Pi Foundation itself gets "first class" support.

- Nathan

[A J]

Thanks Chris,

I do have Corel USB but had only tried some using it on the PC. But I bet it would be faster than the

stock RPi.

[A J]

Thanks Nathaniel,

I will look at the Rpi Ai-Kit for the future. Are the tool chains well supported from the PC to RPi?

[Chris Albertson]

On Jul 16, 2024, at 12:52 PM, A J <aj48...@gmail.com> wrote:

Thanks Nathaniel,

I will look at the Rpi Ai-Kit for the future. Are the tool chains well supported from the PC to RPi?

It makes the RPi about 100 times faster with one very importent disclaimer:  The model MUST fit in the available 4MB of RAM.    The toolchain question is big.   It might be resolved by now.  I don’t know

But none of the integer-only TPUs are any good at training, including transfer learning.  They are only good for fielding a system AFTER you have it working as you like.