There has been a few mails on the YOLO project idea. Trying to address all of
them in one email. The assumption is the students have subscribed.
Please try catch mentors on the freenode IRC in the #beagle-gsoc channel. It
looks like you have seen the elinux page. Please note the timezones for the
potential mentors in the bottom. I am often on that channel on and off between
10:30-19:00 US/Pacific time. Do hang around the channel. Most of us will look
at what has been said on the channel and respond even if it is later so asking
and leaving immediately will not work well.
General comments - Ideally, we would like to see the YOLO model working on the
BBAI (or x15 or even a BBB!) at a full 30fps video frame rate. Having said
that, right now I am seeing around 10 seconds per frame. This is largely due
to YOLO ask implemented with the Darknet framework not taking advantage of the
hardware. There are many possible ways of working on this. Do keep in mind
GSoC is a relatively short period of time. As part of the application, there
should be a convincing explanation on why you think you can accomplish what
you propose in the time frame.
Just to throw out potential work in this general area:
- Attempt leverage the TIDL stuff to accelerate it. Right now, TIDL doesn't
support all the layers so there will have to be some of it done on the
accelerators and some of it done on the ARM.
- Attempt to use the (any day now) updated TIDL stuff with the Tensorflow lite
support to run the model. Jason might prefer this path.
- Attempt to use the model conversion tools in TIDL.
- Attempt to use OpenCL to accelerate things. Please note, a brute force
recompile of the OpenCL port of darknet does not work. Most likely, this is
due to the port focusing on OpenCL with a GPU instead of OpenCL with the DSP
as it is on the BBAI/x15. A preliminary debug suggests it is a memory problem
- Attempt to use the SGX GPU. Currently, only OpenGLES is supported on there.
This would basically be a port to use OpenGLES. Nice thing about this path is
it could be reused on the BBB too.
- It could be a combination of any of the above.
YOLOv3 may not fit on the device. YOLOv2 (or even YOLOv2-tiny) may be a more
Part of this is performance, it would be good to identify what size frames is
being targeted. I have found 320x240 to be convenient as that's common to many
On Saturday, February 29, 2020 11:13:35 pradan wrote:
> Hello everyone,
> I am Prashant Dandriyal, a final year undergraduate of Bachelor of
> Technology in Electronics and Communication Engineering (E.C.E). My
> experiences with embedded systems include simple electro-mechanical
> circuits, 8-bit microcontrollers and 32 bit ones like the EK-M4C123GXL TIVA
> Launchpad and the CC26X2R1 development board. The components by Texas
> Instruments were provided to me by Texas Instruments in part of the *India
> Innovation Challenge and Design Contest* (IICDC-2018).
> I have also been drawn towards the (previously niche) field of *Embedded
> Artificial Intelligence (Embedded AI)*, better knows as* Edge AI* or
> *TinyML* <https://tinyml.org/
>as one of its subsidaries. I have been
> following the TinyML community for some time now. In process of
> implementing Machine Learning at the edge, I have completed course work and
> worked on *TensorFlow*, *TensorFlow Lite *and *Intel OpenVINO toolkit*, all
> in process of shifting inference to the edge. For my final year project, I
> am in process of implementing *On-device Learning on Low Compute-capable
> For the GSoC 2020, I would like to contribute to the project *YOLO models
> on the X15/AI*. As one the mentors is *Mr Hunyue Yau*, I would request you
> all to redirect me to the related communities where I can discuss the idea
> with the mentors.
> I am also going to introduce myself in the *#beagle-gsoc
> channel at riot.im
and ask for help.
> I will be indebted for this help as it will enable me to finalize my
> project and begin with the preparations.
> Thanking You,
> Prashant Dandriyal