Mentor Proposal: Python deep learning inference on video

[shenk....@gmail.com]

I would like to propose to mentor following topic, which attracted much interest (46/300 proposals) last year, below.

I would also be keen to mentor a project on a high-level helper for one-shot learning training.

#### _IDEA:_ Python deep learning inference on video

- _**Description:**_ OpenCV's [DNN module](https://docs.opencv.org/trunk/d6/d0f/group__dnn.html) allows high-level inference on individual images. But, performing inference on video requires producing much boilerplate code and skills not directly relevant to computer vision. The goal of this project is to develop a high-level helper class in python to perform optimized inference on videos (eg, pose detection, emotion detection) with data storage (eg, output and bounding boxes) in dataframes for easy access.
- _**Expected outcomes:**_
- Review the papers on the topic in the resources below
- Define and implement an API for proposed methods
- Optimize batch processing of video input for neural networks
- Implement the [Model Zoo](https://github.com/opencv/opencv/tree/master/samples/dnn) for video use cases
- Handle optimal neural network inference output for further processing
- Write examples and tutorials
- Resources
- [DNN Tutorial](https://docs.opencv.org/master/d2/d58/tutorial_table_of_content_dnn.html)
- [List of DNNs](https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV)
- [List of major DNN models](https://github.com/kjw0612/awesome-deep-vision)
- _**Skills Required:**_
- Coding in Python. Experience with deep neural networks.
- _**Mentors:**_ Justin Shenk
- _**Difficulty:**_ Medium

---

[Vipul Bajaj]

Hi Justin

I'm interested in both the projects you offered and have the relevant experience and expertise for them. I've recently started working on one-shot learning and I've been working with audio-video stuff for around 3 years now.

I’m a senior undergraduate at the Indian Institute of Technology Kanpur(IIT Kanpur). I’ve experience working with deep neural networks through various projects, publications and internships. I've a couple of publications in topics related to the project you are mentoring - Python deep learning inference on video. My preferred frameworks are Pytorch and Tensorflow. I’ve had formal college coursework through a couple of courses in Machine Learning and Computer Vision. I've worked with Prof. Vinay P. Namboodiri ( IIT Kanpur ), Prof. Piyush Rai (IIT Kanpur), Prof. Preethi Jyothi ( IIT Bombay ), Prof. Bryan Hooi(National University of Singapore). I've made a rough proposal for the project which aims at defining a high-level API for video inferencing. Can you please guide me on preparing a final proposal for this project? My CV is available at this link. My homepage is available here. I eagerly await your response.

Thanks in anticipation.

Regards

Vipul Bajaj

B.Tech Electrical Engineering with Double Major in Computer Science and Engineering

IIT Kanpur

[Shadaab Ghani]

Hi Justin,

I am interested in Python Deep Learning Inference in Video.

I am currently pursuing masters in Chennai Mathematical Institute with a focus in machine learning. I have taken courses in machine learning and deep learning in CMI. I have made a rough proposal regarding this topic. Can you please guide me.

Thank You

Shadaab Ghani

Chennai Mathematical Institute.

On Saturday, February 29, 2020 at 10:46:01 PM UTC+5:30, shenk...@gmail.com wrote:

[saraansh tandon]

Hey!

I am very interested in the deep learning inference on video idea. I am a 3rd year undergraduate research student at IIIT Hyderabad. I am currently doing my research in the field of computer vision, specializing in biometrics under Prof. Anoop Namboodiri. I have been using opencv and deep learning in PyTorch since the last year and have also recently submitted a paper on analysing the relationship between fingerprint matching and spoof detection to IJCB 2020(under review), which uses a branched deep learning framework. Other than that I have implemented almost all the usual layers of a CNN from scratch as part of the course CS231n offered by stanford(completed informally). Formally I have taken courses on Digital Image Processing, Computer Vision and Deep Learning.

I have working knowledge of a lot of the algorithms and tasks listed in the resource shared by you. I believe that I can contribute a lot to the organization by being a part of this project. I would love to hear from you and start ASAP.


Thanks.
Saraansh Tandon

[Justin Shenk]

Hi Shadab,
Thanks for your message. Please feel free to share the proposal and I will try my best to give feedback.

Regards,
Justin

[Harshit]

Hello Justin,

I am interested in working on this project and have submitted my draft. Please review and provide valuable comments.

Thanks

Harshit