Weekly Progress Thread GSoC 2020: Media Ip Streaming

[nwan]

Hello,

this is my first weekly progress report for the Media Ip Streaming project. I will report everything I achieved throughout one week in this thread and update it weekly. My Mentors for the project are: rma31, henrix, pdp7 and indu. You can find a dedicated project page here: https://elinux.org/BeagleBoard/GSoC/2020_Projects/Media_IP_Streaming

Accomplishments:

• Complete warmup task
  
• Setup General Development Environment (Software and Hardware)
  
• Evaluate existing Drivers and identify challenges for porting
  
• Selection of appropriate Kernel: 4.19-rt
  
• Finish Introductory Video (https://www.youtube.com/watch?v=Pvb3sk3QNuI)

Resolutions to blockers: None

On-going blockers: None

Plans for the next week:

• Setup Beaglebone AI specific Toolchain
  
• Evaluate if existing kernel driver needs changes for AI
  
• Implement a base DTS for CTAG Face BBAI in arch/arm/boot/dts
  
• Change existing DTS Pin Configurations to match BBAI pin muxing
  

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 2 progress report. I was able to allready finish some milestones from this and next week so I came up with some extra tasks for this week.

Accomplishments:

• Setup Beaglebone AI specific Toolchain
• Evaluate if existing kernel driver needs changes for AI
• Implement a base DTS for CTAG Face BBAI in arch/arm/boot/dts
• Change existing DTS Pin Configurations to match BBAI pin muxing 

• Testing of all implemented functionality
• Bug fixing of implemented functionality
• Refactoring of device tree configuration, to just include base configuration in DTS
• Implement Face Specific configuration as overlay for BBAI

Resolutions to blockers:

• I had a blocker last week regarding the ctag face fitting on the Beaglebone AI. I was able to fix this with ordering stackable headers to allow for more room between the board and the cape

On-going blockers: None

Plans for the next week:

• First evaluation of sound card performance
• Further Bug fixing and refactoring
• Documentation of the Sound Card Driver
• Implement Device Tree Overlay for BBB

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 3 progress report. I was able to allready finish some milestones from this and next week so I came up with some extra tasks for this week.

Accomplishments:

Testing of all implemented functionality 
Bug fixing of implemented functionality 

First evaluation of sound card performance

Overlay for BBB
Documentation of Driver Installation

Resolutions to blockers:

• Fix Alsa Config

On-going blockers:

• Currently audio input is not working properly. henrix thinks that alsa config is wrong

Plans for the next week:

• Second evaluation of sound card performance
• Debugging
• Submit pull request to mainline kernel

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 4 progress report. 

Accomplishments:

Debugging and Fixing Input Bug for BBB and BBAI Driver for CTAG Face
Testing audio in- and output
Research and evaluation of existing AVB driver
-> AVB driver will be reimplemented based on existing implementation of Indu

Resolutions to blockers:

• N/A

On-going blockers:

• N/A

Plans for the next week:

• Port drivers to 5.4-rt on BBB and BBAI
• Test audio functionality on 5.4-rt on BBB and BBAI
• Submit Pull Request for 4.19-rt (BBB and BBAI)
• Submit Pull Request for 5.4-rt (BBB and BBAI)
• Submit Pull Request for overlays

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 5 progress report. 

Accomplishments:

• Port drivers to 5.4-rt on BBB and BBAI (Port is currently not working becuause spi bitbang drivers are not loaded)
  

• Test audio functionality on 5.4-rt on BBB and BBAI
• Submit Pull Request for 4.19-rt (BBB and BBAI)

• Submit Pull Request for overlays

Resolutions to blockers:

• N/A

On-going blockers:

• The spi bitbang drivers dont get load on 5.4-rt. TODO: Ask pdp7 whether spi bitbang not working is a known problem on BBAI on 5.4-rt, or if this is a problem with my implementation.
  

Plans for the next week:

• Review of existing AVB driver
• Review of gPTP daemon
• Review new version of gPTP from 2020
• Start porting gPTP to kernel 5.4-rt

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 6 progress report. 

Accomplishments:

• Review of existing AVB driver
  
• Review of gPTP daemon
  
• Review new version of gPTP from 2020
  
• Start porting gPTP to kernel 5.4-rt
  

Resolutions to blockers:

• N/A

On-going blockers:

• N/A
  

Plans for the next week:

• Finish implementation of gPTP in kernel space
• Review MSRP protocol
• Implement ALSA dummy driver

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 7 progress report. 

Accomplishments:

• Review MSRP protocol
• Implement ALSA dummy driver

On-going blockers:

• Currently I am experiencing problems in porting gptp into the kernel because I was not able to find out until now on how to access the HW clock in kernel space.
  

Resolutions to blockers:

• I will further try to implement the features and do some research. I allready wrote a mail to indu, who is familiar with ptp on linux and when I am not able to finsih until thursday, I will have to work on my schedule together with my mentors in the weekly meeting
  

Plans for the next week:

• Finish implementation of gPTP in kernel space

• Finish implementation of MSRP in kernel space
• Review AVTP protocol
• Start implementing AVTP

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 8 progress report. 

Accomplishments:

• Finish implementation of MSRP in kernel space
• Review AVTP protocol
• Start implementing AVTP

• Review AVDECC protocol
• Start implementing AVDECC (device enumeration part)

On-going blockers:

• I am still trying to implement gptp in kernel space. Currently I am not able to get timestamps from cmsghdr, because those seem to be empty.
  

Resolutions to blockers:

• Because I invested quite some time now in gptp, I will talk to my mentors, if it is still okay to invest some time in fixing the bugs or if I shall move on and use the user space daemon
  

Plans for the next week:

Finish implementation of gPTP in kernel space

Document AVB driver stack

Finish AVTP driver stack

Best,

Niklas (nwan)

21 Jul

[nwan]

Hello,

this is my week 9 progress report. 

Accomplishments:

• Review AVDECC protocol
• Start implementing AVDECC (device enumeration part

• Continue working on AVTP and implement synchronization by HW clock in ALSA driver
• Modularize all AVB components
• Abstract away PTP HW Clock access in kernel space for portability
  

On-going blockers:

• N/A
  

Resolutions to blockers:

• N/A
  

Plans for the next week:

• Finish implementation of AVTP in kernel space (shifted to this week because synchronization by ptp HW clock shall take place in ALSA driver)
• Finish implementation of AVDECC (device enumeration part) in kernel space
• Document AVB driver stack

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 10 progress report. 

Accomplishments:

• Finish implementation of AVTP in kernel space (shifted to this week because synchronization by ptp HW clock shall take place in ALSA driver)
• Finish implementation of AVDECC (device enumeration part) in kernel space
• Document AVB driver stack

On-going blockers:

• N/A
  

Resolutions to blockers:

• N/A
  

Plans for the next week:

• Make completion video
• Bug fixing
• Finish documentation

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 11 progress report. 

Accomplishments:

• Begin completion video
• Bug fixing
• Finish documentation for AVB drivers
  

On-going blockers:

• I was not able to start recording the video because I needed to get a microphone
  

Resolutions to blockers:

• Borrowed the microphone from a friend
  

Plans for the next week:

• Finish youtube video
• Last Bug fixes
• optional: (Join Face drivers with AVB drivers and try to get SPI Bitbang to work)
• Submit pull request of AVB driver stack

Best,

Niklas (nwan)

[nwan]

Hello,

this is my week 12 progress report. 

Accomplishments:

• Finish youtube video
• Last Bug fixes

• Submit pull request of AVB driver stack

On-going blockers:

• N/A
  

Resolutions to blockers:

• N/A
  

Plans for the next week:

• Final Documentation for Submission

Best,

Niklas (nwan)

[nwan]

Hello,

this is my final progress report. I was able to finish the project on time and filled out all required forms at Google. All information regarding the project can be found on my documentation page at: https://niklaswan.github.io/GSoC-Overview/

In the following I will give some short information on the project outcome and list some limitations of the results, further information can be found on the documentation page.

My Achievements during GSoC are:

• Port CTAG Face drivers implemented by henrix for GSoC 2016 to Linux kernel 4.19-rt for BeagleBone AI [Merged driver, Merged overlays, Documentation] ☑
• Port AVB driver stack implemented by indu for GSoC 2017 to Linux kernel 4.19-rt for BeagleBone AI [Open, Documentation] ☑
• Refactor and modularize present AVB driver architecture ☑
• Implement AVDECC protocol part of AVB to allow for device enumeration and control ☑

Limitations of birtual ALSA AVB driver:

• Audio file has to be a multiple of the ALSA period size in length
• Playback of one stream with up to 8 channels
• Playback between on Listener (BBB) and one Talker (BBAI)
• AVDECC identification on an Apple MAC with propriatery AVB soulution of Beagleboard Device using ALSA AVB driver is not working properly (However the avbdiagnose application is able to identify the Beagleboard device)

The whole GSoC experience was a lot of fun and I learned a lot. A big thank you to the BeagleBoard.org-community which allways provided some helpfull pointers and tricks. Also a big thank you to the BeagleBoard.org GSoC team and especially to my mentors rma31, henrix, indu and pdp7.

Best,

Niklas (nwan)

[Jason Kridner]

On Wednesday, September 2, 2020 at 3:05:08 AM UTC-5 nwan wrote:

Hello,

this is my final progress report. I was able to finish the project on time and filled out all required forms at Google. All information regarding the project can be found on my documentation page at: https://niklaswan.github.io/GSoC-Overview/

In the following I will give some short information on the project outcome and list some limitations of the results, further information can be found on the documentation page.

My Achievements during GSoC are:

• Port CTAG Face drivers implemented by henrix for GSoC 2016 to Linux kernel 4.19-rt for BeagleBone AI [Merged driver, Merged overlays, Documentation] ☑
• Port AVB driver stack implemented by indu for GSoC 2017 to Linux kernel 4.19-rt for BeagleBone AI [Open, Documentation] ☑
• Refactor and modularize present AVB driver architecture ☑
• Implement AVDECC protocol part of AVB to allow for device enumeration and control ☑

What were the blockers to making a submission to a mainline kernel mailing list? What could we have done in organizing this project to better enable an upstream submission?

[Jason Kridner]

Oh, and is there a link to the final video?

[nwan]

Regarding blockers for submission to mainline:

For the submission of a mainline patch, the architecture of the whole implementation has to be changed. The synchronization of audio data and the gPTP layer both run in user space. The virtual driver which implements MSRP, AVTP and AVDECC protocols is running in kernel space. Thus those strong dependencies between user space and kernel space code have to be resolved. Because the driver needs gPTP running in user space to work properly.

I allready started to port gPTP to kernel space but I had a blocker which is described here: https://niklaswan.github.io/GSoC-Overview/avb-alsa/

Regarding the link to the final video: https://www.youtube.com/watch?v=xrtGZIX-Pow&feature=youtu.be

You also can find the video link under the Elinux projects page or on my Documentation page.

Best,

Niklas