Weekly Progress Report Thread (BeagleBoot)

[Ravi Kumar Prasad]

Week 1

Tasks done this week:

• Forked etcher and added BeagleBoot functionality to it.
• Tested and fixed bugs for same.
• Link for work done: https://github.com/ravikp7/etcher/commit/79dae6c2b38f8a8b844316981113e0806aa87d2f
• Branch for testing: https://github.com/ravikp7/etcher/tree/beagleboot

Tasks planned for next week:

• Make uboot usb transfer working on all platforms.
• Code refactoring for node-beagle-boot
• Start working on TCP/IP proxy server in node-beagle-boot

[Ravi Kumar Prasad]

Week 1

Tasks done this week:

• More bug fixes to Etcher with BeagleBoot integration.

• Branch for testing: https://github.com/ravikp7/etcher/tree/beagleboot

• Tried to debug issues with usb boot on macOS.
• Code refactoring for node-beagle-boot

Tasks planned for next week:

• Provide pull request to etcher.

• Make uboot usb transfer working on all platforms.

• Enable proper API for netconsole access to be used to integrate with BeagleBoot.
• Start working on TCP/IP proxy server in node-beagle-boot.

Issues faced this week:

• U-boot usb boot isn't working on macOS and Windows host.
• After SPL runs on device, it doesn't broadcast BOOTP. Logs https://pastebin.com/Kf9g2mpg
• The SPL device's interface is claimed and after that in few seconds it gets disconnected.
• node-usb debug log for working uboot transfer from old version of uboot: https://pastebin.com/4jMj6rWs
• Debug log for upstream uboot, not working : https://pastebin.com/87Y1Zsxz 
• It shows 'Device not responding'.

Need help in debugging this on uboot side.

[Ravi Kumar Prasad]

Week 2

Tasks done this week:

• Replaced jshint with eslint in node-beagle-boot, refactored code to fix all lint errors.
• Used 2 spaces instead of 4 for indentation, ran js-beautify on code, used ES6 standards for consistent styling across the code.
• Cleaned up code for netconsole, added proper api triggered when first netconsole packet is received.
• Commits: https://github.com/ravikp7/node-beagle-boot/commits/refactor
• Used Wireshark on Windows to check spl usb boot status, found nothing worthy, the SPL device gets detected, interface is claimed, then there's no further requests from device and device gets disconnected after few seconds.
• Etcher AppImage for testing on Linux: https://drive.google.com/open?id=1xlyn-pzMGYE_gY5uspdVT6Y6xb1KybqJ 
• It should be run as root, as etcher doesn't support sudo prompt for usbboot yet.

Tasks planned for next week:

• Working on TCP/IP proxy server in node-beagle-boot

I've not sent an email to u-boot mailing list regarding the issue as I want to invest more time to look into the issue. I couldn't work much this week due to exams. 

[Ravi Kumar Prasad]

Week 3

Tasks done this week:

• Added mDNS parser to node-beagle-boot.
• Commit for same: https://github.com/ravikp7/node-beagle-boot/commit/684243f68685534da612308dabc548c1dae1a925
• The parser doesn't parse the resource record yet, as I couldn't find much resources on 'SRV Record' structure format.
• It took more time than expected due to the complexity of handling DNS name notation and compressing techniques with the binary-parser.

Question Section of parsed mDNS:

[ { len: 25, QName: 'Cloud9 IDE for beaglebone' },

  { len: 5, QName: '_http' },

  { len: 4, QName: '_tcp' },

  { len: 5, QName: 'local' } ]

{ QType: 255, QClass: 1 }

[ { len: 10, QName: 'beaglebone' }, { len: 5, QName: 'local' } ]

{ QType: 255, QClass: 1 }

[ { len: 1, QName: '2' },

  { len: 1, QName: '7' },

  { len: 3, QName: '168' },

  { len: 3, QName: '192' },

  { len: 7, QName: 'in-addr' },

  { len: 4, QName: 'arpa' } ]

{ QType: 255, QClass: 1 }

[ { len: 23, QName: 'Node-RED for beaglebone' },

  { len: 5, QName: '_http' },

  { len: 4, QName: '_tcp' },

  { len: 5, QName: 'local' } ]

{ QType: 255, QClass: 1 }

Tasks planned for next week:

• Working on parsing and serializing the mDNS, DHCP, ARP, HTTP requests.

Issues faced this week:

Packets from board when running debian, we're not getting captured by node-usb. The problem was partly with not claiming other interface and a delay in device claiming which was present in server code for devices other than ROM. Kumar Abhishek helped me in fixing the issue.

I still have exams on tuesday and thursday, I expect to work full time after that.

[Ravi Kumar Prasad]

Week 4

Tasks done this week:

• Currently, for binary parsing and serializing, different node modules were used. The binary-parser was good enough for parsing, but schemapack for serialization was not good enough. A lot of hackaround l(ike merging fields due to lack of support of less than 8 bit fields, splitting single header in multiple parts) was done in order to get it properly working for our case. 
• I've now replaced both with binary-parser-encoder which is a fork of binary-parser module which aims to provide same format for encoding and parsing. As, it was becoming hard to work with two different modules and formats for same packet.
• Implemented code for response to IGMP, ICMPv6, mDNS and DHCP request.
• Re-written IPv4, BOOTP packet make functions using only binary-parser scheme, will gradually move all packet encoding and parsing to same format.

Tasks planned for next week:

• Working on parsing and serializing the ARP, TCP, HTTP requests.
• Replacing all existing protocol encoding scheme with same scheme as binary-parser.
• Route BeagleBone's all other TCP/IP requests to the server.

Issues faced this week:

• When the device interfaces are not claimed and packets are captured over the usb bus, then Wireshark parses each packet protocol correctly. But, when the interface is claimed and then packets are captured over the usb bus, Wireshark doesn't take care of the RNDIS header, it parses it as Ethernet and then shows rest of the packet as Data. 
• I tried to save the capture as .pcap and modify it using bittwist to take offset of bytes upto the rndis header end, in order to get it then parsed by wireshark correctly. But, it failed to do so.
• So, it's quite difficult to keep verify if the binary data for the packets are correctly encoded or parsed without checking with Wireshark.
• Like, I encountered a bug with binary-parser when parsing a 20 bit FlowLabel field in IPv6 header, it skipped a byte, which caused wrong header info on parsing. A lot of time was wasted on it then I realised it by checking every byte. It'd not have been a problem if Wireshark could parse it correctly for verification.

[Ravi Kumar Prasad]

Week 5

Tasks done this week:

• Worked on ICMPv6 checksum calculation using pseudo header for IPv6.
• Last week I implemented mDNS response for a specific type of record. This week, I worked on a general algorithm for encoding all possible responses and requests.

Tasks planned for next week:

• Complete TCP/IP proxy server.
• Work on refactoring BeagleBoot code.
• Start working on U-boot console UI in BeagleBoot app.

[Ravi Kumar Prasad]

Week 6

Tasks done this week:

• I tried to send DHCP request to debian (running on beaglebone), which the board doesn't respond to.
• To make sure the board does receive the requests properly, I used tcpdump on debian (beaglebone) to log all requests on the usb0 interface.
• Since, the board send requests for a very short period over the usb0 interface as soon as debian starts, so tcpdump can't capture all the requests.
• The tcpdump log for the DHCP request:

debian@beaglebone:~$ sudo tcpdump -i usb0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on usb0, link-type EN10MB (Ethernet), capture size 262144 bytes 20:08:52.308930 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 04:a3:16:df:50:c1 (oui Unknown), length 300

• I've now started work on integrating uboot console to the BeagleBoot electron app.

Tasks planned for next week:

• Work on U-boot console UI in BeagleBoot app.

Issues faced:

• I thought the board is rejecting the requests as I was using the same rndis header for the cdc interface. I read cdc spec documents to figure out that it's a cdc ecm interface. But couldn't find any specific header structure for ecm packets. Eventually, verified by tcpdump that request is indeed received.
• The board isn't responding to dhcp (either directed or broadcast) or arp requests, so I'm kind of blocked here on tcp/ip server side.

[Ravi Kumar Prasad]

Week 7

Tasks done this week:

• Figured out why the board was not accepting the dhcp or arp requests. The problem was that the usb eth link got down everytime the interfaces were claimed.
• Here's the output for ip route command after claiming interfaces
• debian@beaglebone:~$ ip route 0.0.0.0 via 192.168.0.1 dev eth0 default via 192.168.0.1 dev eth0 192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.105 192.168.0.1 dev eth0 scope link 192.168.6.0/30 dev usb1 proto kernel scope link src 192.168.6.2 linkdown 192.168.7.0/30 dev usb0 proto kernel scope link src 192.168.7.2 linkdown
  
• To fix this issue, I read the USB CDC ECM specs document. Here's what it says "The Data Class interface of a networking device shall have a minimum of two interface settings. The first setting (the default interface setting) includes no endpoints and therefore no networking traffic is exchanged whenever the default interface setting is selected. One or more additional interface settings are used for normal operation, and therefore each includes a pair of endpoints (one IN, and one OUT) to exchange network traffic. The host shall select an alternate interface setting to initialize the network aspects of the device and to enable the exchange of network traffic."
• So, I made changes in code to claim the ECM interface with zero endpoints and set alternate interface with two endpoints for transfer. This made the usb1 link up.
• I couldn't get the RNDIS (usb0) interface up for networking, tried different interface claiming orders. I guess, it requires some control transfer message and I couldn't find any reference for same yet.
• On Windows also, the ECM (usb1) link got up using same method after replacing the USB Composite Device drivers with WinUSB drivers.
• Read Nodejs net, http module documentation.
• I've started work on local proxy http server which can be used to access the cloud9 server from board by parsing tcp requests directed to from 192.168.6.2/1.

Tasks planned for next week:

• For handling requests other than those directed at 192.168.6.1, I plan to create sockets using tcp info from received tcp packet and writing data to socket from the received tcp payload.
• Implement local port mapping for the ones from board to ones randomly generated by the proxy server for respective proxy requests.
• Finally, hope to complete this proxy server this week.

Issues faced this week:

• On Windows, I was unable to install WinUSB driver for all interfaces as Zadig (driver installer) and Windows Device Manager were not listing all the required interfaces. Then, I found an option in Zadig to list Composite devices, which showed the device with all interfaces, then I could replace the driver.
• The options field in the tcp header should end with 1 byte 0, but, on parsing the tcp packets received from board, there's isn't one and I can't figure out how to determine the tcp header end. Still looking into this.

[Ravi Kumar Prasad]

Week 8

Tasks done this week:

• Added a TCP server with port 80 to listen to requests made to the cloud9 server in board.
• Implemented parsing and encoding of TCP header. I still can't find why the option part (variable in size) in TCP header doesn't end with a one byte '0' as said in RFC document and everywhere to signify the header end. From, observation I found the tcp headers in all requests are of size 12 bytes, so I'm using that fix size.
• I'm currently working on TCP header fields (required with tcp packets for the cloud9 server data) info to be sent to the board.
• I've read the RFC documents to understand the TCP options, the timestamps, Window size etc.
• Currently, working on to implement the tcp header encoding with proper field data.

Tasks planned for next week:

• Complete the handling of local requests made by board to 192.168.6.1
• Work on to create sockets using tcp info from received tcp packet and writing data to socket from the received tcp payload.

[Ravi Kumar Prasad]

Week 9

Tasks done this week:

• Implemented code for putting ethernet packets to host network and capturing same.
• Using this node module https://github.com/mscdex/cap
• So far I've been able to send and receive ping from beaglebone to internet as well as local devices on the host's subnet by the following approach.

Architecture: 

• When a IP packet comes out of the board, the source mac address is changed to mac address of host and source ip is set to some ip in the host's subnet.
• An available IP in host's subnet can be found by sending ping to other addresses n the subnet.
• I also tried to send DHCP request from board to the router in order to get an available IP from it, but there wasn't any response. For this DHCP request, the source mac address was that of beaglebone. But, there wasn't any DHCP response.
• The destination mac address is set to that of host's gateway (router).
• The router's mac address is known by sending an ARP request for host's gateway IP. For accessing gateway IP: https://www.npmjs.com/package/network
• Whenever a device sends in the subnet or the router sends ARP for the IP taken (192.168.0.145 in image), a proxy ARP response is sent that it has mac same of that host.

Configuration to be done on board :

• Each time, I've to manually set the default gateway to 192.168.6.1 from board to be able to send network packets over usb0. sudo /sbin/route add default gw 192.168.6.1
• Also, nameservers are not set up for the usb0 interface.
• There's symlink for /etc/resolv.conf , so everytime I create a file /etc/resolv.conf and it gets removed after reboot.

What's still not working:

• Ping from local devices on subnet to beaglebone.
• DNS name resolution, I can see a DNS query sent from board to Google DNS, but there's no response. So, working on to fix that.
• Accessing HTTP server from beaglebone board.

Tasks planned for next week:

• Fix the remaining issues with the proxy server.
• Test it on all platforms. So far tested only on LInux and Windows.
• Provide builds for testing to the community. 

[Jason Kridner]

On Jul 17, 2018, at 11:26 AM, Ravi Kumar Prasad <7ra...@gmail.com> wrote:

Week 9

Tasks done this week:

• Implemented code for putting ethernet packets to host network and capturing same.
• Using this node module https://github.com/mscdex/cap
• So far I've been able to send and receive ping from beaglebone to internet as well as local devices on the host's subnet by the following approach.

Architecture: 

• When a IP packet comes out of the board, the source mac address is changed to mac address of host and source ip is set to some ip in the host's subnet.

Why would you not make the source IP the host’s IP? Are you filtering out packets with the host’s IP as the destination IP?

• An available IP in host's subnet can be found by sending ping to other addresses n the subnet.
• I also tried to send DHCP request from board to the router in order to get an available IP from it, but there wasn't any response. For this DHCP request, the source mac address was that of beaglebone. But, there wasn't any DHCP response.
• The destination mac address is set to that of host's gateway (router).

As long as the destination IP isn’t on the local subnet. 

• The router's mac address is known by sending an ARP request for host's gateway IP. For accessing gateway IP: https://www.npmjs.com/package/network
• Whenever a device sends in the subnet or the router sends ARP for the IP taken (192.168.0.145 in image), a proxy ARP response is sent that it has mac same of that host.

Configuration to be done on board :

• Each time, I've to manually set the default gateway to 192.168.6.1 from board to be able to send network packets over usb0. sudo /sbin/route add default gw 192.168.6.1
• Also, nameservers are not set up for the usb0 interface.
• There's symlink for /etc/resolv.conf , so everytime I create a file /etc/resolv.conf and it gets removed after reboot.

What's still not working:

• Ping from local devices on subnet to beaglebone.
• DNS name resolution, I can see a DNS query sent from board to Google DNS, but there's no response. So, working on to fix that.
• Accessing HTTP server from beaglebone board.

Tasks planned for next week:

• Fix the remaining issues with the proxy server.
• Test it on all platforms. So far tested only on LInux and Windows.
• Provide builds for testing to the community. 

--
https://beagleboard.org/gsoc
---
You received this message because you are subscribed to the Google Groups "BeagleBoard GSoC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard-gs...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ravi Kumar Prasad]

On Wednesday, July 18, 2018 at 5:03:33 PM UTC+5:30, Jason Kridner wrote:

On Jul 17, 2018, at 11:26 AM, Ravi Kumar Prasad <7ra...@gmail.com> wrote:

Week 9

Tasks done this week:

• Implemented code for putting ethernet packets to host network and capturing same.
• Using this node module https://github.com/mscdex/cap
• So far I've been able to send and receive ping from beaglebone to internet as well as local devices on the host's subnet by the following approach.

Architecture: 

• When a IP packet comes out of the board, the source mac address is changed to mac address of host and source ip is set to some ip in the host's subnet.

Why would you not make the source IP the host’s IP? Are you filtering out packets with the host’s IP as the destination IP?

If the source IP is set same as the host's IP, then SSH from host to board isn't possible. If I ssh to host IP from host, the ssh request won't show up in capture. 

The packets are filtered with the destination IP as assigned IP (one available from host's subnet) and destination MAC address as that of host.

[Ravi Kumar Prasad]

Week 10

Tasks done this week:

• Worked on generalizing the setup for the proxy server as earlier code was written specific to my network setup.
• For communication with devices in host's subnet, made arp table for them.
• Fixed the checksum regeneration with TCP packets using TCP pseudo header, which were getting rejected earlier.

Tasks planned for next week:

• Thorough testing of the proxy server.
• Work on to integrate the proxy server and u-boot console in electron app.

[Ravi Kumar Prasad]

Hello everyone,

I need help in testing the TCP/IP proxy server for BeagleBone.

Tested working on: Linux (Ubuntu 18.04), Windows 10

On OSX, the device can't be claimed as libusb debug logs states that "Another process has opened the device for exclusive access"

Here are the instructions:

Prerequisite hardware: BB(AM335x), USB cable, Serial cable (Required to issue initial commands)

Prerequisite packages: Linux, OSX: libpcap , Windows: npcap

For Windows: Use Zadig to replace default drivers with WinUSB.

1. Clone https://github.com/ravikp7/node-beagle-boot

2. Checkout branch tcpip

3. Install it

    sudo npm install or npm install

4. Run command

    sudo npm run proxy or (For Windows, from Admin cmd) npm run proxy 

    This will show the Proxy IP to be used to access BeagleBone

5. Connect BeagleBone through serial cable and open serial connection.

6. Connect BeagleBone through USB.

7. Now from serial terminal, login to debian and run following commands

    sudo /sbin/route add default gw 192.168.6.1 (To route packets whose destination IP is other than it's gateway)

    sudo ifconfig usb1 mtu 480 (Limit IP packets size to fit USB bulk transfer max size)

8. Add DNS nameserver to following file

    sudo vi /etc/resolv.conf

Add add following line to this file nameserver 8.8.8.8 and save it.

NOTE: All these changes are temporary and get lost after reboot.

9. Now Internet can be accessed by BeagleBone, BB's node server can be accessed from proxy IP and SSH can be done using this proxy IP.

Regards,

Ravi

--
https://beagleboard.org/gsoc
---
You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard GSoC" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard-gsoc/ugOgYfAV1U0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard-gs...@googlegroups.com.

[Ravi Kumar Prasad]

Week 11

Tasks done this week:

• Tested and fixed bugs in the proxy server.

• Fixed bootloader server which got problems due to proxy server code.

• More code refactor.

• Tested U-boot v2018.07, issue still exists with Windows and OSX.

Tasks planned for next week:

• Work on BeagleBoot electron app to add proxy server.

[Ravi Kumar Prasad]

Week 12

Tasks done and issues faced this week:

• Enabled network configuration on BeagleBone side over the USB serial interface. No serial cable needed to configure network now :)

• Commit for same https://github.com/ravikp7/node-beagle-boot/commit/54be20f147af9a5329bbaaaebd58e364c2be4ca3

• It runs and asks username and password, that's it, it then automatically sends those network configuration commands and enters password itself when sudo is required.

• Initially, I tried using this module to enable cdc acm communication, but it's having 'LIBUSB Transfer Stall' while Control Transfer.

• That module was missing some control transfers that were listed in USB CDC ACM spec document.

• But, before implementing same, I tried doing direct In and Out Transfer on the ACM Data interface and Voila! It works without any configuration over the control interface.

Steps to run the proxy server:

• Clone https://github.com/ravikp7/node-beagle-boot/
• Checkout branch tcp-ip
• Run sudo npm start
• Select an option and press Enter.
• It will show the proxy IP address and board can access internet.
• Now this proxy IP can be used to ssh in board and also access the local node server running board.

Since, this is the final week. I'll summarize the work:

• Added node-beagle-boot support to Etcher for usb mass storage and flashing of Beaglebone devices. Work link: https://github.com/ravikp7/etcher/commits/beagleboot
• Refactored bootloader server code and Netconsole support (Initially implemented by Jason) code.
• Added TCP/IP proxy server to node-beagle-boot, which works good on Windows and Linux. On OSX we'd need to override the kernel drivers with kext.
• For the cross platform issues, I'll be working with Parthi to fix those in the upcoming weeks.

Tasks before Final Submission:

• Updating Documentation.

Tasks left:

• Work on BeagleBoot electron app to add proxy server, uboot console (netconsole) and downloading images.

Finally I'd like to thank Kumar Abhishek and Jason Kridner for their guidance throughout the summer.

[Ravi Kumar Prasad]

Hello everyone!

First of all I'm sorry for not being available during the final week and delaying the final presentation video as, I was having some health issues. Now I'm doing well.

Here's my Final presentation video https://youtu.be/r2lbyPsbrf4

It was again a great experience working on the project during the summer. I learnt a lot during this period. I want to thank all mentors for their guidance and support. Looking forward to contributing more to the BeagleBoard.org Foundation.

Thanks

Regards

Ravi Kumar Prasad

 

--