Rogue1 - System Test Rack

[Wayne Keenan]

Hi,

I've been working on an automated test rack for MCU and BLE automated building and testing on and off for a few months and since our initial meeting I've iterated on it and have created a simple advertising test case for the hcipy library against a micro:bit.

I'm using the hcipy lib because at this point it doesn't really matter what the target BLE API is, I'm much more focused on making sure the test infrastructure works and the testing process and any test related interface/API is easy to use.

The two test scenarios for the first iteration of the automated testing of Bleson are BLE scanning and advertising.  

I did a very very basic first version of a scanning test, the advertising test is next on the list.

Below are a few details about the setup of the rack, 'screenshots' at the bottom.

Rack SBC & MCU hardware:

• 4x Raspberry Pi 3 (all net bootable)
  
• 2x Micro:bit
  
• 1x Ti CC2650STK  + CC-DEVPACK-DEBUG
  
• Nordic nrf51dk  (PCA100028)
  
• Nordic nrf52dk  (PCA100040)
  
• Rigado BMD300 eval kit (nrf52)
  
• Arduino Primo (nrf52 & ESP8266)
  
• Arduino Genuino (non-RF)

All are powered by an PoE managed switch and so can be remotely power cycled by a master Pi.

Each nRF5x based board has a remote GDB server attached (e.g. firmware upload, 'soft' reset)

There is a 'buildbot' master instance running on one of the Pi's

On the Pi with all the MCU's attached there is a buildblot slave.

I've (superficially) looked at GitHub integrations for event detection and status reporting but will forego these for now, instead, the bleson GitHub repo will be periodically polled by buildbot for changes and the build and test results will appear on a custom webpage, or maybe a Google Sheet.

The initial test setup for bleson test cases will be (fairly) simple and fixed:

• 1x Pi, running latest Stretch (stock)
  
• 1x CC2650STK
  
• 2x microbit
• one running 'standard' firmware, (with its GPIO's connected to a Pi or MCU to detect the electrical side effects of invoking the BLE IOPinService)
• (ideally) one running a custom (central) firmware which receives serial commands from the bleson test scripts to test the bleson created peripheral on the Pi.

However, the test variables that I plan (hope) to implement in the future are:

1. Pi OS Distribution
   
2. Pi OS Distribution Version, 
   
3. OS Kernel (i.e. bluez) version
   
4. Python Versions (3.x)
   
5. MCU Firmware
   
6. Architecture (e.g. add a x86_64 SBC)
   

Some 'screenshots':

[Barry Byford]

Nice!

[IanH]

Great work!

I've made a fork of my bluepy.device code into the Cellule space. It would be interesting if you could try running `sudo python3 central.py` from it on one or more of the Pi's to see if you can receive advertising reports using it. We may not want to use `bluepy.device` code in its current from so I didn't want to tip it straight into `python-bleson`, but it would be useful to know if the basic approach was going to work on a wider range of hardware.

Cheers
Ian

[Wayne Keenan]

Please see the output below.

Just as a FYI: It doesn't need root as the python binary in the virtual env (mounted on a loopback device because NFS doesn't support  'setcap') has had this run against it: 

sudo setcap cap_net_raw+eip $(eval readlink -f `which python`)

(rogue1_buildbot_slave_env)pi@rogue1:~/local.projects/bluepy.device $ python central.py
startup_next_state 0
Wait...
Sending:Type=01 Payload=b'030c00'
Wait...
Got:Type=04 Payload=b'0e0401030c00'
Delegate called: Type=04 Payload=b'0e0401030c00'
Calling completer for opcode 0x0C03 (<class 'commands.Reset'>)
startup_next_state 1
Wait...
Sending:Type=01 Payload=b'010c089060000000000020'
Wait...
Got:Type=04 Payload=b'0e0401010c00'
Delegate called: Type=04 Payload=b'0e0401010c00'
Calling completer for opcode 0x0C01 (<class 'commands.SetEventMask'>)
startup_next_state 2
Wait...
Sending:Type=01 Payload=b'011000'
Wait...
Got:Type=04 Payload=b'0e0c0101100007b600070f000922'
Delegate called: Type=04 Payload=b'0e0c0101100007b600070f000922'
Calling completer for opcode 0x1001 (<class 'commands.ReadLocalVersion'>)
startup_next_state 3
Wait...
Sending:Type=01 Payload=b'0120080700000000000000'
Wait...
Got:Type=04 Payload=b'0e0401012000'
Delegate called: Type=04 Payload=b'0e0401012000'
Calling completer for opcode 0x2001 (<class 'commands.LESetEventMask'>)
startup_next_state 4
Wait...
Sending:Type=01 Payload=b'6d0c020100'
Wait...
Got:Type=04 Payload=b'0e04016d0c00'
Delegate called: Type=04 Payload=b'0e04016d0c00'
Calling completer for opcode 0x0C6D (<class 'commands.WriteLEHostSupported'>)
startup_next_state 5
Wait...
Sending:Type=01 Payload=b'0b200701100010000000'
Wait...
Got:Type=04 Payload=b'0e04010b2000'
Delegate called: Type=04 Payload=b'0e04010b2000'
Calling completer for opcode 0x200B (<class 'commands.LESetScanParameters'>)
startup_next_state 6
Wait...
Sending:Type=01 Payload=b'0c20020100'
Wait...
Got:Type=04 Payload=b'0e04010c2000'
Delegate called: Type=04 Payload=b'0e04010c2000'
Calling completer for opcode 0x200C (<class 'commands.LESetScanEnable'>)
startup_next_state 7
All done
Wait...
Got:Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fb1'
Delegate called: Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fb1'
Reports received:type=03 addrtype=01 addr=b'\x1bB\xdfu\xb22' RSSI=-79 adv={255: b'\x06\x00\x01\t \x00)\xb4V\x8e\x0c%\xb2\x8c\xff`\x82\xac\xa1\xfb|\x15\x05X\xe5\x17\x7f\xa3\x7f'}
Wait...
Got:Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd7'
Delegate called: Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd7'
Reports received:type=00 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-41 adv={1: b'\x06', 9: b'BBC micro:bit [tegip]'}
Wait...
Got:Type=04 Payload=b'3e0c02010401f615b0c93af500d9'
Delegate called: Type=04 Payload=b'3e0c02010401f615b0c93af500d9'
Reports received:type=04 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-39 adv={}
Wait...
Got:Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fae'
Delegate called: Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fae'
Reports received:type=03 addrtype=01 addr=b'\x1bB\xdfu\xb22' RSSI=-82 adv={255: b'\x06\x00\x01\t \x00)\xb4V\x8e\x0c%\xb2\x8c\xff`\x82\xac\xa1\xfb|\x15\x05X\xe5\x17\x7f\xa3\x7f'}
Wait...
Got:Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd7'
Delegate called: Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd7'
Reports received:type=00 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-41 adv={1: b'\x06', 9: b'BBC micro:bit [tegip]'}
Wait...
Got:Type=04 Payload=b'3e0c02010401f615b0c93af500d8'
Delegate called: Type=04 Payload=b'3e0c02010401f615b0c93af500d8'
Reports received:type=04 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-40 adv={}
Wait...
Got:Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fb6'
Delegate called: Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fb6'
Reports received:type=03 addrtype=01 addr=b'\x1bB\xdfu\xb22' RSSI=-74 adv={255: b'\x06\x00\x01\t \x00)\xb4V\x8e\x0c%\xb2\x8c\xff`\x82\xac\xa1\xfb|\x15\x05X\xe5\x17\x7f\xa3\x7f'}
Wait...
Got:Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd7'
Delegate called: Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd7'
Reports received:type=00 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-41 adv={1: b'\x06', 9: b'BBC micro:bit [tegip]'}
Wait...
Got:Type=04 Payload=b'3e0c02010401f615b0c93af500d9'
Delegate called: Type=04 Payload=b'3e0c02010401f615b0c93af500d9'
Reports received:type=04 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-39 adv={}
Wait...
Got:Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fae'
Delegate called: Type=04 Payload=b'3e2b020103011b42df75b2321f1eff06000109200029b4568e0c25b28cff6082aca1fb7c150558e5177fa37fae'
Reports received:type=03 addrtype=01 addr=b'\x1bB\xdfu\xb22' RSSI=-82 adv={255: b'\x06\x00\x01\t \x00)\xb4V\x8e\x0c%\xb2\x8c\xff`\x82\xac\xa1\xfb|\x15\x05X\xe5\x17\x7f\xa3\x7f'}
Wait...
Got:Type=04 Payload=b'3e1d02010001437b308e58f4110201050d095075636b2e6a732037623433b5'
Delegate called: Type=04 Payload=b'3e1d02010001437b308e58f4110201050d095075636b2e6a732037623433b5'
Reports received:type=00 addrtype=01 addr=b'C{0\x8eX\xf4' RSSI=-75 adv={1: b'\x05', 9: b'Puck.js 7b43'}
Wait...
Got:Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd1'
Delegate called: Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705dd1'
Reports received:type=00 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-47 adv={1: b'\x06', 9: b'BBC micro:bit [tegip]'}
Wait...
Got:Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705de3'
Delegate called: Type=04 Payload=b'3e2602010001f615b0c93af51a0201061609424243206d6963726f3a626974205b74656769705de3'
Reports received:type=00 addrtype=01 addr=b'\xf6\x15\xb0\xc9:\xf5' RSSI=-29 adv={1: b'\x06', 9: b'BBC micro:bit [tegip]'}

[IanH]

Great, so that looks like it's working. I should try to get a microbit working at my end too.

Thanks
Ian