one byte per item in list ( byte array ) for bluetooth RX non BLE

[Ben Biles]

I'm trying to use lists to store individual bytes like a byte array but it seams the

meters are getting numbers larger than 256, maybe more than one byte per item in the list !

just wonder if you can see something wrong here ? It might be a lists question , in which case I'll move this question to the list section on the forum.

The meters were working if I just receive 1 byte at a time and add them to individual variables but I can see the sense in

using a list and checking its x bytes long. I'm clearing the values at the end with setGlobal createEmptyList. Not sure about that!

I want to check for 2 byte header ^^ then test I have received 8 bytes of audio meter data and draw the audio meters.

[ABG]

You are misusing the BytesAvalabletoReceive block.

Instead of adding received data to your array, you are

adding the cumulative backlog length in the BlueTooth

input buffer.

Post your .ino file here, and start reading the

samples in the Arduino and Bluetooth sections of FAQ

https://groups.google.com/forum/#!msg/mitappinventortest/wqJin0qO8M0/8xUafdROAgAJ

ABG

[Ben Biles]

sorry, should have posted this... whats a .ino file , like an .aia ?

[ABG]

 whats a .ino file , like an .aia ? 

It's the source code of the program that's sending the data to your app.

ABG

 

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[Ben Biles]

my application code is baremetal C on my ARM / DSP custom board I made myself. It just sends 10 bytes with DMA over UART to an HC-06 BT module. It picks up on the RX UART pin over DMA any inputs from the app inventor app.

There's really nothing to see in that code. just a 10 byte packet of two 0x5E bytes ( header )  followed by 8 1 byte uint8_t meter values. 1 byte per audio meter in each packet.

the list seams like it just keeps growing, should i be looping over it with indexing or something when the list reaches the 8 bytes needed? 

[Chris Ward]

Hi Ben

the list seams like it just keeps growing, should i be looping over it with indexing or something when the list reaches the 8 bytes needed?

We don't know because we have not seen the code that sends the data. Post your .c file here -by the way, .ino Arduino files also use C.

Are you using a Terminal application to check the received data is as expected?  

Hint: In your App Inventor code, you should only be reading the bytes available ONCE (all the data), and parse it from there - in the first instance, just populate a TextBox and see what you have actually got, because if the packet header complies with Bluetooth, then only the text data will be collect by the App Inventor function.

[ABG]

Can you post your latest AI2 .aia export file here?

It would be easiest to just fix it,

compared to whipping up an example from scratch.

ABG

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[ABG]

It took me a while to locate and extract the relevant parts

using the backpack.

You have 90% redundancy in your blocks,

because you don't use the Any Component blocks

and component lists.

You can also eliminate lots of event blocks

using the new Make Generic event block feature.

See attached for my take on how to flush a buffer and plot

byte values to an arbitrary set of canvases.

I did it like this to make it easier for your version control,

bringing in only relevant blocks.

ABG

[Ben Biles]

This is great ! I'll try and learn how it works tomorrow.

At a quick glance I can see I was missing the byte buffer concept and had no idea how to flush one in App Inv.

like the dynamic lists according to channel number !

Yes the project is 99% redundant and parts only added to test hardware dev.

I'll try and make get this into working example tomorrow , late in Japan here !

[Ben Biles]

OK, I tired this in my main app and I get the same results I had in my last attempt.

gainMeterValues just keeps increasing in size and the app just gets slower and slower.

I fixed remove list item index 1 twice in the flushBufferPastCarats function but didn't seam to help

set Debug text should overwrite previous text rather than append and keep growing, I think ?

if there are more than 1 byte available and bytes are not caretCode "^^" then the list items should be removed.

So i'm not sure why the list keeps growing ?

[ABG]

 fixed remove list item index 1 twice in the flushBufferPastCarats function 

No, to remove list items 1 and 2 (^,^) you must remove item1 twice.

You are confusing lists with arrays.

After removing an item from a list, the remaining item numbers immediately collapse. 

The old item 2 has moved up to item 1,

like standing in line at the bank and the person in front steps up to the window, making you number 1 (next).

So i'm not sure why the list keeps growing ?

Maybe you're drinking from a fire hose?

Compare the delay() in your secret C program and

the clock timer delay.  Also show us the complete clock timer code and settings.

ABG

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[Ben Biles]

I just printed incoming bytes and I can see I'm 6 bytes short between "^^"

I better check my 'secret code' :)

Its not really secret , Its using HAL stm32fxx code for DMA UART etc and I have no idea where I would

start to show a concise example of how its working.

I start a 10 byte circular buffer in DMA the same way I am successfully receiving data from BT. Then the data I add into the buffer is sent out over UART.

I will make some more checks the BT data is coming across as intended.

The byte array going to DMA is correct when the callback is made but I better check with something else these bytes are going out correctly.

Maybe I can make a simple arduino BT reciever and check the packets are being sent as 10 bytes.

Also will try now and set app inventor to receive 10 bytes and see what I get. I think app inventor blocks until 10 bytes received if you set the byte number.

[Ben Biles]

the HC-06 module is set to 57600 so I think that would limit the firehose..

I think I read App Inventor detects the speed of the BT incoming data or do we need to set serial speed ?

[Chris Ward]

Hello Ben

I think I read App Inventor detects the speed of the BT incoming data or do we need to set serial speed ?

Set the serial speed to suit the hardware. On the Arduinos,  9600 is typical.

What could be happening is that the data is arriving too fast for the App? Are you streaming data or just sending a one-off?

When the App Receives streamed data, it should do so within a Clock Timer. The timer should run faster than the data arrives, so that each time a data packet is sent, the whole packet is parsed by the App before the  next packet arrives. 

What is the purpose of the data in the App? Is the App just storing data for later analysis (perhaps on a PC) or is it displaying the data at runtime on the App's device (Smartphone)?

If the data is for human consumption on the device, it should not be displayed faster than every 5 seconds or more, because we humans cannot focus on changing information at high speed. So the microcontroller would send data every 8 seconds (a delay() in the send data loop) and the App process would run every 4 seconds and take <1 seconds to process.

Now, about Lists. You do not have to remove Items, you can simply "flush" the List by resetting it with:

 Edited: See text in red.

[Ben Biles]

OK, I have fixed the data stream so there is now ^^ followed by 8 bytes of meter data.

I had set 4bytes TX in the UART HAL rather than 10! oops !

The meters are moving fast but the list is still growing so not sure what data is being displayed.

Next i will slow down HC-06 rate to 9600 and try flushing the entire list like you say.

The audio meters would be ideal at 22 frames per second. Us sound engineers are very used to watching a horrible amount of meter data simultaneously I'm afraid. Yes it can be exhausting !

I'll have to think more about the timing also. I had assumed the narrow pipe would just restrict the speed and the UART would not ask DMA from more than it needs. that could still be true. will think about it more.

I need to fix also the incoming meter data from DSP. I'm treating it like signed int32 but it could be -1 to +1 float, will check that to :)

Will get there in end and post solution here.

[ABG]

The baud rate is a red herring.

Switching in a baritone fast talker for a soprano fast talker won't help your slow listening brain.

You still have your lips wrapped around the fire hose and no one manning the open fire hydrant valve.

You did not mention the delay() call argument in your C code.

That is the key.

ABG

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[ABG]

Note:  Chris's one line flush will empty the buffer, but lose data.

It could be at the end of the plot routine, so the next incoming batch

will be aligned fresh once enough data has arrived.

ABG

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[Ben Biles]

love the metaphor.. your right.. I need to time the outgoing packets somehow.

at the moment a new packet is sent the moment the DMA callback is triggered. its triggered after ever packet is sent and nothing

slowing it down. its a runaway train :)

 

not sure I can use a delay() , micro has other tasks,  but could use a timer / interrupt maybe.

Chriss, Yes In my earlier attempts somewhere in previous posts I did a list flush after the plots.

But.. there is just to much data coming to quickly i think. but then again , my android phone is a quad core cpu beast with a ton of memory that can do more than wiggle a few lines at 25 frames per second ? 

[ABG]

Double check your AI2 Clock delay to see if it is low.

But there's a limit to that, because it takes time to run the code.

You can slow the transmissions down without a delay() call 

by keeping track of the time since last transmission and only transmitting

if current time exceeds (last transmit time + required delay milliseconds).

I showed how in this thread ...

https://groups.google.com/forum/#!msg/mitappinventortest/BOj2-ThEe3s/VlmXdcVOAgAJ

ABG

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[Ben Biles]

OK, this seams to work really well !

I just look for the two ^^ bytes and then specify the 8 following bytes.

I've done away with the flush buffer. I could send a audio channel number descriptor byte after the two ^^ and

then it can could scale with your scale-able plot function.

It might be dropping packets , but I don't really care as long as its working as fast as possible :)

EDIT : the timer is down to 10 so I guess its not going to go any faster :)

[ABG]

Glad it's working for you.

However, I notice you only read an even number of bytes at a time

in your search for ^^ and in your block data capture.

So if you slip a byte in the incoming data stream,

you won't get a ^^ match until you slip another byte.

ABG

On Tuesday, May 14, 2019 at 12:44:27 PM UTC-4, Ben Biles wrote:

[Ben Biles]

didn't think of that one ! I better add that byte to define number channel count.

I'll try using a timer on the transmit end eventually, its probably better to have control over the byte stream speed! 

 

my project is here in case your interested, https://hackaday.io/project/7041-multichannel-audio-dsp-field-mixer-recorder

I'll mention your help on there next time I do a project log.

[Ben Biles]

Hi ABG,

I went back to using your original idea using the buffer flush as its faster.

there a rough idea of it working here

https://www.youtube.com/watch?v=jfS_0wlP3sM

I changed the caretCode to 0x08 because for some reason its drawing the caretCode and some other odd glitches.

I could tell it was the caretCode since it was drawing at 90 pixels in. The glitch is still happening in the demo video but since

its at 0x08 its less noticeable in low level noise on the meters.

I tried slowing down the packet TX to really slow and it still happens. At 1st I thought it was just packet overrun.

Any ideas of what I could try ? I can't really see how these caretCode numbers are getting through to the plot !

[ABG]

Here's a hypothesis for you ...

The clock.Timer fires at uncontrollable spots in the data stream,

so it's possible it fires between consecutive caret symbols (or their equivalents.)

The length of the list meterCanvases is used to determine

if there's enough data to plot.  

If there is only 1 Canvas in that list

(I see you activating them by check marks),

that would let a caret mark slip through into the plot routine.

If you have the print() primitive available in your C program,

you can avoid this by switching to a text Delimited data stream ...

You are using the Clock Timer to chop off pieces of the incoming data stream

blindly.

There is a way to insure you get only complete messages, using message Delimiters ...

Use println() to separate messages when you transmit.

Print() all data

Set Delimiter = 10 in BlueTooth Designer.

Check for Bytes Available > 0 in blocks,

but then request -1 bytes to get only full  messages.

See https://groups.google.com/forum/#!msg/mitappinventortest/lVaCyAeWm_4/ADjvy05wAwAJ attachments.

Set clock Timer Milliseconds to faster (<)  than xmit milliseconds.

If you still want to stick to a Bytes-only data stream, here's an alternative fix ...

Extend the outer IF list length > 1 test in the flushable function

to add an ELse IF  list length > 0 to return true because list length = 1.

ABG