Losing data on pyserial when viewing graphs

[Ian Reischauer]

Hi everyone!

I've generated a GUI using pyqt5 on windows. 

The GUI has two tabs,

Tab1 -> allows the user to communicate with an embedded system where it also shows tabular data

Tab2 -> displays 6 pyqt graph objects. 

The communication with the embedded system:

This is handled in a thread which continuously checks for data on the serial line (PySerial). 

Once data is received it is quickly processed and then the relevant parameters are updated.

These parameters include values that are displayed on the graphs.

Updating of the Graph:

The graphs are updated using two timers which trigger threads. 

The first thread (which I trigger every 50 ms) shifts the arrays which store the data to give the illusion of a moving graph:

ex: 

self.UpdateGraphDataTimer = QTimer()

self.UpdateGraphDataTimer.timeout.connect(self.UpdateGraphData)

self.UpdateGraphDataTimer.start(50)

def UpdateGraphData(self):

.

.

.

.

#Update Time Axis

self.current = time()

dt = self.current - self.prior

self.TimeAxis[:-1] = self.TimeAxis[1:]

self.TimeAxis[:-1]  = round(self.TimeAxis[-1]+dt, 2)

self.prior = self.current

#Update Y axis

self.Engine_RPM[:-1] = self.Engine_RPM[1:]

self.Engine_RPM[:-1]  = newval 

It should be noted that all 6 graph objects share the same time axis

The second thread then updates the graphs every 75ms:

self.UpdateGraphDisplayTimer = QTimer()

self.UpdateGraphDisplayTimer.timeout.connect(self.UpdateGraphDisplay)

self.UpdateGraphDisplayTimer.start(75)

def UpdateGraphDisplay(self):

.

.

.

.

.

self.EngineRPM_Graph_Object.setData(self.TimeAxis,self.Engine_RPM)

These arrays are generated as follows:

self.time_period = 300

self.TimeAxis = np.zeroes(self.time_period)

self.Engine_RPM = np.zeroes(self.time_period)

Ok so the Issue:

Everything seems to work fine till the arrays fill and the graphs start filling. At this point I assume I'm seeing a massive decrease in performance as the thread that's responsible for obtaining data from the serial line seems to execute less and less often.

Slowly but surely there is buffer overflow. This seems to start happening as soon as the graph scrolls/ arrays are filled. It can be delayed by increasing self.time_period i.e. creating a larger array, however this is undesirable.

Am I doing something incorrectly? How can I ensure that the serial line gets serviced and the performance is kept up while keeping the scroll effect going?

Any help will really be appreciated

[Patrick]

Hi,

Maybe take a look at this:

https://forum.arduino.cc/t/speed-of-analogread/134428/19#msg1270996

I tested it out a while back and did get about 1 megasamples/s streaming from an Arduino Due and plotted in realtime, so perhaps it can be of some use.

Just one tip with this example and threads. You'll see it starts up a native python Thread to acquire the data, while the Qt event loop handles the UI updates. That's fine, but needs some care. You'll see there is a Lock on the data updates to protect it from simultaneous access. If you need to communicate UI updates or similar from inside the native python thread, do not do it directly. Instead, create some Qt Signals and emit() those from inside the python thread, then connect() and respond to the signals from inside the Qt thread.

Patrick