Update shown data from another thread?

[Daniel Polski]

Hello,

I receive different types of data in real time, and am thinking about how to “connect” it to and show it in various (custom) fltk widgets. I would like some advice about which direction to think.

 

Let’s say I have 2 incoming data types:

 

rawData1

rawData2

 

And 3 fltk widgets:

 

fltkWidget1 (showing rawData1)

fltkWidget2 (showing rawData2)

fltkWidget3 (showing rawData1 & rawData2)

 

As described above, many widgets might show the same underlying data.

 

The idea I have is to separate the data from the graphical representation (create custom fltk widgets which draws their rawData), and have a dedicated thread managing the incoming data.

That thread could store pointers to the raw data and update changes, but how do it inform the specific fltk widgets that their underlying data has changed and they need to redraw?

 

Maybe the rawData classes should keep internal static lists of fltk widgets showing the data? Then they could iterate through all their fltk widgets when the data has changed, and set some dirty flag?

 

..Or maybe you have ideas about a better solution?

[Albrecht Schlosser]

On 10.03.2017 11:23 Daniel Polski wrote:

> I receive different types of data in real time, and am thinking about
> how to “connect” it to and show it in various (custom) fltk widgets. I
> would like some advice about which direction to think.
>

> ...

>
> As described above, many widgets might show the same underlying data.
>
> The idea I have is to separate the data from the graphical
> representation (create custom fltk widgets which draws their rawData),
> and have a dedicated thread managing the incoming data.

That's a very good idea and likely really necessary.

> That thread could store pointers to the raw data and update changes, but
> how do it inform the specific fltk widgets that their underlying data
> has changed and they need to redraw?

The best way to do this seems to be Fl::awake(Fl_Awake_Handler func,
void *data)
http://www.fltk.org/doc-1.3/group__fl__multithread.html#ga22a404bcaf6641369e0725627d881556

This will call your 'func()' in a safe way from FLTK's main thread w/o
any FLTK locking issues. You can use data to point to any internal
structures (aka a message) so you know which widgets need updates. In
this func() you can call widget->redraw() so that FLTK will redraw this
widget after it processed your func().

> Maybe the rawData classes should keep internal static lists of fltk
> widgets showing the data? Then they could iterate through all their fltk
> widgets when the data has changed, and set some dirty flag?

See above. You may still need to sync the main thread and your threads
updating 'data' but that can be very short locks, semaphores etc.

> ..Or maybe you have ideas about a better solution?

For more information you may want to take a look at the "Adanced FLTK"
chapter in the docs that deals mainly with threading issues.

http://www.fltk.org/doc-1.3/advanced.html

Note that, although there are other ways to do it,
Fl::awake(Fl_Awake_Handler func, void *data) is the recommended and most
reliable way.

[Daniel Polski]

> That thread could store pointers to the raw data and update changes, but
> how do it inform the specific fltk widgets that their underlying data
> has changed and they need to redraw?

The best way to do this seems to be Fl::awake(Fl_Awake_Handler func,
void *data)
http://www.fltk.org/doc-1.3/group__fl__multithread.html#ga22a404bcaf6641369e0725627d881556

This will call your 'func()' in a safe way from FLTK's main thread w/o
any FLTK locking issues. You can use data to point to any internal
structures (aka a message) so you know which widgets need updates. In
this func() you can call widget->redraw() so that FLTK will redraw this
widget after it processed your func().

Aha..That kind of solution sounds like something I’m looking for but unfortunately I don't understand your explanation fully (even though it probably is crystal clear for someone more familiar with fltk than I am).

Do you have some spare time to elaborate & explain a little further, maybe with some pseudo code? 

[Daniel Polski]

>> That thread could store pointers to the raw data and update changes, but
>> how do it inform the specific fltk widgets that their underlying data
>> has changed and they need to redraw?
>

> The best way to do this seems to be Fl::awake(Fl_Awake_Handler func,
> void *data)
> http://www.fltk.org/doc-1.3/group__fl__multithread.html#ga22a404bcaf6641369e0725627d881556
>
>
> This will call your 'func()' in a safe way from FLTK's main thread w/o
> any FLTK locking issues. You can use data to point to any internal
> structures (aka a message) so you know which widgets need updates. In
> this func() you can call widget->redraw() so that FLTK will redraw this
> widget after it processed your func().

Aha.. This sounds exactly like something I need, but I don't understand
your explanation fully even after reading the docs (even though it

probably is crystal clear for someone more familiar with fltk than I am).

Do you have some spare time to elaborate & explain a little further?
(Some pseudo code maybe?)

[Greg Ercolano]

On 03/10/17 02:23, Daniel Polski wrote:
> I receive different types of data in real time, and am thinking about
> how to “connect” it to and show it in various (custom) fltk widgets.
> I would like some advice about which direction to think.

Depending on the speed of the data coming in,
I'd choose between:

a) use child threads if you need to preprocess data as quickly
as it arrives, arrange it into buffers, and then tell the
fltk main thread the data is ready to be parsed into widgets.
Use locks to synchronize the child threads with the main thread,
and Fl::awake() or fltk timers to regularly check for new data
from the fltk main thread.

b) use an fltk timer for lower speed, "block" oriented reading.
This can avoid the need for threads, and all the problems threading
can cause. See Fl::add_timeout().

c) Use fltk's idle loop to poll the data, e.g. Fl::wait() and friends.

There's probably other approaches I'm forgetting.

If it's realtime video, audio, or similar /large/ amount of data
that is coming in fast, you'd probably want to use a combo of threads and
locks.

When using threads, it's important fltk's main loop run in the main thread,
and keep the child threads simple, and don't let child threads directly
manipulate the widgets; only let the main thread interact with the GUI.

The child threads should just prep the data into a buffer or some such
the main thread can then pass to the widgets. It's best if the child
threads create their own buffers, fill them, then make the buffers
available to the main thread, using locks to synchronize.

For instance, a simple example with just ONE raw data source is below.

Warning: I wrote this off the top of my head just now. Threads are hard
and it's easy to make mistakes, (e.g. a sneaky path that forgets to unlock, etc)
so don't consider this a well tested example.

Also: There's also MANY different ways to skin a cat with threads,
this is one of many approaches, using alternating allocated buffers.
Another way would be a ring buffer, or a single allocated buffer
that basically just fills, etc. Use whatever's appropriate.

// pseudo code!

// Global data to main and child threads
global char *buf = 0; // a pointer to data child has read that main thread needs to handle
// this doubles as a 'data ready flag' as well as passing the data itself
global *buflock = unlocked; // a lock to manage the above buf pointer. Both threads use this lock to sync

// This function runs as a child thread forever.
// It's job is to read data into a buffer and pass the data
// to the main thread whenever it can. Our reading of data can
// safely block without affecting the main thread's response.
// Main thread uses timer to poll for data we provide it.
// We use a lock to synchronize with the main thread.
// The only data common to the two threads are the two globals (buf + its lock)
//
function ChildThread() {
// Forever loop for child thread
while ( 1 ) {
tmpbuf = malloc(size_of_data) // allocate a new local buffer
read_data_from_source(tmpbuf) // fill the thread's local buffer

// Small loop to wait for fltk main thread to finish data we last sent
while (1) {
// Using locks, see if global 'buf' is null
// When it is, that means the fltk main loop has handled last
// data we put there, free()ed the buffer, and is ready for new data.
//
lock(buflock) // blocks until buflock is not locked
if ( lbuf != 0 ) { // fltk main thread hasn't handled our /last/ data, wait..
unlock(buflock)
sleep_a_little() // yield some cpu so we don't slam the processor with our while(1) loop
read_any_new_data_from_source(tmpbuf) // add any new data from source
continue // keep waiting until fltk main thread has nulled out buf
}
buf = tmpbuf // update global buffer with our new data
unlock(buflock) // unlock so fltk main thread can handle the buffer
break
} // end wait for main thread
} // end of forever loop
} // end of function

// FLTK main thread runs this every 1/10th sec
function MainThread() {
lock(buflock)
if ( buf ) {
// Child thread provided us with new data?
send_buf_to_widget() // send new data to widget(s)
free(buf) // free the buffer the child thread made for us
buf = 0 // zero the global buf, signaling child thread we're ready for new data
}
unlock(buflock)
}

int main() {
create_widgets()
start_child_thread(ChildThread)
start_fltk_timer(MainThread, .10) // use timer at 1/10th sec intervals to handle data from child
Fl::run()

}

> As described above, many widgets might show the same underlying data.

In the example I provided, if you have more than one source of data,
create separate locks and buf pointers for each, and synchronize each similarly.

When using threads, as always, avoid deadlocks and races.
Keep locks as short as possible in the child threads, so the main thread
has plenty of time to work with the locked data.

> That thread could store pointers to the raw data and update changes,
> but how do it inform the specific fltk widgets that their underlying
> data has changed and they need to redraw?

fltk timers is one way I like, but there's a few other ways,
like Fl::awake() (which I haven't used)

> Maybe the rawData classes should keep internal static lists of
> fltk widgets showing the data?

I don't know about static being needed, just keep pointers to widgets
at hand, and make sure only the main thread works with them.
Don't let the child work directly with any FLTK widgets or GUI stuff,
that's to be avoided.

> Then they could iterate through all their fltk widgets when the data has changed, and set some dirty flag?

In my case above, the 'flag' is the buffer pointer itself,
and the lock makes sure no two threads access the buffer pointer
at the same time.

[Albrecht Schlosser]

Yes, I know it was only a short listing of what you could use, no direct
advice how to use it.

> Do you have some spare time to elaborate & explain a little further?
> (Some pseudo code maybe?)

Likely not today, but I may try later. Did Greg's explanations and
pseudo code help?

What I described was more or less Greg's point (a), but there are other
ways to do it depending on your requirements, particularly: what is the
expected data rate, in terms of new data points per second. How fast do
you expect your data to arrive?

[Daniel Polski]

Thank you Albrecht and Greg,

I currently don't know the exact incoming data rate, but I'd guess it *can* be in the range of 100-500 packets/second, but normally much lower. The data is pretty simple though (integers/doubles/strings) and won't need much processing. It's normally not the same raw data container beeing updated fast, but instead many containers beeing updated with new numbers. 

I wrapped up an experiment in the style of my initial idea, and would like some help in understanding if, and if so why, it's not a suitable solution to use with fltk or if it has some other drawbacks I'm not aware of. (I didn't implement automatic removal from listeners list and such things to make the example shorter and focus on the overall idea).

I think I should describe some of the goals in more detail to make it easier to understand what I'm doing:

One custom widget will be a table with around 1000 rows, each row having around 20 columns of "streamed" numbers.

(Hopefully with a "color fading effect" after updates.. Seems like add_timeout / repeat_timeout could be used, but that's a later experiment)

Double clicking on a row in that table will open up a new window, showing a graphical representation of some of the numbers (still "streamed" and updated)

It's actually not important that every update of the raw data gets flushed and drawn in the gui, as long as the drawing gets done fast enough to look "fast". (I can't really see what the numbers are changed to/from when the fps gets high enough anyway..)

Anyway.. Here's my litte experiment:

//------------

#include <FL/Fl.H>

#include <FL/Fl_Widget.H>

#include <FL/Fl_Window.H>

#include <Windows.h>

#include <iostream>

#include <thread>

#include <list>

#include <mutex> 

class widgetData{

public:

void addListener(Fl_Widget* aWidget){_my_listeners.push_back(aWidget);}

void value(int data);

int value();

private:

std::list<Fl_Widget*> _my_listeners;

int _my_value;

std::mutex value_mtx;

};

void widgetData::value(int data){

value_mtx.lock();

_my_value = data;

value_mtx.unlock();

std::list<Fl_Widget*>::const_iterator iter;

for (iter = _my_listeners.begin(); iter != _my_listeners.end(); ++iter) {

(*iter)->redraw();

}

}

int widgetData::value(){

int ret;

value_mtx.lock();

ret = _my_value;

value_mtx.unlock();

return ret;

}

class aDataDisplay: public Fl_Widget{

public:

aDataDisplay(int x, int y, int w, int h, const char *label = 0) : Fl_Widget(x, y, w, h, label){ _my_data = NULL; }

void draw(){ std::cout << label() << " drawing "<<_my_data->value()<<std::endl;}

int handle(int event){ return 0; }

void setData(widgetData* data){ _my_data = data; data->addListener(this); }

private:

widgetData* _my_data;

};

struct thread_data

{

bool keep_running;

widgetData* data;

};

void receiveData(thread_data* thread_data)

{

while (thread_data->keep_running){

Sleep(1000);

std::cout << "increasing number" << std::endl;;

thread_data->data->value(thread_data->data->value() + 1);

Fl::check();

}

}

int main(int argc, char* argv[])

{

widgetData* rawData = new widgetData();

rawData->value(0);

thread_data* tData = new thread_data();

tData->data = rawData;

tData->keep_running = true;

std::thread t(receiveData, tData);

Fl_Window *window = new Fl_Window(300, 180, "test");

aDataDisplay display1(0, 0, 10, 10, "display 1");

display1.setData(rawData);

aDataDisplay display2(0, 0, 10, 10, "display 2");

display2.setData(rawData);

window->end();

window->show(argc, argv);

Fl::run();

tData->keep_running = false;

t.join();

return 0;

}

[Albrecht Schlosser]

On 11.03.2017 16:03 Daniel Polski wrote:
> Thank you Albrecht and Greg,

You're welcome.

> I currently don't know the exact incoming data rate, but I'd guess it
> *can* be in the range of 100-500 packets/second, but normally much
> lower. The data is pretty simple though (integers/doubles/strings) and
> won't need much processing. It's normally not the same raw data
> container beeing updated fast, but instead many containers beeing
> updated with new numbers.

OK, the reason why I was asking is that the data rate and hence the
display update rate is very important. Human beings can only notice
updates in a range of up to 60 fps or maybe a little more. Everything
above ~20 fps (or maybe 15) looks like smooth motion, hence videos use
25-30 fps. Everything above 30 fps would be wasted CPU cycles.

If your incoming data processing worker threads would (try to) trigger
display updates with 100-500 fps this would (a) be wasted effort and (b)
likely overload the CPU/GPU. This would block the worker threads and the
entire system. Not a good idea, IMHO.

So I'd say as a rule of thumb: if your expected data rate is lower than
50 data points per second (dpps) you may try to trigger a display update
for each new data point. If it is far lower than 30 dpps on average this
has the advantage that every update will be done instantly.


My initial reply would have been better suited to this "slow data
model": for every new data item your worker thread would have called
Fl::awake(func,data) once
and the main thread would have updated the GUI (widgets). I won't
elaborate about Fl::awake() now since this doesn't seem useful for your
case with high data rate.

In your case you will certainly have to limit the display updates to a
sensible frame rate. One way to do this is to let the worker thread(s)
update the data in internal buffers (one for each thread or for each
data source) and run a timer that polls these internal buffers every
50ms (20fps) or so. If new data arrived the main FLTK thread (in a timer
callback) can safely update widgets and call redraw() on all changed
widgets.

Another way would be to let the worker threads keep an internal clock
and gather data until the next display update needs to be done (maybe
every 40 ms = 25 fps) and only then call Fl::awake(func,data).

> I wrapped up an experiment in the style of my initial idea, and would
> like some help in understanding if, and if so why, it's not a suitable
> solution to use with fltk or if it has some other drawbacks I'm not
> aware of. (I didn't implement automatic removal from listeners list and
> such things to make the example shorter and focus on the overall idea).
>
> I think I should describe some of the goals in more detail to make it
> easier to understand what I'm doing:

> ...

> Anyway.. Here's my litte experiment:

(code removed)

OK, I took a look at your code and rewrote it somewhat for testing
purposes. For those following this thread: the code uses C++11 features
and I could compile and run my rewritten version under Linux. Compile
commands are documented in the source file and below.

However I got error messages when trying to compile using MinGW (32-bit
or mingw-w64). I don't know why though:

$ g++ --version
g++.exe (x86_64-win32-seh, Built by MinGW-W64 project) 6.1.0

$ g++ -o rptest1 -std=c++11 -DINTERVAL=25 `fltk-config --cxxflags`
rptest1.cxx `fltk-config --ldflags` && ./rptest1
rptest1.cxx:56:8: error: 'mutex' in namespace 'std' does not name a type
std::mutex value_mtx;
^~~~~
rptest1.cxx: In member function 'void widgetData::value(int)':
rptest1.cxx:61:3: error: 'value_mtx' was not declared in this scope
value_mtx.lock();
^~~~~~~~~
rptest1.cxx: In member function 'int widgetData::value()':
rptest1.cxx:72:3: error: 'value_mtx' was not declared in this scope
value_mtx.lock();
^~~~~~~~~
rptest1.cxx: In function 'int main(int, char**)':
rptest1.cxx:137:3: error: 'thread' is not a member of 'std'
std::thread t(receiveData, tData);
^~~
rptest1.cxx:155:3: error: 't' was not declared in this scope
t.join();
^

If anybody knows how to compile the attached file with MinGW I'd be
interested...

Anyway, I could compile the attached file rptest1.cxx under Linux, so
here are my comments:

I didn't change the way your program works. The bad thing is that your
worker threads access FLTK widgets (they call Fl::check() and redraw()
and maybe more). Although this can work to a certain degree it is not
portable nor reliable and may crash in unexpected ways. In my first
tests it appeared to work, even with my modifications that show the
values as widget labels (I added copy_label() and use a working draw()
method).

Well, seems to work? No, it doesn't. It appeared to work with one update
per second (INTERVAL=1000 ms), but it failed when I shortened the update
interval (INTERVAL < 50 ms == 20 fps). Here's what I got under Linux:

$ ./rptest1
Run test with INTERVAL = 30 ms ...
[xcb] Unknown request in queue while dequeuing
[xcb] Most likely this is a multi-threaded client and XInitThreads has
not been called
[xcb] Aborting, sorry about that.
rptest1: ../../src/xcb_io.c:179: dequeue_pending_request: Assertion
`!xcb_xlib_unknown_req_in_deq' failed.
Aborted (core dumped)

Note that calling XInitThreads as the error message suggests would not
help. FLTK uses a single-threaded approach with the window system (here
X11) and the issue arises from the worker threads calling FLTK methods.

But sometimes it works with similar intervals, and sometimes it appears
to hang and doesn't even output the "Run test with INTERVAL ..." message.

This shows exactly what I expected: unreliable and unpredictable
behavior. Please keep this in mind.

I'm interested in what happens if you run this modified version under
Windows with short intervals or even with 1000 ms. And: how do you
compile it? Do you use MS Visual Studio?

Now, how to fix this? Never call Fl::check(), Fl::wait() or other
methods that can access widgets from the worker thread(s). You defined a
mutex that synchronizes access to the incoming data, so I think this is
all you need, if that's necessary at all.

What I would do is start a timer (Fl::add_timeout()) right before
Fl::run(). In the timer callback I'd check if new data arrived in one of
the thread buffers. If necessary, lock the mutex for this check. Then
process the data and update the widgets. Lock the mutex (again) to mark
the data as read (processed) so the worker thread(s) can remove the data
from the buffer(s). After that I'd restart the timer with
Fl::repeat_timeout() with a reasonable interval (20-50 ms). This way you
can decouple worker threads retrieving data and the FLTK main thread
that reads the data from internal buffers and updates the widgets.

Did this help?

Some more notes: calling Fl::lock() from a worker thread to sync with
the main FLTK thread is generally OK but can block the worker thread for
a significant time while the FLTK main loop is updating the display
(drawing to the display is a relatively slow operation). The timer
mechanism proposed above has the advantage that the FLTK threads and the
worker threads need to synchronize (lock) only while each thread
accesses the data buffer. This can be very short and is similar to what
the docs say about "lockless programming". And more than that: you get a
reliable display update rate independent of incoming data.

[Albrecht Schlosser]

On 11.03.2017 18:50 Albrecht Schlosser wrote:

> OK, I took a look at your code and rewrote it somewhat for testing
> purposes. For those following this thread: the code uses C++11 features
> and I could compile and run my rewritten version under Linux. Compile
> commands are documented in the source file and below.

> ...

> I didn't change the way your program works. The bad thing is that your
> worker threads access FLTK widgets (they call Fl::check() and redraw()

> and maybe more). ...

As shown in my previous reply the given code (rptest1.cxx) could hang or
crash. The new code (see attached file rptest2.cxx) uses a timer with a
fixed rate (currently 20 fps) to update and redraw the widgets if the
value retrieved by the worker thread changed.

Interesting: I did some CPU measurements and found:
- the mutex lock() and unlock() CPU usage is negligible
- and so is the worker thread's increment of the value
- the far most CPU usage can be seen in the widget's update and
redraw() methods.

Hence it is really important to update the widgets only if necessary and
limit redraw() frequency to a sensible interval.

PS: Daniel, I apologize for getting your name wrong in my first post
with source code (rptest1.cxx).

I intend to start a FLTK example code repository on GitHub for
frequently asked questions. Would you please allow me to use the code
you wrote initially for this example? I would add your name in a notice
if you like. The code should be in the public domain, at least usable
without any restrictions (maybe zlib license or similar, I need to check
that).

[Daniel Polski]

(Quite busy with my month old son, but was too curious so had to test your modified version. Will write a more detailed reply in a few days) 

I'm interested in what happens if you run this modified version under
Windows with short intervals or even with 1000 ms.

Just tested it, both in debug and release build (with /O2 optimization) and it seems to work just fine both with INTERVAL set to 10 or even 1. 

 

And: how do you
compile it? Do you use MS Visual Studio?

Using Visual Studio 2013 in Windows 7. I think I might have installed some newer compiler in this environment long time ago, but not sure. (GUI programming in windows is new to me, I normally code in linux but the target platform for this application is Windows).

[Albrecht Schlosser]

On 11.03.2017 22:49 Daniel Polski wrote:
>
> (Quite busy with my month old son, but was too curious so had to test
> your modified version. Will write a more detailed reply in a few days)

Looking forward to your report.

> I'm interested in what happens if you run this modified version under
> Windows with short intervals or even with 1000 ms.
>
>
> Just tested it, both in debug and release build (with /O2 optimization)
> and it seems to work just fine both with INTERVAL set to 10 or even 1.

I'm glad that it works for you.

> And: how do you
> compile it? Do you use MS Visual Studio?
>
> Using Visual Studio 2013 in Windows 7. I think I might have installed
> some newer compiler in this environment long time ago, but not sure.
> (GUI programming in windows is new to me, I normally code in linux but
> the target platform for this application is Windows).

Thanks for the info. I can try Visual Studio 2015 Community, let's see
if it works. But don't hold your breath...

[Daniel Polski]

Ah, will be interesting to take a look at your new version, thanks again!

 

PS: Daniel, I apologize for getting your name wrong in my first post  

with source code (rptest1.cxx).

No problem!

 

I intend to start a FLTK example code repository on GitHub for
frequently asked questions. Would you please allow me to use the code
you wrote initially for this example? I would add your name in a notice
if you like. The code should be in the public domain, at least usable
without any restrictions (maybe zlib license or similar, I need to check
that).

Feel free to use anything I post in the "open" for any purpose.

No need to add a name this time though since it's a "bad example" of how to do things.. Nothing to be proud of ;)

Unrelated: I seem to have a problem posting to the group with thunderbird (now posting through groups.google.com in a web browser).

When I click reply on a message in thunderbird it seems to be posted to fltkg...@googlegroups.com, is that correct?

[Albrecht Schlosser]

On 11.03.2017 23:03 Daniel Polski wrote:

> I intend to start a FLTK example code repository on GitHub for
> frequently asked questions. Would you please allow me to use the code
> you wrote initially for this example? I would add your name in a notice
> if you like. The code should be in the public domain, at least usable
> without any restrictions (maybe zlib license or similar, I need to
> check
> that).
>
>
> Feel free to use anything I post in the "open" for any purpose.

Thanks.

> Unrelated: I seem to have a problem posting to the group with
> thunderbird (now posting through groups.google.com in a web browser).
> When I click reply on a message in thunderbird it seems to be posted
> to fltkg...@googlegroups.com, is that correct?

Yes, that's correct. That's what I always do.

[Greg Ercolano]

On 03/11/17 14:03, Daniel Polski wrote:
> When I click reply on a message in thunderbird it seems to be posted
> to fltkg...@googlegroups.com, is that correct?

As Albrecht mentioned, that's correct.

Just curious, were you expecting it to be a different address?
I ask because perhaps you saw something different on the website
somewhere that I'd need to fix.

> Quite busy with my month old son

Congrats!

[Daniel Polski]

(Quite busy with my month old son, but was too curious so had to test
your modified version. Will write a more detailed reply in a few days)

> I'm interested in what happens if you run this modified version under
> Windows with short intervals or even with 1000 ms.

Just tested it, both in debug and release build (with /O2 optimization)
and it seems to work just fine both with INTERVAL set to 10 or even 1.

[Daniel Polski]

Den 2017-03-12 kl. 00:45, skrev Greg Ercolano:
> On 03/11/17 14:03, Daniel Polski wrote:

>> When I click reply on a message in thunderbird it seems to be posted
>> to fltkg...@googlegroups.com, is that correct?
>

> As Albrecht mentioned, that's correct.
>
> Just curious, were you expecting it to be a different address?
> I ask because perhaps you saw something different on the website
> somewhere that I'd need to fix.

No, I asked since my emails from thunderbird doesn't seem to end up in
the list for some reason.

>> Quite busy with my month old son
>

> Congrats!

Thank you!

[Albrecht Schlosser]

On 12.03.2017 09:17 Daniel Polski wrote:
> Den 2017-03-12 kl. 00:45, skrev Greg Ercolano:
>> On 03/11/17 14:03, Daniel Polski wrote:
>>> When I click reply on a message in thunderbird it seems to be posted
>>> to fltkg...@googlegroups.com, is that correct?
>>
>> As Albrecht mentioned, that's correct.
>>
>> Just curious, were you expecting it to be a different address?
>> I ask because perhaps you saw something different on the website
>> somewhere that I'd need to fix.
>
> No, I asked since my emails from thunderbird doesn't seem to end up in
> the list for some reason.

This mailing list is moderated, so it can take a while until mails make
it to the list. I just enabled you to post w/o moderation so your posts
should now arrive more or less instantly.

[Daniel Polski]

Den 2017-03-11 kl. 20:37, skrev Albrecht Schlosser:
> On 11.03.2017 18:50 Albrecht Schlosser wrote:
>
>> OK, I took a look at your code and rewrote it somewhat for testing
>> purposes. For those following this thread: the code uses C++11 features
>> and I could compile and run my rewritten version under Linux. Compile
>> commands are documented in the source file and below.
>> ...
>> I didn't change the way your program works. The bad thing is that your
>> worker threads access FLTK widgets (they call Fl::check() and redraw()
>> and maybe more). ...

Ok, I understand. Calling Fl:: functions isn't thread safe (and is not
claimed to be). That could be "fixed" by calling Fl::lock() I think, but
then I still could have the problem with wasting resources if getting
the data to be shown "too fast" from the external system.

Your new example which use a timer to solve that problem looks good.

I did a new experiment and ended up with having a static list of changed
objects in the widgetData class. I think the biggest 2 drawbacks with
that solution would be that
a) Accessing the list needs locking, so the writer thread gets locked
out while fltk is processing the changed objects.
b) If getting data for an object in a higher rate than the fltk fps,
that object will get added several times to the changed_objects list and
be processed several times, since no handling of duplicates is made.

Any other drawbacks I'm not aware of..?

[Albrecht Schlosser]

On 13.03.2017 09:34 Daniel Polski wrote:
> Den 2017-03-11 kl. 20:37, skrev Albrecht Schlosser:
>> On 11.03.2017 18:50 Albrecht Schlosser wrote:
>>
>>> OK, I took a look at your code and rewrote it somewhat for testing
>>> purposes. For those following this thread: the code uses C++11 features
>>> and I could compile and run my rewritten version under Linux. Compile
>>> commands are documented in the source file and below.
>>> ...
>>> I didn't change the way your program works. The bad thing is that your
>>> worker threads access FLTK widgets (they call Fl::check() and redraw()
>>> and maybe more). ...
>
> Ok, I understand. Calling Fl:: functions isn't thread safe (and is not
> claimed to be).

There are certainly Fl:: functions you can call in a thread but as a
simplified rule of thumb: yes. However it is more than that: all widget
methods can also be dangerous, e.g. adding or deleting widgets, showing
or hiding windows, and many more. Changing widgets by assigning a new
value to an internal int variable should be safe, but not assigning a
label with label() or copy_label(). Note: although label() would only
assign a new pointer it might also free the previous label's memory and
have to reset an internal flag, so this is not thread safe (but could be
done with Fl::lock(), see below).

> That could be "fixed" by calling Fl::lock() I think, but

Fl::lock() can be really slooooow since the main thread uses Fl::lock()
while it is processing events and drawing. Drawing is the slowest part
of it. This would block your worker thread for a very long time as
compared to the quick mutex lock for accessing data structures. The
latter should always be preferred.

> then I still could have the problem with wasting resources if getting
> the data to be shown "too fast" from the external system.

That's true as well.

> Your new example which use a timer to solve that problem looks good.

ACK. I'd always use it if the worker thread can deliver more than about
30 data points per second. If the worker thread is slow, let's say
reading one line of text from a very slow modem line (not very realistic
today), then calling Fl::lock() once and modifying the widget directly
might be okay, but dependent on the data flow still too much. But
anyway: if the main thread blocks the worker thread and you miss an
input interrupt or you get a data overrun that's much more problematic.

> I did a new experiment and ended up with having a static list of changed
> objects in the widgetData class. I think the biggest 2 drawbacks with
> that solution would be that

> a) Accessing the list needs locking, so the writer thread gets locked
> out while fltk is processing the changed objects.

I think you can minimize that effect if you lock the list only while you
erase one object from the list, but that's not my strongest part.
According to:
http://en.cppreference.com/w/cpp/container/list

"Addition, removal and moving the elements within the list or across
several lists does not invalidate the iterators or references. An
iterator is invalidated only when the corresponding element is deleted."

Reading the last part makes me wonder if your code is correct because
the iterator is invalidated when the *corresponding* element is
deleted". Isn't that something your code does with:

widgetData::changed_objects.erase(iter++);

You may want to rewrite timer_cb() like this (untested!):

static void timer_cb(void *data) {

std::list<widgetData*>::iterator iter =
widgetData::changed_objects.begin();
while (iter != widgetData::changed_objects.end()) {
widgetData *data = iter++;
data->update_listeners();
widgetData::changed_objects_mtx.lock();
widgetData::changed_objects.erase(data);
widgetData::changed_objects_mtx.unlock();

}

Fl::repeat_timeout(1.0 / 20.0, timer_cb); // restart timer with 20 fps
}

Note: this should be considered pseudo code. I'm absolutely not sure
that this would work, particularly if
widgetData::changed_objects.erase(data); is correct, but I hope you got
the idea. You may need more locks when you start the iterator, but since
the worker thread only adds elements this may not be necessary.

> b) If getting data for an object in a higher rate than the fltk fps,
> that object will get added several times to the changed_objects list and
> be processed several times, since no handling of duplicates is made.

That could be fixed easily, couldn't it? In the test program it wouldn't
matter much though, you'd only use copy_label() multiple times, which is
not very expensive, but it allocates and deallocates memory which should
be avoided if possible. You could store the raw value in the widget and
do the copy_label() only:
(a) if the raw value changed, or
(b) in the draw() method before calling Fl_Box::draw()

It's up to you...

OTOH: in your case the widget uses only the last value. What if you want
to display every single value like in an oscilloscope?

> Any other drawbacks I'm not aware of..?

Not at a first glance over your code. I don't have the time to look
closer or even test right now.