Problems with usleep()

[Michael Bäuerle]

Doing some more build tests, the current FLTK failed on my machine with
the ancient egcs compiler:
----------------------------------------------------------------------
Compiling Fl_sleep.cxx...
Fl_sleep.cxx: In function `static void Fl::msleep(long unsigned int)':
Fl_sleep.cxx:44: `useconds_t' undeclared (first use this function)
----------------------------------------------------------------------

The problem is that 'usleep()' is not really portable. The original BSD
implementation [1] defined it as:

void usleep(unsigned int)

Later SUSv2 [2] redefined it as:

int usleep(useconds_t)

In 2001 it became part of POSIX.1 [3] and was marked as "obsolete".
The new data type 'useconds_t' was defined in [5] as:
|
| The type useconds_t shall be an unsigned integer type capable of
| storing values at least in the range [0, 1000000].

Otherwise 'usleep()' is allowed to fail with -1/EINVAL.

In 2008 it was removed from POSIX.1 [4] and was replaced by
'nanosleep()'.

=================

This means: The current usage in "Fl_sleep.cxx" fails on old Unix
systems and will fail on new Unix systems (that follow POSIX.1-2008 and
successors) in the future.

In addition the way FLTK uses the POSIX.1-2001 variant is invalid
because it overflows the data type 'useconds_t':
|
| void Fl::sleep(double seconds)
| {
| Fl::usleep((unsigned long long) (seconds*1000000));
| }
|
| void Fl::usleep(unsigned long long microseconds)
| {
| ::usleep((useconds_t) microseconds);
| }

A strictly conforming OS implementation is allowed to fail for every
parameter value of 1 'seconds' or greater. The return value is not
checked by FLTK at all and the error will silently result in "no delay".


IMHO this implementation should be completely rewritten. I can create
a STR with patch if the devs agree with my analysis.

Because it may be hard to impossible to implement this correctly without
the help of the build system, and it requires testing, I propose to do
this after the 1.3.3 release. My old test system is exotic and on
mainstream platforms like Linux the 'usleep()' implementation seem to
accept larger values and can handle them.

_________________________
[1] http://www.retro11.de/ouxr/43bsd/usr/man/cat3/usleep.0.html
[2] http://pubs.opengroup.org/onlinepubs/7908799/xsh/usleep.html
[3] http://pubs.opengroup.org/onlinepubs/009695399/functions/usleep.html
[4]
http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xsh_chap03.html
[5]
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/sys/types.h.html

[Albrecht Schlosser]

On 29.10.2014 12:42, Michael Bäuerle wrote:

[ Very good analysis and comments elided ]

Thanks for your very good analysis.

> IMHO this implementation should be completely rewritten. I can create
> a STR with patch if the devs agree with my analysis.

I didn't look at the details, but my experience says that we can "trust"
you and your analysis.

> Because it may be hard to impossible to implement this correctly without
> the help of the build system, and it requires testing, I propose to do
> this after the 1.3.3 release.

Honestly said, I don't believe that such "portable" sleep functions
belong into FLTK at all. My fault was that I didn't object when they
were added, but I was somewhat "offline" for a while and didn't see it
immediately.

FLTK's philosophy is NOT to have the bloat other libraries have, and as
such the Fl::*sleep() family of functions doesn't belong into FLTK,
although it is not a big addition. However, it is not needed at all in
the FLTK library, and there's no reason for a GUI toolkit to expose such
sleep methods.

> My old test system is exotic and on
> mainstream platforms like Linux the 'usleep()' implementation seem to
> accept larger values and can handle them.

So before you start doing work that should not be done, and before we
make configure and CMake even more complicated, we should better REMOVE
the entire function family NOW!

I apologize for "crying", but this is REALLY URGENT (thanks for the
heads-up before the release), because it is easier to add functions than
to remove them later.

URGENT REQUEST TO DEVS:

Please vote for REMOVAL of Fl::*sleep() methods before the release of
FLTK 1.3.3. I'll do it, if voted positive OR if I don't get enough votes
before the release.

In the latter case we can add them later with correct (positive) voting,
if applicable. I don't remember that anybody voted before someone added
these new methods (please correct me, if I missed it).

My vote for removal is +1.

[MacArthur, Ian (Selex ES, UK)]

> URGENT REQUEST TO DEVS:
>
> Please vote for REMOVAL of Fl::*sleep() methods before the release of
> FLTK 1.3.3. I'll do it, if voted positive OR if I don't get enough
> votes
> before the release.
>
> In the latter case we can add them later with correct (positive)
> voting,
> if applicable. I don't remember that anybody voted before someone added
> these new methods (please correct me, if I missed it).
>
> My vote for removal is +1.

I didn't object when they were added (beyond noting that I wasn't all that keen on the Win32 implementation) but I take Albrecht's point that they don't entirely look like something that should be in a "fast, light" GUI toolkit.

(As it stands, I do not (and indeed am unlikely ever to) use them, so it wouldn’t make any difference to me if they were excised, or moved to their own lib, or some such thing.)

I guess that makes me sort of +1 on removing them for now?


ISTR (and I may well be making this up...) that there was something, a define or such, in the POSIX that allows to detect at compile time (not at configure time) whether usleep() and/or nanosleep() are defined and compile in accordingly, so if we decided to keep these, it may not need a configure change.

Though... if usleep() exists with inconsistent parameters, then that may well require a configure time check.








Selex ES Ltd
Registered Office: Sigma House, Christopher Martin Road, Basildon, Essex SS14 3EL
A company registered in England & Wales. Company no. 02426132
********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

[Manolo Gouy]

> Please vote for REMOVAL of Fl::*sleep() methods before the release of FLTK 1.3.3. I'll do it, if voted positive OR if I don't get enough votes before the release.
>
> In the latter case we can add them later with correct (positive) voting, if applicable. I don't remember that anybody voted before someone added these new methods (please correct me, if I missed it).
>
> My vote for removal is +1.

My vote is also +1 for removal

[Michael Bäuerle]

MacArthur, Ian (Selex ES, UK) wrote:
>
> ISTR (and I may well be making this up...) that there was something, a
> define or such, in the POSIX that allows to detect at compile time (not
> at configure time) whether usleep() and/or nanosleep() are defined and
> compile in accordingly, so if we decided to keep these, it may not need
> a configure change.

In general, the header file "unistd.h" [1] should define the macro
"_POSIX_VERSION" to indicate what standard is implemented.

But 'nanosleep()' was not mandatory before POSIX.1-2008 (instead was
part of the "Timers option" since POSIX.1b-1993).

To make this worse:
Some fairly recent *BSD systems claiming to implement only POSIX.1-1990,
even if they have 'nanosleep()' (a single missing, but mandatory
function of the newer POSIX standards may be the cause).
But in POSIX.1-1990 there was no 'usleep()' too and you need further
checks. This is sadly much more nontrivial than it looks like.

> Though... if usleep() exists with inconsistent parameters, then that may
> well require a configure time check.

That is one problem.

Another problem is the handling after the call was interrupted by a
signal. Every abstract implementation of a delay should be able to
guarantee that *at least* the specified time has elapsed - more is
acceptable, but not less. This means simply ignoring the return value
is not acceptable and the aborted delay must be resumed after the signal
handler returns. Otherwise the resulting guarantee my be as undefined as
"anything from zero to infinite delay".

A third problem is that a 'usleep()' implementation is not required
to be thread-safe [2]. FLTK currently use different other such functions
and this is a problem for MT-applications and should be addressed for
1.3.4. I have recently taken a look at this, but currently no solution
ready.

________________
[1]
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html
[2] http://pubs.opengroup.org/onlinepubs/009695399/functions/usleep.html

[MacArthur, Ian (Selex ES, UK)]

> In general, the header file "unistd.h" [1] should define the macro
> "_POSIX_VERSION" to indicate what standard is implemented.

Yes, just been looking at that.

That's not what I was thinking of though - I had imagined there were feature macros with finer granularity for this, but it turns out I've spent too much time with various embedded real-time OS recently and just got confused... Some of the "posix-like" OS I've been poking at recently *do* have feature macros for specific timer API's and such, but not POSIX itself, nor Linux so far as I can see. So, nothing we can usefully use...

>
> But 'nanosleep()' was not mandatory before POSIX.1-2008 (instead was
> part of the "Timers option" since POSIX.1b-1993).
>
> To make this worse:
> Some fairly recent *BSD systems claiming to implement only POSIX.1-
> 1990,
> even if they have 'nanosleep()' (a single missing, but mandatory
> function of the newer POSIX standards may be the cause).
> But in POSIX.1-1990 there was no 'usleep()' too and you need further
> checks. This is sadly much more nontrivial than it looks like.

OK. Tricky...

>
> > Though... if usleep() exists with inconsistent parameters, then that
> may
> > well require a configure time check.
>
> That is one problem.
>
> Another problem is the handling after the call was interrupted by a
> signal. Every abstract implementation of a delay should be able to
> guarantee that *at least* the specified time has elapsed - more is
> acceptable, but not less. This means simply ignoring the return value
> is not acceptable and the aborted delay must be resumed after the
> signal
> handler returns. Otherwise the resulting guarantee my be as undefined
> as
> "anything from zero to infinite delay".

OK, yup... I suppose clock_nanosleep() set to TIMER_ABSTIME helps get us round the signals issues etc... except it likely is not widely supported on older systems either...

No good answers here.

> A third problem is that a 'usleep()' implementation is not required
> to be thread-safe [2]. FLTK currently use different other such
> functions
> and this is a problem for MT-applications and should be addressed for
> 1.3.4. I have recently taken a look at this, but currently no solution
> ready.

Note that the graphics context itself is not thread safe, so making a thread-safe GUI library may not really be practical. As a result, we probably don't need to sweat too hard on this, perhaps?

[Greg Ercolano]

On 10/29/14 06:41, Albrecht Schlosser wrote:
>> Because it may be hard to impossible to implement this correctly without
>> the help of the build system, and it requires testing, I propose to do
>> this after the 1.3.3 release.
>
> Honestly said, I don't believe that such "portable" sleep functions
> belong into FLTK at all. My fault was that I didn't object when they
> were added, but I was somewhat "offline" for a while and didn't see it
> immediately.

I can't remember the history of why these were added,
my guess is wrapper functions similar to what we were
doing with strcasecmp() etc.

But yeah, we shouldn't have them if we don't use them
ourselves, and if we do, perhaps they should be static
(private to FLTK's internals) or at least documented
as for internal use only if they have to be public.

[Ian MacArthur]

Fabien added them fairly recently - I don’t think we use them internally at all, they were more about making a portable timer interface...

[Michael Bäuerle]

Greg Ercolano wrote:

> Albrecht Schlosser wrote:
> >
> > Honestly said, I don’t believe that such "portable" sleep functions
> > belong into FLTK at all. My fault was that I didn’t object when they
> > were added, but I was somewhat "offline" for a while and didn’t see it
> > immediately.
>
> I can’t remember the history of why these were added,
> my guess is wrapper functions similar to what we were
> doing with strcasecmp() etc.
>
> But yeah, we shouldn’t have them if we don’t use them
> ourselves, and if we do, perhaps they should be static
> (private to FLTK’s internals) or at least documented
> as for internal use only if they have to be public.

Even if we decide to export them as official API, the functions should
be really portable and reliably working before we do that.

Greping through the FLTK source tree, the only usage of usleep() seems
to be the demo program in "examples/progress-simple.cxx". Therefore I
agree with Albrecht to say +1 for removing them in 1.3.3.

[Michael Bäuerle]

MacArthur, Ian (Selex ES, UK) wrote:

> Michael Baeuerle wrote:
> >
> > A third problem is that a ’usleep()’ implementation is not required
> > to be thread-safe [2]. FLTK currently use different other such
> > functions and this is a problem for MT-applications and should be
> > addressed for 1.3.4. I have recently taken a look at this, but
> > currently no solution ready.
>
> Note that the graphics context itself is not thread safe, so making
> a thread-safe GUI library may not really be practical. As a result,
> we probably don’t need to sweat too hard on this, perhaps?

I agree for the GUI part, and for that part a locking mechanism via
'Fl::lock()' and 'Fl::unlock()' is already available to protect the
FLTK internals.

What is missing is a similar mechanism to protect the OS internals.
Consider an application with a main GUI thread and additional worker
threads. All the worker threads are not allowed to use OS functions
that are both, not thread-safe and used internally by FLTK (in the main
thread).

I have searched the FLTK source based on SUSv2 [1] and found:
- getpwnam() [2] used in
=> src/filename_expand.cxx
Possible thread-safe replacement: getpwnam_r()
- localtime() [3] used in
=> src/Fl_Clock.cxx
Possible thread-safe replacement: localtime_r()
- rand() [4] used in
=> src/Fl_Preferences.cxx
Possible thread-safe replacement: rand_r()
But rand_r() is marked as obsolete in POSIX.1-2008 and the recommended
drand48() is not thread-safe too
- readdir() [5] used in
=> src/scandir_posix.c
There is no thread-safe replacement because this is a last ressort
fallback if the OS provides no scandir(). See my comments in the
source why readdir_r() is broken and should not be used
- strtok() [6] used in
=> src/Fl_Native_File_Chooser_GTK.cxx
Possible thread-safe replacement: strtok_r()
- getenv() [7]
=> src/Fl_get_system_colors.cxx
=> src/Fl_File_Chooser2.cxx
=> src/fl_open_uri.cxx
=> src/fl_utf8.cxx
This propagates the problem to all other locations where
'fl_getenv()' is used instead
=> src/Fl_File_Icon2.cxx
=> src/fl_utf.c
No standard thread-safe replacement up to current POSIX.1-2008
- putenv() [8] used in
=> src/Fl_get_system_colors.cxx
=> src/Fl_display.cxx
No standard thread-safe replacement up to current POSIX.1-2008,
'setenv()' is not thread-safe too
- strerror() [9] used in
=> src/Fl_File_Chooser2.cxx
=> src/fl_open_uri.cxx
=> src/Fl_File_Icon2.cxx
=> src/Fl_Help_View.cxx
Possible thread-safe replacement: strerror_r() or strerror_l()

For some of them there are thread-safe replacements, but currently not
for all. So avoiding them completely is not always possible.

If somebody want to write a FLTK based MT-application on a POSIX OS,
there is currently only one option:
A) Use only thread-safe POSIX APIs in *all* worker threads
This means avoiding *all* non-thread-safe functions listed in [1].
Because with shared FLTK library such functions may be added to
FLTK in the future, it is not sufficient to avoid only the ones
listed above!

Because for some of the functions in [1] there is no thread-safe
replacement defined in the current POSIX standard (or only one that
is broken), the requirement A) makes worker threads complicated if
things must be delegated to the main thread and the result transferred
back. It would be desirable to have:
B) Something like 'Fl::lock_os()' to control a mutex that enables all
the threads (including the GUI thread) to serialize access to
non-thread-safe OS functions.


I don't know how the situation is on Windows.

______________
[1] http://pubs.opengroup.org/onlinepubs/7908799/xsh/threads.html
[2]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/getpwnam.html
[3]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/localtime.html
[4] http://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html
[5]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir.html
[6]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/strtok.html
[7]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/getenv.html
[8]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/putenv.html
[9]
http://pubs.opengroup.org/onlinepubs/9699919799/functions/strerror.html

[Albrecht Schlosser]

On 30.10.2014 11:17, schrieb Michael Bäuerle wrote:
> Greg Ercolano wrote:
>> Albrecht Schlosser wrote:
>>>
>>> Honestly said, I don’t believe that such "portable" sleep functions
>>> belong into FLTK at all. My fault was that I didn’t object when they
>>> were added, but I was somewhat "offline" for a while and didn’t see it
>>> immediately.

> Even if we decide to export them as official API, the functions should
> be really portable and reliably working before we do that.
>
> Greping through the FLTK source tree, the only usage of usleep() seems
> to be the demo program in "examples/progress-simple.cxx". Therefore I
> agree with Albrecht to say +1 for removing them in 1.3.3.

Yes, that's the system usleep(). There were some usages introduced with
the new sleep functions (likely as test cases) in test/sudoku.cxx.

I reverted all the relevant svn updates and committed it in svn r10419.
The comment should read "... Mac OS X" (not "Mac OX"), obviously.

I'm going to test the new Windows IDE builds now, but I can't test Mac
OS X, so any help would be appreciated. TIA

[Manolo Gouy]

>
> I'm going to test the new Windows IDE builds now, but I can't test Mac OS X, so any help would be appreciated. TIA

I have tested the make build under Mac OS X. It runs smoothly. I will now test the Xcode build.
I have committed a minor detail so the version of the Fluid program is correctly shown by the « About Fluid » menu item.

[fabien]

I really thought that we would make it better if some minor incompatibilities would have been found as we usually did in the past, but I do agree that it was preferable to remove it before a major release if it could create more problems than really helping,
I sincerely apologize if it caused some late minute worries, it seemed well accepted for a while but it is true that many of us don't check on the forum as frequently as before.

Here's my 2 cents though for future potential improvements:
I think fltk 'philosophy' does not say that we should have a three legged sheep,  i.e: we do implement partial thread and semaphores API because we needed somehing portable here
, while ignoring the importance of thread sleep api in modern multithreaded development ?

Many modern toolkits and sdks seem to disagree and implement sleep() in a portable way, why not us.
Of course, one  could continue reinventing the wheel and add defines when needed or sacrifice the 'light' aspect of their deployment and add libs like boost that do implement sleep functionaly from the thread api.
About GUI toolkit state of art that contradicts what you may think : just have look at the highly regarded new JUCE (very light yet powerful) c++  toolkit, it does implement sleep too in ts Thread class ...

I was willing to provide a substitute that would avoid extra toolkit need with a _very_ light, easy to use implementation.
Just thinking loud here, no judgment, I'm just curious about what others think on the mid term.

-Fab

[fabien]

Here's a proposal for the unix / nanosleep() aware os'es side:

We would implement only one sleep function accepting a double value then liek expressed previously would decompose into secs, nsecs

// thread sleep for secs seconds, return true if sucessful false otherwise

Example implementation for the discussed unix side case (only for now) not relying on usleep() and still high precision one-size-fits-all sleep:

#include <errno.h>

#include <time.h>

bool fl_sleep_ux(double secs)

{

  struct timespec tv;

  tv.tv_sec = (time_t) secs;

  tv.tv_nsec = (long) ((secs - tv.tv_sec) * 1E9);

  return nanosleep(&tv, &tv)==0 ? true : false;

}

-Fab

[Michael Bäuerle]

fabien wrote:
>
> Here's a proposal for the unix / nanosleep() aware os'es side:
>
> We would implement only one sleep function accepting a double value then
> liek expressed previously would decompose into secs, nsecs
>
> // thread sleep for secs seconds, return true if sucessful false otherwise

I would prefer a variant without return value ("call and forget"). It
should handle all errors internally, e.g. resume the delay after
interruptions by signals.

> Example implementation for the discussed unix side case (only
> for now) not relying on usleep() and still high precision
> one-size-fits-all sleep:
>
> #include <errno.h>
>
> #include <time.h>
>
> bool fl_sleep_ux(double secs)
>
> {
>
> struct timespec tv;
>
> tv.tv_sec = (time_t) secs;
>
> tv.tv_nsec = (long) ((secs - tv.tv_sec) * 1E9);
>
> return nanosleep(&tv, &tv)==0 ? true : false;
>
> }

Such an implementation would not be backward compatible too. And the
caller must handle interruptions by signals itself.

Using 'nanosleep()' is only possible on modern systems. For a long
time it was part of the POSIX realtime extension and we must expect
that some supported systems don't implement it. Later it was moved
to the Timers open, this means it was still no mandatory function.
It became mandatory as part of the POSIX.1 Base not before the latest
version from 2008.

Therefore we need a fallback in any case and this fallback is likely
not thread-safe out-of-the-box. So we should first think whether this
is required/desired or not. If not, 'usleep()' can be considered as
the fallback or maybe the FLTK timer callback facility.

[Albrecht Schlosser]

On 04.12.2014 12:15 Michael Bäuerle wrote:

> fabien wrote:
>>
>> Here's a proposal for the unix / nanosleep() aware os'es side:
>>
>> We would implement only one sleep function accepting a double value then
>> liek expressed previously would decompose into secs, nsecs
>>
>> // thread sleep for secs seconds, return true if sucessful false otherwise
>

> I would prefer a variant without return value ("call and forget"). It
> should handle all errors internally, e.g. resume the delay after
> interruptions by signals.

If we decided to add such a function (and I'm not keen on doing it),
then I agree with you that we should do it "right". And I agree that the
only "right" implementation would be with a guaranteed delay, so that we
don't need a return code.

>> Example implementation for the discussed unix side case (only
>> for now) not relying on usleep() and still high precision
>> one-size-fits-all sleep:
>>
>> #include <errno.h>
>>
>> #include <time.h>
>>
>> bool fl_sleep_ux(double secs)
>>
>> {
>>
>> struct timespec tv;
>>
>> tv.tv_sec = (time_t) secs;
>>
>> tv.tv_nsec = (long) ((secs - tv.tv_sec) * 1E9);
>>
>> return nanosleep(&tv, &tv)==0 ? true : false;
>>
>> }
>

> Such an implementation would not be backward compatible too. And the
> caller must handle interruptions by signals itself.

I believe that backwards compatibility is the hardest problem. We do not
only need to check if a particular sleep function exists, but we also
need to _know_ about the semantics of a particular implementation (does
it fail if interrupted, do we need to resume etc.).

> Using 'nanosleep()' is only possible on modern systems. For a long
> time it was part of the POSIX realtime extension and we must expect
> that some supported systems don't implement it. Later it was moved
> to the Timers open, this means it was still no mandatory function.
> It became mandatory as part of the POSIX.1 Base not before the latest
> version from 2008.
>
> Therefore we need a fallback in any case and this fallback is likely
> not thread-safe out-of-the-box. So we should first think whether this
> is required/desired or not. If not, 'usleep()' can be considered as
> the fallback or maybe the FLTK timer callback facility.

This sounds difficult to get right. I'm not keen on digging deep into
the internals and ancient implementations of any of these functions. If
we did it, we'd have to do it right on all supported platforms, and this
is likely to be impossible or at least very hard.

After all I don't see any reason why we should have such a function in
the FLTK core. We do not actively use threads, and we do not even have
any portable thread functions (except test/threads.{cxx|h} which serves
as an example, but not as a full implementation). The only thread
related functions we provide are the locking functions and of course
Fl::awake() and related functions.

Finally, as I said above, I agree with Michael that "our" version of a
sleep function should guarantee a minimum delay. However I do also see
the necessity of a sleep function that may be interrupted in a thread.
Hence the programmer needing such a function would have to implement it
anyway (or should we make it even more complicated and provide both?).

That said: if the majority of devs voted for adding such a sleep
function, and if someone (Michael together with Fabien or anybody else)
could implement a sleep function with guaranteed properties on all our
target platforms (back to ancient platforms like ... (don't know, fill
in the oldest, maybe SGI?)), then ... okay.

However, I still believe that doing it _right_ would only be much work
for little gain or even more support problems if anything doesn't work
as expected by someone or on a platform we couldn't test.

[fabien]

../..

> Example implementation for the discussed unix side case (only
> for now) not relying on usleep() and still high precision
> one-size-fits-all sleep:
>
> #include <errno.h>
>
> #include <time.h>
>
> bool fl_sleep_ux(double secs)
>
> {
>
>   struct timespec tv;
>
>   tv.tv_sec = (time_t) secs;
>
>   tv.tv_nsec = (long) ((secs - tv.tv_sec) * 1E9);
>
>   return nanosleep(&tv, &tv)==0 ? true : false;
>
> }

Such an implementation would not be backward compatible too. And the
caller must handle interruptions by signals itself.

It is just an example to illustrate the discussion with proof driven considerations, of course it is not intended to be the final version to implement, consider it as pseudo code here.

The signal you are talking about (i.e. EINTR) can be read with errno if false is returned so it is still possible to test.

IMHO, a good design should permit to to frequent tasks easily, less frequent (i.e. lower granularity) tasks still possible.

Keeping the return of the sleep method high level though(i.e. not an int) , will certainly ease integration to various heterogeneous systems.

Using 'nanosleep()' is only possible on modern systems. For a long 

time it was part of the POSIX realtime extension and we must expect
that some supported systems don't implement it. Later it was moved
to the Timers open, this means it was still no mandatory function.
It became mandatory as part of the POSIX.1 Base not before the latest
version from 2008.

 

Indeed, which is why I mentioned on the nanosleep() aware systems only upper in my previous message.

Therefore we need a fallback in any case and this fallback is likely
not thread-safe out-of-the-box. So we should first think whether this
is required/desired or not. If not, 'usleep()' can be considered as
the fallback or maybe the FLTK timer callback facility.

For the tread-safe point, I think this should be the vorder line not cross, we should just document the limitations of some systems not implementing safe API. A work is still possible when really needed in that case.
Later this can be improved, while keeping the current behavior compatible.

That's what we always did, nobody says it would be easy but in the past we used to collaborate together to make it compatible and satisfactory.

Where are the developers I used to know ? Are we just all getting too old or can we still build something interesting together :)

I don't even have the availability of the old compilers you mentioned Michael so more tha ever let's make it happen and see how compatible we can be.

Worst case scenario, some systems just won't have it if not detected during configuration, then we can fall back on usleep or other strategy to implement it.

Albrecht and others I read him on that thread as it seems, are our necessary evil ; that will not forgive incompatibilities or bad practices as the result of creative thinking and this is very important to have people that do care about / review this.
1.x branch is not 2.x or 3.x branch, it has always been considered the most stable branch and I understand we want it to stay that way.
Yet, we are developers ; it would be nice to continue to develop building blocks together, not to mention it is both useful and funnier that abi compatibility discussions ;-)

-Fab

[Michael Bäuerle]

fabien wrote:
>
> Where are the developers I used to know ? Are we just all getting
> too old or can we still build something interesting together :)
>
> I don't even have the availability of the old compilers you
> mentioned Michael so more tha ever let's make it happen and see
> how compatible we can be.
>
> Worst case scenario, some systems just won't have it if not
> detected during configuration, then we can fall back on usleep or
> other strategy to implement it.
>
> Albrecht and others I read him on that thread as it seems, are
> our necessary evil ; that will not forgive incompatibilities or bad
> practices as the result of creative thinking and this is very
> important to have people that do care about / review this.
> 1.x branch is not 2.x or 3.x branch, it has always been considered
> the most stable branch and I understand we want it to stay that way.
> Yet, we are developers ; it would be nice to continue to develop
> building blocks together, not to mention it is both useful and
> funnier that abi compatibility discussions ;-)

I think Albrecht is right when he looks closely at the ABI. This is
important and all users of the shared library will suffer otherwise.

And I agree with Albrecht too, that things like the delay stuff are
not really related to the primary job FLTK is intended for.
Therefore one can consider it as a kind of bloat for all users that
don't need it. And not being bloated is *the* fundamental difference
of FLTK compared to other GUI toolkits!

But I agree with you, that a portable delay function would be an
example for code that should not be reinvented again and again in
every new project - because a portable implementation is nontrivial.
The lightweight UTF-8 normalization code that I have posted here
some time ago is another example of such stuff. Useful *if* you
need it, but simply bloat otherwise.

To preserve the "L" in FLTK, I think if we really want to add such
code, it should go into a separate auxiliary library similar to the
image stuff (that can be linked only if needed). What do the others
think about going this way in general?

The biggest drawback will be that changes to the build system(s) are
required.

[fabien]

I think Albrecht is right when he looks closely at the ABI. This is
important and all users of the shared library will suffer otherwise.

That remark would tend to revive a discussion about the dll approach itself, 

some very used framework today (i.e. juce) don't even use the lib model but collection of source file to include to your project.

This is extreme I would think but it is just to balance with that kind of argument, look back and try to add all the time spent with that shared lib implementation.

Now of course, some will argue that abi compatible impl. permit them to not rebuild the software that uses the shared lib and it is a gain, well i would challenge that and say that any  change in a depency should be retested or at least reviewed, so I would not mind the extra time to update the apps relying on it. It my humble opinion and does not need to be shared but this cause among other things , the toolkit to be very sluggish to evolve.

Finally, nobody says that it should be released in 1.3.3 branch, and add-ons are really an abi problem as long as they are not used yet if we do it properly.

And I agree with Albrecht too, that things like the delay stuff are
not really related to the primary job FLTK is intended for.
Therefore one can consider it as a kind of bloat for all users that
don't need it. And not being bloated is *the* fundamental difference
of FLTK compared to other GUI toolkits!

Again, look around the other frameworks and toolkits, they all seem to disagree ; light or not.

If we followed that principle, then we should not have any threading api, and what we use internally should be kept internal in terms of threading needs, but is it a viable approach nowadays where parallelism is common in most modern developments ?

But I agree with you, that a portable delay function would be an
example for code that should not be reinvented again and again in
every new project - because a portable implementation is nontrivial.
The lightweight UTF-8 normalization code that I have posted here
some time ago is another example of such stuff. Useful *if* you
need it, but simply bloat otherwise.

To preserve the "L" in FLTK, I think if we really want to add such
code, it should go into a separate auxiliary library similar to the
image stuff (that can be linked only if needed). What do the others
think about going this way in general?

It would be more than time to elude all that archaic problems (like threads, sems, sleep, wait api) and use a modern fltk impl c++11.

Of course this would be a different project, but it's roadmap shouyd be very humble at the contrary of 2.x and 3.x to be achievable in a reasonable time:

just keeping the framework functionally equivalent but then remove all home-made obsolete code and replace it with (formerly boost::) C++11 api !

Now all of a sudden, the lib source code base would be much smaller, compatible and probably more robust too (i.e.: back in the days we were all developping opengl apps or other real-time sensitives apps where even allowing exceptions was not appealing due to the performance drawback. it was in the early 90's ... today I would love too be able to throw exceptions when the contract of interface or the preconditions in terms of parameters to supply are not met ...)

In our still very simple example discussed, we elude the parameter problem but what would happen if i call the sleep api with NaN, or if it were an integer with a negative number ? with no exception support it makes it more difficult to be robust.

Sill talking about testing, Matt Melcher was the one guy that implemented testing on FLTK (kudos to him about that !!) , again here imagine modern tdd development techniques applied to fltk.

I am surprised as an example that we don't talk about adding the nowadays common google test framework to our base code and start running UTs automatically on Jenkins as well as builds (Again Matt did the build server part back in the days it was a huge gain where is it now ?).

Now this would be a different discussion but we are already there as it seems there is nothing impressive is going to happen on fltk with its very heavy compatibility driven heritage IMO.

-Fab

[Ian MacArthur]

On Thu Dec 04 2014 16:52:19, 'fabien' via fltk.coredev wrote:
>
> That remark would tend to revive a discussion about the dll approach itself,

Hi Fabien,

Regardless of what you or I may think about using shared objects, there’s no doubt that right here, right now, that *is* the way the distort operate, so we do need to provide shared object support, and we do need to preserve the ABI.

Further, for folks using fltk in plugins (and I understand that there are a few audio plugins that use fltk) you really need to have a fltk DLL that works, or the plugin architectures generally fail...

> Again, look around the other frameworks and toolkits, they all seem to disagree ; light or not.

This is a bit of an odd thing...

I do a lot of multi-threaded code, and a lot of it is cross-platform. And I know quite a lot of engineers who are doing similar work. Pretty much all of them are using some sort of library or abstraction layer to provide portability of code over differing threading API’s, semas, mutexes, queues, whatever.

And in every case I can think of, the lib they are using already provides some sort of timer abstraction. I can’t image they’d even consider using the timer abstractions provided by the GUI lib rather than the timer abstractions in their portability layer...

So it just makes no sense to me to provide that capability in the GUI lib - as far as I can tell, the folks that need that capability are already getting it from another lib dedicated to abstracting those sorts of OS services anyway.

> If we followed that principle, then we should not have any threading api, and what we use internally should be kept internal in terms of threading needs,

Fltk does not have a threading API. What we have is an API that provides a basic mechanism to protect the graphical context (which is not thread safe) from being too badly perturbed in multi-threaded code.

Likening that to a threading API is something of a stretch...

There are libs that do that job well; they should be used to do that job.

Fltk’s job is to do the GUI, not abstract away the entire operating system portability issue.

[fabien]

> That remark would tend to revive a discussion about the dll approach itself, 

Hi Fabien, 

Regardless of what you or I may think about using shared objects, there’s no doubt that right here, right now, that *is* the way the distort operate, so we do need to provide shared object support, and we do need to preserve the ABI. 

Further, for folks using fltk in plugins (and I understand that there are a few audio plugins that use fltk) you really need to have a fltk DLL that works, or the plugin architectures generally fail... 

Hi Ian, 

WDL/iPlug and Juce that I both used are not relying on toolkit dll deployment and are vastly used in audio applications way more than fltk. So that kind of statement would sound irrelevant to me:

Please don't confuse the need to comply to a plugin architecture (relying on the use of shared libs at the plugin level) with the  need of  the gui framework you use into that plugin to be linked as a shared dll. More precisely, you talked about audio so let's cite VST, VST2, AU, AAX, RTAS common industry standard audio plugin formats for which none of them would _not_ work with the aforementioned gui /audio frameworks, all linked statically.

At the contrary, linking with shared dll's is often a source of problems with plugins: you usually like that your plugin is not suddenly going to stop to work because someone installed an incompatible plugin breaking dependent dll, whatever the root cause is....

Now the plugin themselves are shared dll of course but that is completely independent to the framework you used to generate it.

> Again, look around the other frameworks and toolkits, they all seem to disagree ; light or not. 

This is a bit of an odd thing... 

I do a lot of multi-threaded code, and a lot of it is cross-platform. And I know quite a lot of engineers who are doing similar work. Pretty much all of them are using some sort of library or abstraction layer to provide portability of code over differing threading API’s, semas, mutexes, queues, whatever. 

And in every case I can think of, the lib they are using already provides some sort of timer abstraction. I can’t image they’d even consider using the timer abstractions provided by the GUI lib rather than the timer abstractions in their portability layer... 

So it just makes no sense to me to provide that capability in the GUI lib - as far as I can tell, the folks that need that capability are already getting it from another lib dedicated to abstracting those sorts of OS services anyway. 

Again here this would be out of scope to me, as you seem to assume that all the use of such an api would only happen when using mt code which is totally independent from its use: some games in the test folder of our fltk use sleep without much or any multithreaded needs. You may just want sometimes to add a simple delay in an app without necessarily go deep or at all into the multithreaded code. Also, when using multithreaded code in light applications, one may not always need to use additional toolkits than fltk but just need simple timing changes and in that case it would also be appreciated to have this API in place.

> If we followed that principle, then we should not have any threading api, and what we use internally should be kept internal in terms of threading needs, 

Fltk does not have a threading API. What we have is an API that provides a basic mechanism to protect the graphical context (which is not thread safe) from being too badly perturbed in multi-threaded code. 

Likening that to a threading API is something of a stretch... 

There are libs that do that job well; they should be used to do that job. 

Fltk’s job is to do the GUI, not abstract away the entire operating system portability issue. 

Point taken about the limitations of the current threading API, still all of this could be available to all fltk users with no code addon nor any external by just using c++11 new threading api (that does include slep functionality of course) and much more of its new features.
See that's exactly where i was going in my latest message in that thread: if it's decided it's not fltk job to do so (even at a minimum portable level) ,  it could also be not necessary to add an extra toolkit to achieve this by stripping FLTK and letting C++11 deal with all this.

Now meanwhile, I'm thinking that it would be great if we could only use  c++11 extensions with fltk in near future, did anyone tried this recently ?

-Fab

[fabien]

Replying to myself: that indeed worked (at least with my ms visual studio 13 and Xcode5 testing ) I could wrote with no need to add any other framework:

this code in any demo app main() function:

std::forward_list<std::string> l;

l.push_front("Is The Answer");

l.push_front("42 ");

for_each(l.begin(), l.end(), [](std::string n) { 

std::cout << n << std::endl; 

std::this_thread::sleep_for(std::chrono::milliseconds(1000));

});

Gave the correct answer to life, universe and the rest.

Amazing job folks, fltk is more fun than ever to use with c++11 !

-Fab

[Sanel Zukan]

"'fabien' via fltk.coredev" <fltkc...@googlegroups.com> writes:
> Now meanwhile, I'm thinking that it would be great if we could only use
> c++11 extensions with fltk in near future, did anyone tried this recently ?

And this would render FLTK unusable for old platforms and compilers,
which is the main reason why FLTK is so compelling for some users. That
story was told here numerous times.

Not to say that some of those users (like me), find c++11 and everything
after that, abomination.

> -Fab

Best,
Sanel

[Albrecht Schlosser]

On 04.12.2014 16:46 Michael Bäuerle wrote:

> To preserve the "L" in FLTK, I think if we really want to add such
> code, it should go into a separate auxiliary library similar to the
> image stuff (that can be linked only if needed). What do the others
> think about going this way in general?

That's certainly an option.

However we should only add code that is really useful for many users and
well supported. Platform dependent code like the sleep functions should
be in separate specialized OS abstraction libraries and not in our FLTK
library.

As said before: if we did this, it should really be bullet-proof, but
who can guarantee this for all the ancient platforms we support? Once we
add it, we take the "responsibility" that it works as expected. If the
FLTK core doesn't need it, why should we do this?

> The biggest drawback will be that changes to the build system(s) are
> required.

True.

[fabien]

On Friday, December 5, 2014 4:00:26 AM UTC-6, sanelz wrote:

"'fabien' via fltk.coredev" <fltkc...@googlegroups.com> writes:

../.. 

> Now meanwhile, I'm thinking that it would be great if we could only use
>  c++11 extensions with fltk in near future, did anyone tried this recently ?

And this would render FLTK unusable for old platforms and compilers,

You did not  get the point, here we were discussing about _using_ c++11 on the top of fltk 1.3.3 and be able to build with it for those who want it, then on another thread I confirmed that this is still the case (at least on win/mac osx recent toolchains) so the question is closed.

which is the main reason why FLTK is so compelling for some users. That
story was told here numerous times.

Not to say that some of those users (like me), find c++11 and everything
after that, abomination.

'Everything' after c+_11 ? it is the latest stable standard for C++ that I know of being able, could you be more specific ?

C++11 is almost a different language, much more pleasant to develop with, simplifying even stl use (again read my previous emails about this).

It contains a lot of concept that were initially developed in the famous boost library.

It addresses a lot of modern development paradigms that c++ implementation did not address initially and that are present in dot c sharp language for a while already as an example.

-Fab

[Sanel Zukan]

"'fabien' via fltk.coredev" <fltkc...@googlegroups.com> writes:
> On Friday, December 5, 2014 4:00:26 AM UTC-6, sanelz wrote:
>>

>> "'fabien' via fltk.coredev" <fltkc...@googlegroups.com <javascript:>>

>> writes:
>>
> ../..
>
>> > Now meanwhile, I'm thinking that it would be great if we could only use
>> > c++11 extensions with fltk in near future, did anyone tried this
>> recently ?
>>
>> And this would render FLTK unusable for old platforms and compilers,
>>
> You did not get the point, here we were discussing about _using_ c++11 on
> the top of fltk 1.3.3 and be able to build with it for those who want it,
> then on another thread I confirmed that this is still the case (at least on
> win/mac osx recent toolchains) so the question is closed.

This reminds me on the famous talk for adding optional GC in C++:
imagine they add it and could be turned off. How to use those libraries
that are using it?

The same can be applies here: imagine there are C++11 features on top of
FLTK that can be turned off. Then someone adds another layer of widgets,
sleep() or whatever functions on top of that. Then suddenly, those C++11
features aren't optional any more.

> which is the main reason why FLTK is so compelling for some users. That
>> story was told here numerous times.
>>
>> Not to say that some of those users (like me), find c++11 and everything
>> after that, abomination.
>>
>> 'Everything' after c+_11 ? it is the latest stable standard for C++ that I
> know of being able, could you be more specific ?

There is a stable revision named C++14 and gcc/clang are already
supporting some of the requirements. And also, there is C++17 with some
known features in preparation.

> C++11 is almost a different language, much more pleasant to develop with,
> simplifying even stl use (again read my previous emails about this).

I think this is subjective matter.

> It contains a lot of concept that were initially developed in the famous
> boost library.
> It addresses a lot of modern development paradigms that c++ implementation
> did not address initially and that are present in dot c sharp language for
> a while already as an example.

Most of those concepts aren't initially developed in Boost, but are
collected from either previous projects (e.g. boost::regex or
Boost.Preprocessor) or are copied from other languages with C++ specific
sugar on top of it.

I think we can talk about this topic all night long, without making a
single step forward or backward. The main questions are: will it make
FLTK faster, ligter or more portable than currently is?

> -Fab

Best,
Sanel

[fabien]

../..

>> And this would render FLTK unusable for old platforms and compilers, 

>>
> You did not  get the point, here we were discussing about _using_ c++11 on
> the top of fltk 1.3.3 and be able to build with it for those who want it,
> then on another thread I confirmed that this is still the case (at least on
> win/mac osx recent toolchains) so the question is closed.

../..

The same can be applies here: imagine there are C++11 features on top of
FLTK that can be turned off. Then someone adds another layer of widgets,
sleep() or whatever functions on top of that. Then suddenly, those C++11
features aren't optional any more.

Again, we were already at this stage of the thread into another other approach question of being able to use c+11 outside fltk,

as the vast majority voted that we should not deal with other matters than our gui toolkit to summarize, so no add-ons, no c++11 add-ons (for 1.3.x).

This point of the discussion was just to confirm users can still use whatever language features they see fit outside the toolkit.

> which is the main reason why FLTK is so compelling for some users. That
>> story was told here numerous times.
>>
>> Not to say that some of those users (like me), find c++11 and everything
>> after that, abomination.
>>
>> 'Everything' after c+_11 ? it is the latest stable standard for C++ that I
> know of being able, could you be more specific ?

There is a stable revision named C++14 and gcc/clang are already
supporting some of the requirements. And also, there is C++17 with some
known features in preparation.

What I meant is the standard was only out in august 2014 I see no visual studio platform or llvm platform released on Xcode that handle it yet.
IMO this is far to be implemented enough for being useable on more than one experimental compiler platform, see http://blogs.msdn.com/b/vcblog/archive/2014/08/21/c-11-14-features-in-visual-studio-14-ctp3.aspx for more detailed information on visual studio support.

Probably,  we may start to be able use it in a more multiplatform way only starting from  vs2014 final version on the msvc side at least.

C++11 is the most common extension available on stable releases of our current compilers and platform (that includes windows and visual studio)

../..

> simplifying even stl use (again read my previous emails about this).

I think this is subjective matter.

Can you be more specific here, is it subjective to have auto variables types to simplify the iterator use, or the new begin() end() methods available on standard arrays too now, or new for iteration that greatly simply the use of stl ADT's ?
This is very concrete and real to me.
 

> It contains a lot of concept that were initially developed in the famous
> boost library.
> It addresses a lot of modern development paradigms that c++ implementation
> did not address initially and that are present in dot c sharp language for
> a while already as an example.

Most of those concepts aren't initially developed in Boost, but are
collected from either previous projects (e.g. boost::regex or
Boost.Preprocessor) or are copied from other languages with C++ specific
sugar on top of it.

from the top of my mind: forward_list iterator, for_each, thread (including sleep())  as well as many other constructions were similarly implemented and already available in boost and much more: see http://stackoverflow.com/questions/8851670/relevant-boost-features-vs-c11 as an example for details.

To be clear, I don't really care about who did it first, but care much more about the fact that I can now consider not using boost where i would have used it before ...

I think we can talk about this topic all night long, without making a
single step forward or backward. The main questions are: will it make
FLTK faster, ligter or more portable than currently is?

If we decide to use modern c++ extensions in future, i think that according to the previous discussion we had  on that same thread:
 - it wouldn't be happening for sure on the 1.3.x branch
 - it will either break definitely the compatibility with other old compilers that 1.3.x care about, or risk to make fltk even harder to maintain as it is already today (which would defeat the purpose of such a transition to me as the main goal I envisionned was to strip the current code and break with old compilers compatibility related limitations).

I only considered this idea of using modern extensions of the language for a potential new major version of the FLTK project, that would be this time specifically _only_ about the use of modern extensions and hence remove all resulting obsoleted code, so that it could be finalized and released in a reasonable time as opposed to be discontinued as 2.x and 3.x.
The motivation being to prepare new foundations that will make fltk evolve again at a faster pace, just my view.

-Fab