Fl_Preferences instead of boost::interprocess

[anmol....@gmail.com]

I need to have a single mutex across all instances of a shared lib. Sadly, a global is duplicated if you use a static lib inside a shared lib.

So, the mutex must be process-wide.

Clearly one can do this using boost::interprocess - but we also have the Fl_Preferences - 

 

dataP = Fl_Preferences(nullptr, "Runtime");

Can we use a pointer here as a process wide mutex?

std::mutex *m;

if (dataP.entryExists{"mutex"))

    dataP.get("mutex", (double *) &m,  0.0);

else {

    std::mutex *m = new std::mutex;

    dataP.set("mutex", m);

}

m.lock();

//

// critical section

//

m.unlock();

 Later on, I can simply wait on this mutex for a global access to a shared resource.

I used double as there is no option for long long - so this seems to be the only option to store 64-bit pointer.

[Philip Rose]

From: anmol....@gmail.com
Sent: 11 November 2021 15:01
To: fltk.general
Subject: [fltk.general] Fl_Preferences instead of boost::interprocess

--

 

There’s a version of Fl_Preferences::get() that uses void* to get/set any length item. This is an example that gets and sets an object of type time_t.

 

     // Check this is to be a new session (> 1hour after previous session ended

     time_t today = time(nullptr);

     void* p_last = new time_t;

     settings_->get("Session End", p_last, &today, sizeof(time_t));

     time_t last_session_end = *(time_t*)p_last;

     char action[100];

     if (difftime(today, last_session_end) > 3600.0) {

           // It is > 60 minutes since we last saved a record - new session

           session_start_ = today;

           strcpy(action, "Starting new session");

           settings_->set("Session Start", &today, sizeof(time_t));

           resuming_ = false;

     }

     else {

           // Restore previous session's start time

           settings_->get("Session Start", p_last, &today, sizeof(time_t));

           session_start_ = *(time_t*)p_last;

           strcpy(action, "Resuming session");

           resuming_ = true;

     }
.

Regards Phil.

[anmol....@gmail.com]

Is this thread safe? I looked in the code, and I'm not sure of concurrency.

On Thursday, November 11, 2021 at 9:49:18 PM UTC+5:30 pvr...@btinternet.com wrote:

There’s a version of Fl_Preferences::get() that uses void* to get/set any length item. This is an example that gets and sets an object of type time_t.

 

Regards Phil.

[Philip Rose]

From: anmol....@gmail.com
Sent: 15 November 2021 05:55
To: fltk.general
Subject: Re: [fltk.general] Fl_Preferences instead of boost::interprocess

--
You received this message because you are subscribed to the Google Groups "fltk.general" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkgeneral...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fltkgeneral/5cb4214c-f490-4785-aebc-5711952ffb61n%40googlegroups.com.

 

I do not yet understand how to use threads.

 

Phil.

 

[Ian MacArthur]

On Monday, 15 November 2021 at 05:55:05 UTC anmol.... wrote:

Is this thread safe? I looked in the code, and I'm not sure of concurrency.

I think you might have to explain what it is you are trying to do here, rather than asking about specific implementation aspects... I don't have a clear grasp of what you are trying to achieve  and "reading between the lines" of your questions and trying to "guess" what you want to do, I'm a little uncomfortable about the direction you are heading...

Now, probably I am just misunderstanding what you want to do, but just in case, can you elucidate the objective specifically, please? 

There may well be a better way!

[Matthias Melcher]

Fl_Preferences is not thread-safe. Neither for the file-base preferences nor for the internal preference database. It's just a file format reader and write for the super old Microsoft .ini files, representing them as a data tree. It would be more useful if it was updated to XML or JSON, but the API is more geared at unique identifiers. 

For thread safe access to any kind of data, you should read up on locks and semaphores. If you do want to use multithreading, there is no way around understanding those. C++17 and up offer all these calls in a very nice cross-platform way as part of the standard library.

[Philip Rose]

From: 'Matthias Melcher' via fltk.general
Sent: 15 November 2021 15:02
To: fltk.general
Subject: Re: [fltk.general] Fl_Preferences instead of boost::interprocess

 

Fl_Preferences is not thread-safe. Neither for the file-base preferences nor for the internal preference database. It's just a file format reader and write for the super old Microsoft .ini files, representing them as a data tree. It would be more useful if it was updated to XML or JSON, but the API is more geared at unique identifiers. 

 

For thread safe access to any kind of data, you should read up on locks and semaphores. If you do want to use multithreading, there is no way around understanding those. C++17 and up offer all these calls in a very nice cross-platform way as part of the standard library.

My app would benefit from multi-threading. Occasionally it goes off and performs some web activity that then holds up the main tasks. However it would be a major rework to introduce it now. I understand the concepts of locks and semaphores, but not how they are implemented in C++, either pre 17 or 17+.

 

Regards Phil.

[Ian MacArthur]

On Monday, 15 November 2021 at 15:02:08 UTC Matthias Melcher wrote:

For thread safe access to any kind of data, you should read up on locks and semaphores. If you do want to use multithreading, there is no way around understanding those. C++17 and up offer all these calls in a very nice cross-platform way as part of the standard library.

 As an addendum to what Matt said (Hi Matt!) and going somewhat off-topic for the thread, I'd add that whilst you need to gain an understanding of locks and semaphores as part of multithreading, it is also important to learn about mechanisms for safely accessing resources concurrently from multiple threads *without* using those locks.

The key point being that, if two or more concurrent threads find themselves in contention for a shared resource, then they can no longer run in parallel, and your multithreaded program can end up being serialized so that what you have, in effect, is just a single thread split up in a really complicated way...

So if you can contrive to *safely* access the resource without using a lock, then that will usually be better, even if it takes a bit more work.

And sometimes you have no safe option and the lock is the only way...

[Bill Spitzak]

If Fl_Preferences needs a mutex it must have it's own mutex. However I find it hard to believe that there is any need to read the Fl_Preferences database from more than one thread. Once it is read it should be easy to make sure two threads querying it can work at the same time.

BTW the whole idea of the toolkit providing a file format is not good so it would be nice to stay away from changing this any more.

--
You received this message because you are subscribed to the Google Groups "fltk.general" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkgeneral...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fltkgeneral/f1cc0e89-ca3a-4ea2-ac92-f81a2cf88a58n%40googlegroups.com.

[Bill Spitzak]

Oh I see what you are trying to do. As your code almost certainly tries to call this multithreaded and it actually tries to set a value, it is not thread safe. Fixing this would require another mutex and you would have exactly the same problem (or your problem maybe could be trivially solved by just using *that* mutex). I would not bother with using Fl_Preferences for anything, certainly not as some kind of message passing system.

You need to declare the mutex as part of your shared library. This may mean you need to pass the mutex as an argument to those functions you seem to be loading from other lower libraries (?). Actually I am rather unclear what your problem is. You may want to lock the mutex in the shared library code, rather than relying on some other code locking it.

To view this discussion on the web visit https://groups.google.com/d/msgid/fltkgeneral/61921ced.1c69fb81.9b10d.e779SMTPIN_ADDED_BROKEN%40gmr-mx.google.com.

[anmol....@gmail.com]

I am linking a static lib into a shared lib. The static lib is not threading-safe. So, I needed a global mutex saved into some kind of shared memory. 

This mutex will ensure only single thread access for the static lib.

If the shared lib is loaded, I will check for the global mutex. If its not there, I will allocate it and save it to Fl_Preferences.

Any successive access will use this mutex.

Using void * with Fl_Preferences is great as I can store a pointer. 

Since I am actually accessing a global mutex, not a shared variable, concurrency issues are controlled.

I cannot use a global variable because 

>>>

Variables that are declared as global in a DLL source code file are treated as global variables by the compiler and linker, but each process that loads a given DLL gets its own copy of that DLL's global variables. The scope of static variables is limited to the block in which the static variables are declared. As a result, each process has its own instance of the DLL global and static variables by default.

[anmol....@gmail.com]

If multiple shared libs are loaded by a single process, and each shared lib is linked with fltk as static lib, will they access the same Fl_Preferences internal prefs database ?

[Matthias Melcher]

Um (hi Ian, Bill ;-) , a few things:

1. threads have thread-local memory, but can still access the global space of the main thread. So if your main thread creates a Mutex, all children cann access that thread. Just pass a pointer to the Mutex in the appropriate call.

2. FLTK should always be running in the main thread. You can call FLTK in child threads, but some platforms can not handle window creation or deletion outside of the main thread!

3. Just in case you plan to link FLTK to s dynamically linked library: You can not link FLTK statically multiple times in the same app. If you link FLTK (or any other library) dynamically once, you can't link it again statically.

4. In a previous mail, I understood that you want to use the separate thread to download something across a network. If you want to notify FLTK in the main thread, that your library finished loading, you can use the sequence Fl::lock(); Fl::awake(handle_downloaded_data); Fl::unlock(); in your thread. This will call the handle_downloadPdata(void*) function in your main FLTK thread, which then can do UI stuff. Note: for FLTK locking to work, you must call Fl::lock(); once right before Fl::run();

FLTK 1.4.0: Multithreading support functions (awake())

FLTK 1.4.0: Multithreading support functions (lock())

[Ian MacArthur]

On Wednesday, 17 November 2021 at 15:15:38 UTC Matthias Melcher wrote:

Um (hi Ian, Bill ;-) , a few things:

1. threads have thread-local memory, but can still access the global space of the main thread. So if your main thread creates a Mutex, all children can access that thread. Just pass a pointer to the Mutex in the appropriate call.

Very true - indeed, unless you specifically ask for TLS (thread local storage) when you declare a variable in a global scope, the address space you get in every thread is basically the global address space of the main process, so "sharing" a mutex that way will typically Just Work.

In some ways, that's the key difference between threads and processes - threads have a shared memory context, processes do not.

 

2. FLTK should always be running in the main thread. You can call FLTK in child threads, but some platforms can not handle window creation or deletion outside of the main thread!

And by "some platforms" here we might well mean "most platforms", though things are changing... I always tend to assume "all platforms" at present as that makes the code reliably portable.

If you put the effort in, you can make GL contexts work "correctly" from the non-main threads but it can be tricky.

Other rendering paths... well, it's best to play it safe.

Particularly under WinXX, I have found it is very easy to create application code that *seems* to work Just Fine, and perhaps even does with some GPU drivers and etc., but then produces random corruptions elsewhere in the memory that are insanely hard to debug.

 

3. Just in case you plan to link FLTK to s dynamically linked library: You can not link FLTK statically multiple times in the same app. If you link FLTK (or any other library) dynamically once, you can't link it again statically.

Yes, I can't really see that working out -  This is under WinXX I think? Anyway, I think you'd need to build fltk as a DLL and use that, if there's a lot of other DLL's and they all want to interact with the fltk lib at runtime.

4. In a previous mail, I understood that you want to use the separate thread to download something across a network. If you want to notify FLTK in the main thread, that your library finished loading, you can use the sequence Fl::lock(); Fl::awake(handle_downloaded_data); Fl::unlock(); in your thread. This will call the handle_downloadPdata(void*) function in your main FLTK thread, which then can do UI stuff. Note: for FLTK locking to work, you must call Fl::lock(); once right before Fl::run();

FLTK 1.4.0: Multithreading support functions (awake())

FLTK 1.4.0: Multithreading support functions (lock())

And also: https://www.fltk.org/doc-1.4/advanced.html for more discussions on multi-threading with fltk.

[anmol....@gmail.com]

I used to think so. But in a specific case - global's in static libs are duplicated in a process address space if you link them into multiple DLL. The suggested solution of using fltk as a DLL is the best option. 

I ended up using a static mutex.

Also, in case someone views this thread, if you only wish to be notified of an event, saving a std::atomic using void* works well.

[anmol....@gmail.com]

Only under WinXX. This thread explains the issue - https://stackoverflow.com/questions/19373061/what-happens-to-global-and-static-variables-in-a-shared-library-when-it-is-dynam

[Philip Rose]

From: anmol....@gmail.com
Sent: 23 November 2021 03:32
To: fltk.general
Subject: Re: [fltk.general] Fl_Preferences instead of boost::interprocess

For what it’s worth, under Virtual Studio, I can only statically link any library into the final application. Although I access FLTK methods in my own library, they can only be linked when I bring all the .libs together in the application.

 

Phil.

[Ian MacArthur]

On Tuesday, 23 November 2021 at 03:32:34 UTC anmol... wrote:

Only under WinXX. This thread explains the issue - https://stackoverflow.com/questions/19373061/what-happens-to-global-and-static-variables-in-a-shared-library-when-it-is-dynam

Indeed - it is probably worth noting that the way dynamic libraries (or shared objects) are bound into the application runtime is platform specific, and the way that WinXX handles this is quite different from what Linux does - indeed the various "unices" aren't entirely consistent with each other either..

This has an effect on symbol visibility and the binding of common symbols generally, and there is no "simple answer" to this - if you are using SO/DLL libs, you need to understand what each platform you are targeting does.

[Ian MacArthur]

On Tuesday, 23 November 2021 at 05:49:25 UTC Phil wrote:

For what it’s worth, under Virtual Studio, I can only statically link any library into the final application. Although I access FLTK methods in my own library, they can only be linked when I bring all the .libs together in the application.

OK... I'm guessing that "Virtual Studio" is a typo. for "Visual Studio" here, so we are talking about Win32 dev...?

So, anyway, if you have built the fltk DLL's with VS, they should then Just Work, and dynamic linking should be fine - it's not my preferred approach, personally, but I have used it on Windows with both the VS and the mingw toolchains and it works OK. (Though you can not mix'n'match VS and mingw libs, they disagree with each other!)

That said, figuring out *how* to do DLL builds etc. with VS I always find challenging - I have never got along with VS, and I've been trying to use it, on and off, since it was called MSVC...

What are you trying, what works, what doesn't, etc....?

[Philip Rose]

From: Ian MacArthur
Sent: 23 November 2021 08:44

To: fltk.general
Subject: Re: [fltk.general] Fl_Preferences instead of boost::interprocess

 

On Tuesday, 23 November 2021 at 05:49:25 UTC Phil wrote:

It was a typo – too early in the morning after a poor night’s sleep.

 

I used VS to generate the FLTK static .libs. I used Cygwin to generate a .dll for another library I am using when I wanted the git master. I then used a VS utility to convert this to a static .lib. Normally I just use the static .lib in the released version. Just looked in my project work book but I can’t find the exact process. Other libraries, I am downloading static .lib files from various other projects.

 

Phil.

 

 

 

--
You received this message because you are subscribed to the Google Groups "fltk.general" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fltkgeneral...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fltkgeneral/ed311b12-f838-4775-a77d-77e26061d6ffn%40googlegroups.com.

 

[Matthias Melcher]

IIRC the easiest way to create dynamic libraries in VS is by using CMake to generate the VS Solution.

Also, linking to FLTK dynamically is (these days) only useful if an app must link to it multiple times, for example, if the app has plugins, and each of the plugins needs to link to FLTK. By linking dynamically in this case, the OS makes sure that all plugins link to the same *process* instance of FLTK. The main process must also link *dynamically* to FLTK it this case.

If linking statically to multiple libraries, which in turn need FLTK, you'd also link FLTK statically, so all references to FLTK can be resolved at runtime.

In all cases, global FLTK variables are visible in all dynamically linked plugins and/or static libraries, and they are all the exact same instance, and that works on all platforms. 

The above is true for every single *process*. *Threads* OTOH all live within a process and share global memory. Multiple threads inherit the global space of their parent thread. So, yes, multiple threads have access to all FLTK global variables, no matter if FLTK was statically or dynamically linked. FLTK defines a lock (fl_lock(), fl_unlock()) that can be used by all threads within an app to synchronise FLTK access.

[Ian MacArthur]

On Tuesday, 23 November 2021 at 12:27:15 UTC Phil wrote:

I used VS to generate the FLTK static .libs. I used Cygwin to generate a .dll for another library I am using when I wanted the git master. I then used a VS utility to convert this to a static .lib. Normally I just use the static .lib in the released version. Just looked in my project work book but I can’t find the exact process. Other libraries, I am downloading static .lib files from various other projects.

OK - just wanted to reiterate what I said earlier, that you often can't reliably intermix VS-built C++ libs (either static or DLL) with gcc-built C++ libs (either static or DLL.)

There are a number of things that are problematic - notably that MS have historically used their own name-mangling scheme for C++ files that is not compatible with what "everyone else" (well, gcc, clang, a range of others) do. [1]

They (MS) also handle some C++ constructs in specific ways that don't interoperate with gcc well. Or at all...

I'd assume your cygwin DLL with be gcc-based, so may be incompatible with your MS VC builds as a result (though IIRC patches do exist to allow gcc to generate MS VC style output, so cygwin may be using that?)

Anyway, with the MS dynamic linking model, it is best generally if you are "all in" on dynamic linking - trying to have some libs linked static and others as DLL just seems to be... problematic...

So if some lib is needed by more than one other DLL, then that lib is best linked as a DLL too, basically.

[1] - Note that "plain-C" libs can usually be mixed and matched just fine between compilers, since the C spec. defines how this should work. In their wisdom, the C++ committee decided to leave that up to the compiler implementers, so we now have multiple incompatible ways of doing the same things...