On the building of fluid-cmd and fluid...

[imacarthur]

I was looking at the build output from recent builds and I note that for WIN32 targets the builds are compiling everything twice, once for fluid and then again for fluid-cmd.

The same is true for fltk-options and fltk-options-cmd now too, of course.

This seems a bit redundant, since the only actual difference is in the final link stage of the build. 

It seems like we could just compile everything once, then have two linker passes - though I don't have the necessary cmake-fu to know the ways of that. (The obj files currently generated are identical between the two branches, as best I can discern.)

And...

TBH, if we don't mind a hack, we could just compile and link everything once: the only actual difference between a "console" app and a "Windows" app is a single byte in the PE header block, so if we wanted (and I have done this in the past...) we can just compile for "Windows" and then hexedit the subsystem byte to create the "console" version.

Strictly speaking, the subsystem ID is a 16-bit short, but the values are either 0x002 for Windows or 0x003 for console, so only the 0x02 or 0x03 values ever change.

Where that byte is moves about with the PE header structure though, at least in principle, though in my experience (with gcc toolchains anyway) this is pretty much always byte 0xDC. (I do not think we can just assume it is byte 0xDC though!)

If we have the MS editbin tool, we can also just build for Windows and then do:

      editbin /subsystem:console wibble.exe

To convert the subsystem in a more controlled fashion than just a hex hack.

That said, the mingw toolchain does not have editbin, AFAIK - though the Msys shell doesn't need the fluid-cmd anyway since it does not drop the stdio stream for Windows apps.

Actually... that might be an option... Can we have the cmake script detect the presence of the editbin tool? 

I assume the MS toolchain, which "needs" the fluid-cmd built as a console app, does provide it, so:

Build fluid as a "Windows" subsystem app normally, fluid.exe

cp fluid.exe fluid-cmd.exe

If find editbin then editbin /subsystem:console fluid-cmd.exe

On mingw systems that do not have editbin, we just end up with a "fluid-cmd.exe" that is actually a Windows subsystem app, but that's OK because under mnigw / Msys that just works anyway...

[Albrecht Schlosser]

On 4/24/23 12:54 imacarthur wrote:

I was looking at the build output from recent builds and I note that for WIN32 targets the builds are compiling everything twice, once for fluid and then again for fluid-cmd.

The same is true for fltk-options and fltk-options-cmd now too, of course.

This seems a bit redundant, since the only actual difference is in the final link stage of the build.

Yes, indeed, I can confirm this.

It seems like we could just compile everything once, then have two linker passes - though I don't have the necessary cmake-fu to know the ways of that. (The obj files currently generated are identical between the two branches, as best I can discern.)

I can take a look into it. I'm not sure if we can avoid it in a "naive" CMake build but I know I did such things already by building an intermediate object library and linking against this object library (this is trivial). This lib would obviously not be installed in the final install step.

TBH, if we don't mind a hack, we could just compile and link everything once: the only actual difference between a "console" app and a "Windows" app is a single byte in the PE header block, so if we wanted (and I have done this in the past...) we can just compile for "Windows" and then hexedit the subsystem byte to create the "console" version.

I'm strictly against such hacks if they can be avoided, see my suggestion above.

Strictly speaking, the subsystem ID is a 16-bit short, but the values are either 0x002 for Windows or 0x003 for console, so only the 0x02 or 0x03 values ever change.

...

If we have the MS editbin tool, we can also just build for Windows and then do:

      editbin /subsystem:console wibble.exe

To convert the subsystem in a more controlled fashion than just a hex hack.

... Can we have the cmake script detect the presence of the editbin tool?

Even with such a tool and the required detection in CMake this would probably be more complicated than just linking twice from the object library.

But thanks for investigating and for the hint that such a tool exists. I didn't know that.

On mingw systems that do not have editbin, we just end up with a "fluid-cmd.exe" that is actually a Windows subsystem app, but that's OK because under mnigw / Msys that just works anyway...

When I added 'fluid-cmd' I wondered if we needed this for all Windows builds or only for MSVC but I decided to build it for all Windows builds, just in case (later this was copied by Matt for fltk-options). If we can reduce the build of the '*-cmd.exe' to just linking it's even less important to consider if we could avoid building these for MinGW etc..

[melcher....@googlemail.com]

I am confused. So under mingw, "fluid-cmd.exe" is a Windows subsystem app, but it still provides stdout and stderr output? I thought that the whole subsystem trouble is at run time and caused by the OS rather than the build system or shell?

Also, if we stop providing fluid-cmd on mingw machines, existing build setups may still rely on its existence, right? 

[imacarthur]

On Tuesday, 25 April 2023 at 15:00:30 UTC+1 Matt wrote:

I am confused. So under mingw, "fluid-cmd.exe" is a Windows subsystem app, but it still provides stdout and stderr output? I thought that the whole subsystem trouble is at run time and caused by the OS rather than the build system or shell?

Not quite: fluid-cmd.exe is built as a console app, under both MS and mingw at present.

My "objection" to that was only that we compile everything twice, even though the actual compiled .obj files are no different. 

The difference between "console" and "windows" apps is at the final link, when the PE header is constructed.

We could presumably compile everything just the once, then have two linker passes for the fluid.exe and fluid-cmd.exe.

The difference between a MS "console" app and a "windows" app, in effect, can be a little as a single bit in the PE header.

If the "console" bit is NOT set, the MS-DOS shell (and it's descendants) drops the stdio and detaches the app from the shell.

If the "console" bit is set, the stdio stays attached, and the shell waits for the process to finish.

If you run the app from the Msys shell, as many mingw users will, this does not happen; the Msys shell pretty much ignores the difference between "console" and "windows" apps, at least in this regard. It's more like bash in this regard.

Note that it does this whether the app was built with mingw or an other toolchain, e.g. MSVC.

The "fluid-cmd" console app was provided, as far as I know, so that MS-DOS shell users could still get stdio and so forth, but for Msys shell users it was kinda moot, since it "worked" anyway.

I'm not proposing to do away with fluid-cmd, just proposing we drop the redundant extra compilations to build it.

 

Also, if we stop providing fluid-cmd on mingw machines, existing build setups may still rely on its existence, right? 

Yeah, I think we'd still need to provide something called fluid-cmd - just in case. It is used in the fltk lib Makefiles now I think, under Windows.

It wouldn't affect my builds, I never use it, but I'm sure there are folks who do. 

I'd assume Greg uses it in his Windows builds, for example?

 

[Greg Ercolano]

On 4/25/23 08:24, imacarthur wrote:

I'd assume Greg uses it in his Windows builds, for example?

    Ya, I need to be able to generate executables via makefiles that detect
    failures from fluid via exit codes; Windows apps invoked from a Makefile
    immediately run 'in the background' and don't ever return exit codes
    indicating failures from code generation (-c flag).

    Also, on headless machines there's no way to see the record of failure
    in the error logs of non-console apps. For instance a permission problem
    reading the .fl file, or a write problem for the .cxx/.h files due to
    code directory permissions or network file access issues.

[Albrecht Schlosser]

On 4/24/23 15:17 Albrecht Schlosser wrote:
> On 4/24/23 12:54 imacarthur wrote:
>> I was looking at the build output from recent builds and I note that
>> for WIN32 targets the builds are compiling everything twice, once for
>> fluid and then again for fluid-cmd.
>>
>> The same is true for fltk-options and fltk-options-cmd now too, of
>> course.

fltk-options consists of only one source file, this shouldn't matter
(see below for fluid).

>> This seems a bit redundant, since the only actual difference is in
>> the final link stage of the build.
>
> Yes, indeed, I can confirm this.
>
>> It seems like we could just compile everything once, then have two
>> linker passes - though I don't have the necessary cmake-fu to know
>> the ways of that. (The obj files currently generated are identical
>> between the two branches, as best I can discern.)
>
> I can take a look into it. I'm not sure if we can avoid it in a
> "naive" CMake build but I know I did such things already by building
> an intermediate object library and linking against this object library
> (this is trivial). This lib would obviously not be installed in the
> final install step.

Fixed in commit 712fc72fef6e92851b69f9d6e448b3ea7cd10215.

This compiles only the main program source file for the executable and
links all other fluid objects from a new temporary object library (which
are compiled only once).

Compiling the main program (e.g. 'fluid.cxx') seems to be necessary
because we need at least one source file. This applies to all 'fluid*'
and 'fltk-options*' executables.

For fluid we can have up to three executables: fluid, fluid-cmd, and
fluid-shared. The latter is not really necessary but included for
testing if shared libs are built.

I believe this is a straight-forward solution and it works well (tested
on macOS, Linux, Linux to MinGW cross-build, mingw-32, and VS 2019).

Ian, can you please test and confirm that this is OK for you?
(MSYS2/MinGW-w64 not tested by me).

[imacarthur]

On Sunday, 7 May 2023 at 21:20:08 UTC+1 Albrecht Schlosser wrote:

I believe this is a straight-forward solution and it works well (tested
on macOS, Linux, Linux to MinGW cross-build, mingw-32, and VS 2019).

Ian, can you please test and confirm that this is OK for you?
(MSYS2/MinGW-w64 not tested by me).

Seems good.

Tested on MSYS2/mingw64, and also MSYS/mingw32. Both fine. 

Cleaned everything out first, just to make sure, everything went smoothly.

Also tested under WSL2, since I have it on this machine - it is also fine, of course, but that's not particularly relevant since WSL2 is Linux (Ubuntu, in this case) so the whole fluid vs. fluid-cmd thing business does not happen there.

Though, at a complete tangent, I notice that the fractals demo runs *really slowly* under WSL on this machine, which seems odd because the OpenGL support (at least under Wayland, anyway) seems otherwise very good, reporting OpenGL version 4.2

[Albrecht Schlosser]

On 5/8/23 11:59 imacarthur wrote:

On Sunday, 7 May 2023 at 21:20:08 UTC+1 Albrecht Schlosser wrote:

I believe this is a straight-forward solution and it works well (tested
on macOS, Linux, Linux to MinGW cross-build, mingw-32, and VS 2019).

Ian, can you please test and confirm that this is OK for you?
(MSYS2/MinGW-w64 not tested by me).

Seems good.

Tested on MSYS2/mingw64, and also MSYS/mingw32. Both fine. 

Cleaned everything out first, just to make sure, everything went smoothly.

Thanks very much for testing and feedback.

..., at a complete tangent, I notice that the fractals demo runs *really slowly* under WSL on this machine, which seems odd because the OpenGL support (at least under Wayland, anyway) seems otherwise very good, reporting OpenGL version 4.2

Sorry, no experience with WSL here.