Doxygen oddity under Msys\mingw (possibly others!) - [General Use]

[MacArthur, Ian (Leonardo, UK)]

So I was regenerating the docs on a Msys/mingw build, and I noticed something odd.

TBH, I think it has been doing this for ages, but I usually do this from a script so hadn't really been watching...!

Anyway, when generating the docs for Fl_Preferences, in Fl_Preferences.cxx circa line 177 there's this construct:

The file name is generated in the form <tt>$(path)/$(application).prefs</tt>.

But when doxygen parses this on my build, it is literally expanding $(path) as the $PATH variable from my shell, and $(application) as empty.
The generated docs then have my PATH embedded in the HTML output, which is really not what we want!

I suspect that the construct should really be more like:

The file name is generated in the form <tt>path/application.prefs</tt>.

(Note that path and application are parameters to the constructor method at that point.)

But I don't really know doxygen well, so I may be talking nonsense here!

What do others see? Thoughts?
--
Ian

Leonardo MW Ltd
Registered Office: Sigma House, Christopher Martin Road, Basildon, Essex SS14 3EL
A company registered in England & Wales. Company no. 02426132
********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

[MacArthur, Ian (Leonardo, UK)]

>
> I suspect that the construct should really be more like:
>
> The file name is generated in the form <tt>path/application.prefs</tt>.
>

Hmm, or maybe we've lost some escape characters and it ought to have been:

The file name is generated in the form <tt>\$(path)/\$(application).prefs</tt>.

I do not know...

[MacArthur, Ian (Leonardo, UK)]

> > I suspect that the construct should really be more like:
> >
> > The file name is generated in the form
> <tt>path/application.prefs</tt>.
> >
>
> Hmm, or maybe we've lost some escape characters and it ought to have been:
>
> The file name is generated in the form
> <tt>\$(path)/\$(application).prefs</tt>.
>
> I do not know...

So it looks like it's not an Msys/mingw issue, it is generally "wrong" - indeed the docs online show a variation of this...

The 1.3.x files have this construct instead:

<tt><i>path</i>/<i>application</i>.prefs</tt>.


Which does produce sensible looking output, so the 1.3.x docs are fine, it just seems to be the 1.4 docs that are hooky...

[Albrecht Schlosser]

On 2/25/21 6:23 PM 'MacArthur, Ian (Leonardo, UK)' via fltk.coredev wrote:
>
>>> I suspect that the construct should really be more like:
>>>
>>> The file name is generated in the form
>> <tt>path/application.prefs</tt>.

No, ...

>> Hmm, or maybe we've lost some escape characters and it ought to have been:
>>
>> The file name is generated in the form
>> <tt>\$(path)/\$(application).prefs</tt>.

Yep.

Congrats! You found the only 2 of 23 occurrences that have not been
escaped correctly. I *think* I fixed these issues (commit 115a6025f,
Jan. 13, 2020) but I may have missed these two.

> So it looks like it's not an Msys/mingw issue, it is generally "wrong" - indeed the docs online show a variation of this...
>
> The 1.3.x files have this construct instead:
>
> <tt><i>path</i>/<i>application</i>.prefs</tt>.
>
> Which does produce sensible looking output, so the 1.3.x docs are fine, it just seems to be the 1.4 docs that are hooky...

Off the top of my head: This has been changed (by Matt) in 1.4, all
these docs have been rewritten but, as said above, not all instances
have been escaped correctly.

Thanks for finding this, it's hard to find!

Fixed in commit 5c1b69cd4fb (19:23:37 UTC +0100).

You should find the corrected docs shortly (usually within an hour or
less) on our GitLab mirror:
https://fltk.gitlab.io/fltk/classFl__Preferences.html#a7e2ec1496b7eb3cc5b2b9e8f6584c979

[Albrecht Schlosser]

On 2/25/21 7:29 PM Albrecht Schlosser wrote:
>
> Fixed in commit 5c1b69cd4fb (19:23:37 UTC +0100).
>
> You should find the corrected docs shortly (usually within an hour or
> less) on our GitLab mirror:
> https://fltk.gitlab.io/fltk/classFl__Preferences.html#a7e2ec1496b7eb3cc5b2b9e8f6584c979

Server has been updated, new docs are online now.

Build log: https://gitlab.com/fltk/fltk/-/jobs/1056715926
(just in case)

[Albrecht Schlosser]

On Am 2/25/21 7:33 PM schrieb Albrecht Schlosser wrote:
> On 2/25/21 7:29 PM Albrecht Schlosser wrote:
>>
>> Fixed in commit 5c1b69cd4fb (19:23:37 UTC +0100).
>>
>> You should find the corrected docs shortly (usually within an hour or
>> less) on our GitLab mirror:

>> https://fltk.gitlab.io/fltk/classFl__Preferences.html#a7e2ec1496b7eb3cc5b2b9e8f6584c979
>
>
> Server has been updated, new docs are online now.

Hmm, but looking at the docs, this seems to be strange:

Fl_Preferences::Fl_Preferences(
const char * path,
const char * vendor,
const char * application
)
...

"The file name is generated in the form $(path)/$(application).prefs." [...]

The argument 'vendor' is not mentioned in the docs (file name).

This looks wrong but I can't check this now. Maybe anybody else feels
inclined?

[MacArthur, Ian (Leonardo, UK)]

> >> Hmm, or maybe we've lost some escape characters and it ought to have
> been:
> >>
> >> The file name is generated in the form
> >> <tt>\$(path)/\$(application).prefs</tt>.
>
> Yep.
>
> Congrats! You found the only 2 of 23 occurrences that have not been
> escaped correctly. I *think* I fixed these issues (commit 115a6025f,
> Jan. 13, 2020) but I may have missed these two.

I don’t think there is a prize. I should have seen this earlier, if I ever checked my build logs...
When doxygen ran, it was printing a chunk or warnings to the console about this, and I hadn’t even noticed until yesterday.

[Manolo]

On Thursday, February 25, 2021 at 7:43:10 PM UTC+1 Albrecht Schlosser wrote:

Hmm, but looking at the docs, this seems to be strange:

Fl_Preferences::Fl_Preferences(
const char * path,
const char * vendor,
const char * application
)
...

"The file name is generated in the form $(path)/$(application).prefs." [...]

The argument 'vendor' is not mentioned in the docs (file name).

This looks wrong but I can't check this now.

That is strange indeed but not wrong, in the sense that it matches what is in the source code.

The path of the file reached by this Fl_Preferences constructor is defined

at line #938 of src/Fl_Preferences.cxx :

    snprintf(filename, sizeof(filename), "%s/%s.prefs", path, application);
This statement is in a member function which receives path, vendor and application

parameters, but the "vendor" parameter is not used to build the filename.

[Albrecht Schlosser]

On 2/26/21 9:29 AM 'MacArthur, Ian (Leonardo, UK)' via fltk.coredev wrote:
>>>> Hmm, or maybe we've lost some escape characters and it ought to have
>> been:
>>>>
>>>> The file name is generated in the form
>>>> <tt>\$(path)/\$(application).prefs</tt>.
>>
>> Yep.
>>
>> Congrats! You found the only 2 of 23 occurrences that have not been
>> escaped correctly. I *think* I fixed these issues (commit 115a6025f,
>> Jan. 13, 2020) but I may have missed these two.
>
> I don’t think there is a prize.

Indeed. I should have added a smiley. ;-)

> I should have seen this earlier, if I ever checked my build logs...
> When doxygen ran, it was printing a chunk or warnings to the console about this, and I hadn’t even noticed until yesterday.

That's interesting. I never saw warnings related to this, otherwise I'd
have fixed these two occurrences as well. I'm using

$ doxygen --version
1.8.17

Which doxygen version are you using?

BTW, my documentation builds (html + pdf) on Linux with doxygen 1.8.17
are completely free of warnings. Are there any other warnings in your
builds?

[Albrecht Schlosser]

[Opening a new thread with a new subject]

On 2/26/21 10:16 AM Manolo wrote in thread "Doxygen oddity under
Msys\mingw (possibly others!)"

Okay, thanks for your research.

This should be checked then:

- is it "correct" (*intended* to be) w/o vendor?
- backwards compatibility with 1.3 ?
- other constructors and paths ?

If it turns out to be wrong and someone has a quick fix, I'd be glad to
read about it here and/or see a commit.

Otherwise we could open a GitHub issue so this will not be forgotten.

Sorry, I'm too busy with other stuff to pursue this myself but didn't
want it to get lost in the noise...

[MacArthur, Ian (Leonardo, UK)]

> > I should have seen this earlier, if I ever checked my build logs...
> > When doxygen ran, it was printing a chunk or warnings to the console
> > about this, and I hadn’t even noticed until yesterday.
>
> That's interesting. I never saw warnings related to this, otherwise I'd
> have fixed these two occurrences as well.

Prior to the patch, I see this in my logs ("see" that is if I ever check them...)

$ make html
Generating HTML documentation...
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\MinGW'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\msys'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\local'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\MinGW'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\MinGW'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\msys'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\MinGW'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\msys'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\WINDOWS'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\system32'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\WINDOWS'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\WINDOWS'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\System32'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Wbem'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Tools'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\TortoiseSVN'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Tools'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\doxygen'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Tools'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Git'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\cmd'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Users'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\ian'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\AppData'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Local'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Microsoft'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\WindowsApps'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Tools'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\CMake'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\Tools'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\graphviz'
D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'


You can figure out what $PATH is set to in that shell from there!

> I'm using
>
> $ doxygen --version
> 1.8.17
>
> Which doxygen version are you using?

The same.

>
> BTW, my documentation builds (html + pdf) on Linux with doxygen 1.8.17
> are completely free of warnings. Are there any other warnings in your
> builds?

No other doxygen warnings, just those.
When I pull to latest the doxygen run is now silent...

[Albrecht Schlosser]

On 2/26/21 11:28 AM 'MacArthur, Ian (Leonardo, UK)' via fltk.coredev wrote:
>
> Prior to the patch, I see this in my logs ("see" that is if I ever check them...)
>
> $ make html
> Generating HTML documentation...
> D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\MinGW'
> D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\msys'

> ...

> D:/Support/fltk-1.4/src/Fl_Preferences.cxx:175: warning: Found unknown command '\bin'
>
> You can figure out what $PATH is set to in that shell from there!

Oh, well, then it's the *expanded* path that triggers the warnings, not
the statement in the sources. This is obviously Windows specific because
Windows uses '\' in the path whereas Linux uses '/' (of course) which
doesn't trigger the "unknown command" warning.

> No other doxygen warnings, just those.
> When I pull to latest the doxygen run is now silent...

Thanks for confirmation.

[MacArthur, Ian (Leonardo, UK)]

> Oh, well, then it's the *expanded* path that triggers the warnings, not
> the statement in the sources. This is obviously Windows specific because
> Windows uses '\' in the path whereas Linux uses '/' (of course) which
> doesn't trigger the "unknown command" warning.

Indeed.
The crux is that the doxygen comment had "$(path)" rather than the escaped form "\$(path)" and so the environment $PATH was being read in and expanded - which doxygen (unsurprisingly) did not like much.

The escaped form "\$(path)" works fine though.

[Matthias Melcher]

When I wrote Fl_Preferences in what feels like the 70's (it was probably
in the middle of 90's), there were many conflicting systems across
platforms.

Windows used RegEdit and .ini files. The FLTK format was supposed to be
compatible, but added "./" to section names, which was discovered too
late (it can however *read* .ini files). I decided to use this format
back then, because sections can be hierarchical, the format can be read
very quickly, and it can store all kinds of data including binary. I
wanted to use the exact same format on all platforms, so .ini files can
be exchanged.

Unix back then used the $(HOME)/.something text files or folders for
pretty much everything. I know that this changed massively over the
years, and various registries and app data storage systems were
introduced by various Linux distributions. To stay compatible, I decided
to not touch what I wrote in the olden days. Sure, XML or JSON support
would be nice, but the API is not really set up for that. I don't think
that integrating an XML reader would make much sense or ever be
complete. There are external libraries for that.

MacOS X was still young, but their much more sorted introduction of
application data directories seemed like a good approach, and in order
to get a unique filename for Fl_Preferences, I intended that using
"Fl_Preferences(USER/SYSTEM, vendor, application)" should look something
like this (based on how Apple generates paths):
".../appdata/org.fltk/fluid.preferences" where "org.fltk" would be the
vendor, and "fluid" would be the application. Well, I ended up spending
more time writing the code than writing the docs.

Bill was not happy about this addition IIRC. He wanted FLTK to remain a
pure GUI library, and in retrospect, he was probably right. OTOH,
FL_Preferences helped with typical UI stuff, sometimes as simple as
opening a window again in the same location where the user last closed
it, and that is more or less the intended scope.

This was all well and fine, but I wanted to be able to write a
preferences file just anywhere. So I added "Fl_Preferences(const char
*any_path)", but that sucked, because preferences files were supposed to
have app and vendor name in the first three lines of text. This lead to
the constructor that is mentioned below, where the vendor field is
purely used to write the second line of text in the file header, but it
is no longer part of the filename.

A last word about the API that is relatively uncommon, using
constructors to access sections of the file. Yes, I would do that
differently today. It is possible though, to open a section twice in the
same code and still create a correct .ini file, and users never need to
remember if they must deallocate anything. Much more problematic is any
lack of protection for multithreading or multiple app instances reading
and writing the same file at the same time.

Hoping my ramblings helped a little bit.

 - Matthias

[melcher....@googlemail.com]

 When I wrote Fl_Preferences in what feels like the 70's (it was probably in the middle of 90's), there were many conflicting systems across platforms.

Windows used RegEdit and .ini files. The FLTK format was supposed to be compatible, but added "./" to section names, which was discovered too late (it can however *read* .ini files). I decided to use this format back then, because sections can be hierarchical, the format can be read very quickly, and it can store all kinds of data including binary. I wanted to use the exact same format on all platforms, so .ini files can be exchanged.

Unix back then used the $(HOME)/.something text files or folders for pretty much everything. I know that this changed massively over the years, and various registries and app data storage systems were introduced by various Linux distributions. To stay compatible, I decided to not touch what I wrote in the olden days. Sure, XML or JSON support would be nice, but the API is not really set up for that. I don't think that integrating an XML reader would make much sense or ever be complete. There are external libraries for that.

MacOS X was still young, but their much more sorted introduction of application data directories seemed like a good approach, and in order to get a unique filename for Fl_Preferences, I intended that using "Fl_Preferences(USER/SYSTEM, vendor, application)" should look something like this (based on how Apple generates paths): ".../appdata/org.fltk/fluid.preferences" where "org.fltk" would be the vendor, and "fluid" would be the application. Well, I ended up spending more time writing the code than writing the docs.

Bill was not happy about this addition IIRC. He wanted FLTK to remain a pure GUI library, and in retrospect, he was probably right. OTOH, FL_Preferences helped with typical UI stuff, sometimes as simple as opening a window again in the same location where the user last closed it, and that is more or less the intended scope.

This was all well and fine, but I wanted to be able to write a preferences file just anywhere. So I added "Fl_Preferences(const char *any_path)", but that sucked, because preferences files were supposed to have app and vendor name in the first three lines of text. This lead to the constructor that is mentioned below, where the vendor field is purely used to write the second line of text in the file header, but it is no longer part of the filename.

A last word about the API that is relatively uncommon, using constructors to access sections of the file. Yes, I would do that differently today. It is possible though, to open a section twice in the same code and still create a correct .ini file, and users never need to remember if they must deallocate anything. Much more problematic is any lack of protection for multithreading or multiple app instances reading and writing the same file at the same time.

Hoping my ramblings helped a little bit.

 - Matthias

[Albrecht Schlosser]

On 2/26/21 12:01 PM 'Matthias Melcher' via fltk.coredev wrote:

> When I wrote Fl_Preferences in what feels like the 70's (it was probably
> in the middle of 90's), there were many conflicting systems across
> platforms.

Always glad to read that you're still around and watching FLTK development.

[...]

> ... This lead to

> the constructor that is mentioned below, where the vendor field is
> purely used to write the second line of text in the file header, but it
> is no longer part of the filename.

Okay, then that's the reason why it's not used in the filename and we
know that this was intended. Great, then we can finish this thread, the
question(s) have been answered.

Well, a small sentence about the usage of the vendor parameter in the
docs could be helpful though. We may want to add that...

> Hoping my ramblings helped a little bit.

Yep, thanks!

> On 2/26/2021 10:48 AM, Albrecht Schlosser wrote:
>> This should be checked then:
>>
>> - is it "correct" (*intended* to be) w/o vendor?
>> - backwards compatibility with 1.3 ?
>> - other constructors and paths ?
>>
>> If it turns out to be wrong and someone has a quick fix, I'd be glad
>> to read about it here and/or see a commit.
>>
>> Otherwise we could open a GitHub issue so this will not be forgotten.

So I conclude there's no need to open an issue.