string.format rejects uppercase %F

[Jonathan van Tuijl]

Hello,

I’ve found string.format doesn’t understand %F.

> string.format('%F', 42)
stdin:1: invalid conversion '%F' to 'format'
stack traceback:
[C]: in function 'string.format'
stdin:1: in main chunk
[C]: in ?

Affects a variety of Lua versions including 5.4.7-rc4. lstrlib.c from that version starting with line 1316:

        case 'a': case 'A':
          checkformat(L, form, L_FMTFLAGSF, 1);
          addlenmod(form, LUA_NUMBER_FRMLEN);
          nb = lua_number2strx(L, buff, maxitem, form,
                                  luaL_checknumber(L, arg));
          break;
        case 'f':
          maxitem = MAX_ITEMF;  /* extra space for '%f' */
          buff = luaL_prepbuffsize(&b, maxitem);
          /* FALLTHROUGH */
        case 'e': case 'E': case 'g': case 'G': {
          lua_Number n = luaL_checknumber(L, arg);
          checkformat(L, form, L_FMTFLAGSF, 1);
          addlenmod(form, LUA_NUMBER_FRMLEN);
          nb = l_sprintf(buff, maxitem, form, (LUAI_UACNUMBER)n);
          break;
        }

It seems %F wasn’t considered for whatever reason. It exists in ISO C sprintf which the documentation says is followed. %F differentiates itself from %f when formatting nan/inf. Should be an easy fix.

Jonathan

[Rolf Kalbermatter]

Op 6/14/2024 om 6:40 PM schreef Jonathan van Tuijl:

> Hello,
>
> I’ve found string.format doesn’t understand %F.
>
> > string.format('%F', 42)
> stdin:1: invalid conversion '%F' to 'format'
> stack traceback:
> [C]: in function 'string.format'
> stdin:1: in main chunk
> [C]: in ?

> It seems %F wasn’t considered for whatever reason. It exists in ISO C
> sprintf which the documentation says is followed. %F differentiates
> itself from %f when formatting nan/inf. Should be an easy fix.

Lua is still trying to be C89 compliant. The %F, %a, %A format specifier
are C99 extensions.

Same with length specifiers such as hh, ll, j, z, t.

Rolf Kalbermatter

[Michael Lenaghan]

On Jun 14, 2024 at 3:26:45 PM, Rolf Kalbermatter <rolf.kal...@kalbermatter.nl> wrote:

Lua is still trying to be C89 compliant.

I wonder how much value there is in being C89 compliant in 2024? Maybe Lua 5.5 could push the baseline forward?

[Jonathan van Tuijl]

Rolf Kalbermatter wrote:

Lua is still trying to be C89 compliant. The %F, %a, %A format specifier
are C99 extensions. 

I see. It does support %a and %A without even checking for C99, though, as evidenced by my snippet.

Same with length specifiers such as hh, ll, j, z, t.

 

These low-level passing details are irrelevant to Lua scripts anyway. Lua does use some internally, depending on the types it’s configured to use.

Jonathan

[Roberto Ierusalimschy]

That is a question for the community (through this list) to answer. As
far as I know, until 2019 Visual Studio did not support C99. (A few
months ago, someone complainded that their version of Visual Studio
could not convert unsigned long to double.) We have very little
information about the support for C99 in compilers for embedded systems.

-- Roberto

[bil til]

Am Sa., 15. Juni 2024 um 17:34 Uhr schrieb Roberto Ierusalimschy
<rob...@inf.puc-rio.br>:

> We have very little
> information about the support for C99 in compilers for embedded systems.

In embedded systems printf typically anyway would NOT be implemented
in any detail, just to save code space.

E. g. Keil ARMCC would use a so called "Microlib" Library for C, which
supports only quite basic printf functionality (though "Microlib" is
optional - but we always use this for controller applications).

(I am a bit surprised that somebody would insist on "%F", just to
enable writing NAN and INF in large letters... this does NOT really
sound very important to me? If you want a wider %f functionality, I
think it typically would e. g. make much more sense to implement
display of character shortcuts for powers of 1000, like kilo, Mega,
milli, micro... but for such "more elaborate" printf typically Lua
users can implement such a functionality quite easily in Lua by
themselves I think ...).

[Roberto Ierusalimschy]

> Am Sa., 15. Juni 2024 um 17:34 Uhr schrieb Roberto Ierusalimschy
> <rob...@inf.puc-rio.br>:
> > We have very little
> > information about the support for C99 in compilers for embedded systems.
>
> In embedded systems printf typically anyway would NOT be implemented
> in any detail, just to save code space.

> [...]

printf is the least of our problems :-) For C99 support, we are talking
about things like these:

- long long int
- preprocessor working with 64 bits
- header stdint.h
- inline directive
- mixing declarations with code
- Compound literals
- 'strtod' with support for hexadecimal (used by the core)

-- Roberto

[bil til]

Am Sa., 15. Juni 2024 um 20:23 Uhr schrieb Roberto Ierusalimschy

<rob...@inf.puc-rio.br>:
>
> > Am Sa., 15. Juni 2024 um 17:34 Uhr schrieb Roberto Ierusalimschy
>

> - long long int

Concerning long long int: Would Lua not support long long int anyway /
automatically for %d specifier, if Lua64bit used?

(and in Lua32bit it would use just 32bit for %d? - in our embedded
applications we would use Lua32, so then anyway %ll would make no
sense?)

(I do not really understand the problem you have with long long int
... your other points I anyway do not really understand, I would need
a short "problematic example" for all of these).

[Michael Lenaghan]

On Jun 15, 2024 at 11:34:38 AM, Roberto Ierusalimschy <rob...@inf.puc-rio.br> wrote:

I wonder how much value there is in being C89 compliant in 2024? Maybe Lua

5.5 could push the baseline forward?

That is a question for the community (through this list) to answer. As
far as I know, until 2019 Visual Studio did not support C99. (A few
months ago, someone complainded that their version of Visual Studio
could not convert unsigned long to double.) We have very little
information about the support for C99 in compilers for embedded systems.

The Wikipedia page on C99 has a summary of new features (https://en.wikipedia.org/wiki/C99#Design), as well as an overview of implementations (https://en.wikipedia.org/wiki/C99#Implementations). 

Here’s what the summary says about Visual Studio:

Visual C++ 2012 and earlier did not support C99.[29][30][31]

Visual C++ 2013 implements a limited subset of C99 required to compile popular open-source projects.[32][33]

Visual C++ 2015 implements the C99 standard library, with the exception of any library features that depend on compiler features not yet supported by the compiler (for example, <tgmath.h> is not implemented).[14]

Visual C++ 2019 (16.6) adds opt-in support for a C99 conformant preprocessor.[34]

That seems to be roughly correct; see Microsoft’s list of "C99 Standard library features” (https://learn.microsoft.com/en-us/cpp/overview/visual-cpp-language-conformance?view=msvc-160#c-standard-library-features-1). (C99 preprocessor support is listed under “C++03/11 Core language features.”)

Btw, the Visual C++ 2015 note calls out `<tgmath.h>` as being unimplemented. Microsoft’s list shows it as being implemented in Visual Studio 2019 16.8.

For what it's worth...

[Foster]

I'd like to stay C89 compliant since that's what my compilers like.

But I can suggest 2 paths forward.

- Do nothing  ( my favorite) 
- Produce an error message - " %F not supported (try %f or maybe use .upper?)

One of those hills that the view isn't worth the climb. 

[Jonathan van Tuijl]

bil til wrote:

(I am a bit surprised that somebody would insist on "%F", just to
enable writing NAN and INF in large letters... this does NOT really
sound very important to me? If you want a wider %f functionality, I
think it typically would e. g. make much more sense to implement
display of character shortcuts for powers of 1000, like kilo, Mega,
milli, micro... but for such "more elaborate" printf typically Lua
users can implement such a functionality quite easily in Lua by
themselves I think ...).

It isn’t insisting on anything in particular. It’s the expectation set by the standard. You should be able to just use that. C99 was 25 years ago. Of course you can work around it. You could throw out most of the libraries because the user can implement a replacement. That isn’t the point.

I had a look at Lua’s use of C99 stuff after this discussion. It does use some of it if it’s available, like snprintf and long long. It provides its own function for %a/%A when compiled for C89 (which outputs in lowercase and is touppered in post for %A, much as a %F workaround could do). It can clearly manage and accept compromise (like not supporting %a/%A modifiers just to provide something).

Jonathan

[Sean Conner]

It was thus said that the Great bil til once stated:

>
> (I do not really understand the problem you have with long long int
> ... your other points I anyway do not really understand, I would need
> a short "problematic example" for all of these).
>

As Roberto said, Microsoft didn't really support C99 for a *long* time,
and there are a ton of embedded C compilers that never got C99 support as
well, and Lua was designed for as wide usage as possible. And for the list
Robert presented of C99 features:

- long long int
- preprocessor working with 64 bits
- header stdint.h

These all guarantee 64-bit support. Prior to C99, no guarantee
at all.

- inline directive

Inline function calls, like C macros, but with better type checking and
less unwanted side effects.

- mixing declarations with code

This allows declaring variables as needed, instead of at the top of a
block scope. Also, you can declare variables inside the for() declaration:

for (int i = 0 ; i < max ; i++)
{
}

to really limit the scope of the index variable.

- Compound literals

For me, this is a "must have". Easiest with an example:

XSetWMProperties(
display,
window,
&(XTextProperty) {
.value = (unsigned char *)"Programmer's Calculator",
.encoding = XA_STRING,
.format = 8,
.nitems = 23,
},
&(XTextProperty) {
.value = (unsigned char *)"Programmer's Calculator",
.encoding = XA_STRING,
.format = 8,
.nitems = 23,
},
argv,
argc,
&(XSizeHints) {
.flags = PMinSize | PMaxSize,
.min_width = width,
.min_height = height,
.max_width = width,
.max_height = height,
},
&(XWMHints) {
.flags = StateHint | InputHint,
.initial_state = NormalState,
.input = True,
},
&(XClassHint) {
.res_name = (char *)"XPCalculator",
.res_class = (char *)"XPCalculator",
}
);

Otherwise, this code would have to be:

XTextProperty name;
XTextProperty icon;
XSizeHints sizehints;
XWMHints wmhints;
XClassHint classhints;

name.value = ...;
name.encoding = ...;
/* and so on, for all the above variables */

XSetWMProperties(display,window,&name,&icon,argv,argc,&sizehints,&wmhints,&classhints);

Remember, two of the hardest problems in computer science are cache
invalidation, naming things, and off-by-one errors. And by using compound
literals, I can avoid having to name things. This also shows another C99
feature---designated initializers.

- 'strtod' with support for hexadecimal (used by the core)

Nothing to add here.

These are all features of C99 that Lua doesn't use or has to work around
for C89.

-spc

[Martin Eden]

On 2024-06-14 18:40, Jonathan van Tuijl wrote:
> It seems %F wasn’t considered for whatever reason. It exists in ISO C
> sprintf which the documentation says is followed.

What I don't understand is why Lua wants to stick with dreadful
C format strings?

Lua 1.0 occurred in 1993, it is 31 years old. I believe it can
create it's own conscious format strings specifiers set.

Not like C that will extend this fancy facility until you ran out of
keys on your keyboard.

What I like about Lua:

  * first-class functions
  * tables
  * memory management

Frankly, I don't care in what language it is implemented. COBOL, LISP,
Visual Basic or C#. As long as I can have binary for system I'm using
(ESP8266, RPI, Linux Mint 21) and as long as I have that things,
I'm happy.

It is my personal opinion. Sure someone depends of it's C API and
praises embedding.

-- Martin

[Luiz Henrique de Figueiredo]

> What I don't understand is why Lua wants to stick with dreadful
> C format strings?

One reason is that it's very hard to get %f and %g right.
See https://stackoverflow.com/questions/3173056/why-does-dtoa-c-contain-so-much-code
dtoa.c contains 6248 lines of code; Lua 5.4.6 contains 30224 lines of code.

[bil til]

Am So., 16. Juni 2024 um 11:57 Uhr schrieb Luiz Henrique de Figueiredo
<l...@tecgraf.puc-rio.br>:

> > What I don't understand is why Lua wants to stick with dreadful
> > C format strings?
>
> One reason is that it's very hard to get %f and %g right.
> See https://stackoverflow.com/questions/3173056/why-does-dtoa-c-contain-so-much-code

As I see it, then main and VERY important feature of Lua print
function is, that it is "crash-proof" (no possibility to crash the
software by mismatch of the format string and the parameters to be
printed, as this is the general problem in C printf).

This is such a large advantage of Lua print, that the exact details of
"formatting singularities" really are of NOT much concern for me - Lua
itself is powerful enough that for a "more challenging" printout
formatting the Lua user can quite easily add own code in Lua (as print
typically anyway is quite a slow function, there are also no real
speed concerns... and for "more challenging" printout, a typical user
anyway might to add own formatting power...).

[Roberto Ierusalimschy]

> (I do not really understand the problem you have with long long int
> ... your other points I anyway do not really understand, I would need
> a short "problematic example" for all of these).

The pseudo-random number generator used by Lua is implemented with
64-bit integers. If the C compiler does not offer 64-bit integers,
the code has to emulate them using structures with two 32-bit integers.
With C99, we could be sure the compiler does support 64-bit integers
(long long), as then we could remove the code that implements our PRNG
with structures.

-- Roberto

[Lourival Vieira Neto]

At least for Lua in Kernel, I would go with C99; I think it would
clean up the code base a bit and I cannot relate to any specific
(free) compiler that doesn't support C99 properly. Recently we have
had some troubles with UBSAN [1], which doesn't like using a
fixed-length array as a flexible one; we needed to "%s/\[1\];/[];/g"
on lobject.h [2].

[1] https://docs.kernel.org/dev-tools/ubsan.html
[2] https://github.com/luainkernel/lua/commit/dfc14670fd27a46540ab0ba3c71cc55319b498d3

Regards,
--
Lourival Vieira Neto