iostream formatting of special numbers

[Robert Wessel]

Does the standard say anything about the format of the output of
special numeric values, like NaNs and Infs, when formatted by
iostreams?

For example:

double c=1, d=0;
std::cout << c/d << std::endl;

produces: "1.#INF" with MSVC, although I don't think the format is
actually specified.

Does the output depend on std::uppercase? IOW, should it be "+Inf" or
"+INF" depending on std::(no)uppercase?

And then do I have to worry that someone will be expecting to parse a
possibly implementation dependent format?

I'm writing the iostream routines for some numeric routines.

Or have I just managed to miss the specification?

[Daniel]

On Friday, April 12, 2019 at 7:47:26 PM UTC-4, robert...@yahoo.com wrote:
> Does the standard say anything about the format of the output of
> special numeric values, like NaNs and Infs, when formatted by
> iostreams?

No.

>
> And then do I have to worry that someone will be expecting to parse a
> possibly implementation dependent format?
>

You can detect NaN and infinity and output your own strings, e.g.

#include <cmath> // std::isnan, std::isinf
#include <limits> // std::numeric_limits

if (std::isnan(x))
{
std::cout << "NaN";
}
else if (x == std::numeric_limits<double>::infinity())
{
std::cout << "+inf";
}
else if (std::isinf(x))
{
std::cout << "-inf";
}
else
{
std::cout << x;
}

So they can rely on that.

Daniel
https://github.com/danielaparker/jsoncons

[Öö Tiib]

On Saturday, 13 April 2019 02:47:26 UTC+3, robert...@yahoo.com wrote:
> Does the standard say anything about the format of the output of
> special numeric values, like NaNs and Infs, when formatted by
> iostreams?

No. Also it is possible for example that in conforming implementation
std::strtod and istream>> parse infs and nans differently.

> I'm writing the iostream routines for some numeric routines.

Basically ... avoid istream>> and ostream<<. These are slow and
behave non-portably.

> Or have I just managed to miss the specification?

No. If you need text transmission between different modules
then use some well-established serialization language like xml,
json or yaml. These have standardized specifications and
well-performing and -tested libraries for generating and
parsing can be found.

[James Kuyper]

On Friday, April 12, 2019 at 7:47:26 PM UTC-4, robert...@yahoo.com wrote:

> Does the standard say anything about the format of the output of
> special numeric values, like NaNs and Infs, when formatted by
> iostreams?

The behavior of to_string() for floating point arguments is defined in
terms of the C standard library function sprintf() with "%f" or "%Lf"
format specifiers (24.3.4p7). I'm fairly certain the specifications for
<iostream> cross-references to_string() somehow, but I'm in a bit of a
hurry, so I'll let someone else track down the precise specification.

You have to cross-reference the C standard to find out what that means
for those special values:
"A double argument representing an infinity is converted in one of the
styles [-]inf or [-]infinity — which style is implementation-defined. A
double argument representing a NaN is converted in one of the styles
[-]nan or [-]nan(n-char-sequence) — which style, and the meaning of
any n-char-sequence, is implementation-defined. The F conversion
specifier produces INF, INFINITY, or NAN instead of inf, infinity, or
nan, respectively." (C 7.21.6.1p8).

While the output is partially implementation-defined, there are parts
you can rely on. However, I see no way to determine from inside your
program where the implementation-defined n-char-sequence ends.

[Daniel]

On Monday, April 15, 2019 at 6:19:13 AM UTC-4, Öö Tiib wrote:
>
> Basically ... avoid istream>> and ostream<<.

> These are slow and behave non-portably.

Agreed, for the non-portably. There's also the issue of how many decimal
digits to preserve, to avoid losing precision. You could set that to
std::numeric_limits::max_digits10 (generally 17), but then you won't
have shortest representations for many numbers.

>
> > Or have I just managed to miss the specification?
>

> If you need text transmission between different modules
> then use some well-established serialization language like xml,
> json or yaml. These have standardized specifications and
> well-performing and -tested libraries for generating and
> parsing can be found.

Agreed, but ...

The json data format doesn't support nan/inf/-inf, it suggests that these be
reported as null. One or two json decoders/encoders support conversion
to strings "nan", "inf", "-inf", and back, but this is non standard and
non interoperable. The yaml data format does support "nan", "inf", "-inf",
but I think it's fair to say it's not nearly as well established as json or
xml.

Another alternative would be one of the JSON like binary format such as CBOR
that supports nan, inf, and -inf.

Daniel

[james...@alumni.caltech.edu]

On Monday, April 15, 2019 at 8:56:18 AM UTC-4, James Kuyper wrote:
> On Friday, April 12, 2019 at 7:47:26 PM UTC-4, robert...@yahoo.com wrote:
> > Does the standard say anything about the format of the output of
> > special numeric values, like NaNs and Infs, when formatted by
> > iostreams?
> The behavior of to_string() for floating point arguments is defined in
> terms of the C standard library function sprintf() with "%f" or "%Lf"
> format specifiers (24.3.4p7). I'm fairly certain the specifications for
> <iostream> cross-references to_string() somehow, but I'm in a bit of a
> hurry, so I'll let someone else track down the precise specification.

Correct citation, but not relevant. Sorry.

The relevant citation is for num_put::do_put(), whose behavior for
floating point types is defined in terms of passing "%f" to printf()
(26.4.2.2.2p2 and p5). ostream::operator<<() for floating point types is
defined in terms of calling do_put() (27.5.2.2p1). The rest of what I
wrote is still relevant:

[Robert Wessel]

On Mon, 15 Apr 2019 03:19:03 -0700 (PDT), 嘱 Tiib <oot...@hot.ee>
wrote:

You've got it backwards. I have a library/class with types that
should support the normal iostream functions. Thus I have to
implement them, and need to handle things like Infs and NaNs.

IOW, I'm trying to figure out what users of this library might be
expecting in those cases.

[Robert Wessel]

Thank you, that's very helpful.

To which version of the C++ standard are you referencing? All of the
relevant information ("punt to printf definition") in C++17 appears to
be in 25.2.2.2.2 (not 26.x), and I'm not sure what 27.5.2.2 is
supposed to be referring to.

It's probably not a great idea, but I'm toying with the idea of using
sprintf() to generate the local forms of those (on first use, and then
saving them).

[james...@alumni.caltech.edu]

On Monday, April 15, 2019 at 4:15:59 PM UTC-4, robert...@yahoo.com wrote:
> On Mon, 15 Apr 2019 03:19:03 -0700 (PDT), 嘱 Tiib <oot...@hot.ee>
> wrote:
>
> >On Saturday, 13 April 2019 02:47:26 UTC+3, robert...@yahoo.com wrote:
> >> Does the standard say anything about the format of the output of
> >> special numeric values, like NaNs and Infs, when formatted by
> >> iostreams?

...

> You've got it backwards. I have a library/class with types that
> should support the normal iostream functions. Thus I have to
> implement them, and need to handle things like Infs and NaNs.
>
> IOW, I'm trying to figure out what users of this library might be
> expecting in those cases.

The requirement that I mentioned which cross-references the C standard
only applies to float, double, and long double. If you're defining your
own type to work with those routines, it's up to you to decide what you
want, but it would probably be best to use the behavior for those types
as inspiration.

That being the case, you should understand the purpose of the
"[-]nan(n-char-sequence)" option. std::nan("n-char-sequence") is defined
as equivalent to std::strtod("NAN(n-char-sequence)"). The behavior of
std::istream::operator>>() is defined in terms of std::strtod(), so C++
users can also use that function to achieve the same result.

The behavior of std::strtod() for such strings is implementation-
defined. For many implementations, calling strtod() with a suitable n-
char-sequence will create a NaN which, when printed out, displays the
same n-char-sequence - but the C standard fails to mandate such a
connection.

For an implementation that pre#defines __STDC_IEC_559__ the C standard
does recommend for the <math.h> functions that "If a function with one
or more NaN arguments returns a NaN result, the result should be the
same as one of the NaN arguments (after possible type conversion),
except perhaps for the sign." (C F10p13). On many implementations, the
same is true of ordinary floating point math operations. The net result
is that if a NaN in the inputs of a calculation is responsible for the
fact that the result of the calculation is also a NaN, then the n-char-
sequence can be used to determine which input NaN was responsible for
that result.

You get to choose whether you want to implement your types to allow the
same kind of tracing - I'd recommend doing so.

[james...@alumni.caltech.edu]

On Monday, April 15, 2019 at 5:20:43 PM UTC-4, robert...@yahoo.com wrote:
> On Mon, 15 Apr 2019 12:20:39 -0700 (PDT),
> james...@alumni.caltech.edu wrote:
>
> >On Monday, April 15, 2019 at 8:56:18 AM UTC-4, James Kuyper wrote:

...

> >The relevant citation is for num_put::do_put(), whose behavior for
> >floating point types is defined in terms of passing "%f" to printf()
> >(26.4.2.2.2p2 and p5). ostream::operator<<() for floating point types is
> >defined in terms of calling do_put() (27.5.2.2p1). The rest of what I
> >wrote is still relevant:
> >
> >> You have to cross-reference the C standard to find out what that means
> >> for those special values:
> >> "A double argument representing an infinity is converted in one of the
> >> styles [-]inf or [-]infinity — which style is implementation-defined. A
> >> double argument representing a NaN is converted in one of the styles
> >> [-]nan or [-]nan(n-char-sequence) — which style, and the meaning of
> >> any n-char-sequence, is implementation-defined. The F conversion
> >> specifier produces INF, INFINITY, or NAN instead of inf, infinity, or
> >> nan, respectively." (C 7.21.6.1p8).
> >>
> >> While the output is partially implementation-defined, there are parts
> >> you can rely on. However, I see no way to determine from inside your
> >> program where the implementation-defined n-char-sequence ends.
>
>
> Thank you, that's very helpful.
>
> To which version of the C++ standard are you referencing?

I'm referring to n4762.pdf, the latest draft of the C++ standard that I
have access to. It's dated 2018-07-07, making it more recent than C++17,
which means it may include things that are not yet official, but nothing
I mentioned in my message falls in that category.

> ... All of the

> relevant information ("punt to printf definition") in C++17 appears to
> be in 25.2.2.2.2 (not 26.x), and I'm not sure what 27.5.2.2 is
> supposed to be referring to.

My mistake. In n4762.pdf, that should have been 27.7.5.2.2p1. In the
draft version that most closely resembles the official C++2017 standard,
n4659.pdf, the corresponding section is 30.7.5.2.2p1.

> It's probably not a great idea, but I'm toying with the idea of using
> sprintf() to generate the local forms of those (on first use, and then
> saving them).

When your other message referred to "types that should support the
iostream requirements", I assumed that they were types you had to
implement from scratch. If using sprintf() is an option, that satisfies
the relevant requirements automatically, without you having to worry
about the details, which is a lot simpler. I'd recommend that approach.

[Robert Wessel]

On Mon, 15 Apr 2019 14:41:37 -0700 (PDT),
james...@alumni.caltech.edu wrote:

>On Monday, April 15, 2019 at 4:15:59 PM UTC-4, robert...@yahoo.com wrote:

>> On Mon, 15 Apr 2019 03:19:03 -0700 (PDT), ? Tiib <oot...@hot.ee>

>> wrote:
>>
>> >On Saturday, 13 April 2019 02:47:26 UTC+3, robert...@yahoo.com wrote:
>> >> Does the standard say anything about the format of the output of
>> >> special numeric values, like NaNs and Infs, when formatted by
>> >> iostreams?
>...
>> You've got it backwards. I have a library/class with types that
>> should support the normal iostream functions. Thus I have to
>> implement them, and need to handle things like Infs and NaNs.
>>
>> IOW, I'm trying to figure out what users of this library might be
>> expecting in those cases.
>
>The requirement that I mentioned which cross-references the C standard
>only applies to float, double, and long double. If you're defining your
>own type to work with those routines, it's up to you to decide what you
>want, but it would probably be best to use the behavior for those types
>as inspiration.

The types in question will interoperable with normal floats, but are
not actual floats in the sense of float/double/long double. But I am
trying to follow the normal C++ behavior with these. I *think* I have
Inf, NaN and zero handling under control in computational situations
(although I'm basically following the IEEE FP rules, signed zeros
excluded, not so much the C/C++ specs.). There's probably some open
issues with when SNaNs actually signal in some of the semi-numerical
functions.

>That being the case, you should understand the purpose of the
>"[-]nan(n-char-sequence)" option. std::nan("n-char-sequence") is defined
>as equivalent to std::strtod("NAN(n-char-sequence)"). The behavior of
>std::istream::operator>>() is defined in terms of std::strtod(), so C++
>users can also use that function to achieve the same result.
>
>The behavior of std::strtod() for such strings is implementation-
>defined. For many implementations, calling strtod() with a suitable n-
>char-sequence will create a NaN which, when printed out, displays the
>same n-char-sequence - but the C standard fails to mandate such a
>connection.

strtod(NAN()) may be suitable as-is, as are the INF() cases. An
implementation where those don't generate the same string as printf
and iostream is probably classifiable as a bit perverse. Probably
need to do some testing.

It would probably force a final commitment to requiring C++11...

>For an implementation that pre#defines __STDC_IEC_559__ the C standard
>does recommend for the <math.h> functions that "If a function with one
>or more NaN arguments returns a NaN result, the result should be the
>same as one of the NaN arguments (after possible type conversion),
>except perhaps for the sign." (C F10p13). On many implementations, the
>same is true of ordinary floating point math operations. The net result
>is that if a NaN in the inputs of a calculation is responsible for the
>fact that the result of the calculation is also a NaN, then the n-char-
>sequence can be used to determine which input NaN was responsible for
>that result.
>
>You get to choose whether you want to implement your types to allow the
>same kind of tracing - I'd recommend doing so.

We're not planning to support NaN payloads, although I'm willing to
reconsider. It would only be a modest change. While I can see the
theoretical benefit for being able to backtrack sources, I've never
seen anyone actually use the facility for such. Of course then I have
to pick a payload size, and figure out how inter-operate with floats.

There seems to be a bit of a definitional issue with the NaN-string
conversions, they appear to allow a NaN payload of zero, which for a
QNaN would acually make it an Inf. Not my issue, though.

[Robert Wessel]

On Mon, 15 Apr 2019 14:58:23 -0700 (PDT),

They are ("from scratch"). I was thinking of hacking out the local
representation by feeding a NaN or Inf float to sprintf.

[James Kuyper]

On 4/15/19 6:22 PM, Robert Wessel wrote:
...

> strtod(NAN()) may be suitable as-is, as are the INF() cases. An
> implementation where those don't generate the same string as printf
> and iostream is probably classifiable as a bit perverse. Probably
> need to do some testing.

The way you said that isn't quite right, but it's possible that what you
meant to say is correct.

std::strtod(), std::nan(), and std::istream::operator>>() should all
produce the same NaN from a given n-char-sequence, since the behavior of
the last two is defined in terms of the behavior of strtod(). An
implementation where that was not true would be non-conforming.

std::sprintf(), std::num_put::do_put(), and std::ostream::operator<<()
should produce the same n-char-sequence (if any) from any given NaN,
since each function is that series has behavior defined in terms of the
preceding one. An implementation where that was not true would be
non-conforming.

Nothing mandates that sprintf() must reverse the conversion performed by
strtod() - the code

std::sprintf(buffer, "%f",
std::strtod("NAN(n-char-sequence)")

doesn't necessarily fill buffer with "NAN(n-char-sequence)". However, I
would agree that an implementation which chooses the option of producing
an n-char-sequence, and which doesn't choose to produce the same
sequence as was passed to strtod(), could be classified as "a bit perverse".

Do those statements match what you intended to say?

[Öö Tiib]

Do I understand it correctly that you need to implement both
istream>> and ostream<< for your own type that does not
need to use the mess around I/O of fundamental floating
types but you want these to look very similar to fundamental
floating point types? Then it is more about expectations of
potential user base (who are rarely language lawyers)?

I would (as user of such type) expect << to output nan, inf
and -inf with uppercase flag of stream not enabled (default
and std::nouppercase) and NAN, INF and -INF with it
enabled (std::uppercase). Also I would expect std::showpos
should cause positive infinity to be output as +inf or +INF.
The >> I would expect to parse all those forms regardless
of flags.

I would dislike indication of NaN payload. There are
multitude of reasons: It is unused by program logic
on vast majority of cases. It increases size of text that
is visually garbage but needs producing and handling
code. IOW waste of both storage and performance
for no gain on vast majority of cases.

[Robert Wessel]

Yes, badly worded on my part (at best).

[Robert Wessel]

On Tue, 16 Apr 2019 05:26:52 -0700 (PDT), 嘱 Tiib <oot...@hot.ee>

wrote:

>On Monday, 15 April 2019 23:15:59 UTC+3, robert...@yahoo.com wrote:

>> On Mon, 15 Apr 2019 03:19:03 -0700 (PDT), ? Tiib <oot...@hot.ee>

Yes.

> (who are rarely language lawyers)?

I'm not expecting them to be language lawyers, but I do think it
worthwhile to be as consistent with the rest of the (C++)
implementation as reasonable. If a calculation results in an infinity
with normal FP types, or with this library, it would be least
surprising to users if they both produced the same string after
formatting. It may not be a big deal for many users, but why
introduce unnecessary differences for no good reason?

>I would (as user of such type) expect << to output nan, inf
>and -inf with uppercase flag of stream not enabled (default
>and std::nouppercase) and NAN, INF and -INF with it
>enabled (std::uppercase). Also I would expect std::showpos
>should cause positive infinity to be output as +inf or +INF.
>The >> I would expect to parse all those forms regardless
>of flags.

As James pointed out, the standard allows "infinity" as well as "inf".
It also appears that there are more than a few non-standard
interpretations floating around. My current thinking is to dig the
local output formatting out of sprintf. That would appear to be
fairly simple for the various forms of infinity, there's a bit more
complexity with NaNs. I'm less clear on what I want to do for
conversions from *text* format.

>I would dislike indication of NaN payload. There are
>multitude of reasons: It is unused by program logic
>on vast majority of cases. It increases size of text that
>is visually garbage but needs producing and handling
>code. IOW waste of both storage and performance
>for no gain on vast majority of cases.

I'm waffling on this. I don't think NaN payloads have much storage or
performance impact (the increased code footprint for producing them
notwithstanding) for most applications. They, after all, would only
happen if there are actual NaNs. I don't think it's unreasonable to
support formatting NaN payloads if the implementation does so for
normal floats.

This may ultimately need some configuration options, but I'd like to
be able to provide good defaults for most cases.

[Öö Tiib]

On Tuesday, 16 April 2019 20:19:18 UTC+3, robert...@yahoo.com wrote:

> On Tue, 16 Apr 2019 05:26:52 -0700 (PDT), Öö Tiib <oot...@hot.ee>
> wrote:
>
>
> > (who are rarely language lawyers)?
>
>
> I'm not expecting them to be language lawyers, but I do think it
> worthwhile to be as consistent with the rest of the (C++)
> implementation as reasonable. If a calculation results in an infinity
> with normal FP types, or with this library, it would be least
> surprising to users if they both produced the same string after
> formatting. It may not be a big deal for many users, but why
> introduce unnecessary differences for no good reason?

Reasoning depends on who are your customers. Some bigger
players in that "standardization" game see benefits in
unreasonable incompatibilities between implementations.
Rest of participants don't want things to be as close to
whatever is "implementation defined" as possible; they
usually want things to be platform agnostic if possible.

>
> >I would (as user of such type) expect << to output nan, inf
> >and -inf with uppercase flag of stream not enabled (default
> >and std::nouppercase) and NAN, INF and -INF with it
> >enabled (std::uppercase). Also I would expect std::showpos
> >should cause positive infinity to be output as +inf or +INF.
> >The >> I would expect to parse all those forms regardless
> >of flags.
>
> As James pointed out, the standard allows "infinity" as well as "inf".
> It also appears that there are more than a few non-standard
> interpretations floating around. My current thinking is to dig the
> local output formatting out of sprintf. That would appear to be
> fairly simple for the various forms of infinity, there's a bit more
> complexity with NaNs. I'm less clear on what I want to do for
> conversions from *text* format.

It would be fine for me if >> did parse "infinity" as well (and then
both "inf" and "infinity" on all platforms) but I would dislike
if << would output "infinity" on some platforms and "inf" on
others. If that can be turned off with some compile time switch
then I would be happy with it.

> >I would dislike indication of NaN payload. There are
> >multitude of reasons: It is unused by program logic
> >on vast majority of cases. It increases size of text that
> >is visually garbage but needs producing and handling
> >code. IOW waste of both storage and performance
> >for no gain on vast majority of cases.
>
>
> I'm waffling on this. I don't think NaN payloads have much storage or
> performance impact (the increased code footprint for producing them
> notwithstanding) for most applications. They, after all, would only
> happen if there are actual NaNs. I don't think it's unreasonable to
> support formatting NaN payloads if the implementation does so for
> normal floats.

The goals and preferences of target audiences can be
diametrically opposed there, but my impression is that
majority likes portability.
I have had experience that the programs that use NaNs as valid
and meaningful values (in meanings of "not measured",
"unknown" and/or "marked as noise") can have majority of data
as NaNs.

> This may ultimately need some configuration options, but I'd like to
> be able to provide good defaults for most cases.

If such NaN payload support can be turnable off compile time then
I would be happy with it.