Does stringification NVs should follow POSIX::setlocale?

[Vladimir Timofeev]

Since 5.16 perl follow current locale after POSIX::setlocale.
Also stringification of NVs.

But why?
It can lead to very subtle bugs :-(

Example:
vovkasm$ perl -MPOSIX -E '$a=0.5;
POSIX::setlocale(POSIX::LC_ALL,"ru_RU.UTF-8"); say "$a";
POSIX::setlocale(POSIX::LC_ALL,"C"); say "$a"'
0,5
0,5

Is this desired behaviour?

--
Vladimir Timofeev <vov...@gmail.com>

[Jan Dubois]

On Thu, Nov 8, 2012 at 4:16 AM, Vladimir Timofeev <vov...@gmail.com> wrote:
> Since 5.16 perl follow current locale after POSIX::setlocale.
> Also stringification of NVs.
>
> But why?
> It can lead to very subtle bugs :-(
>
> Example:
> vovkasm$ perl -MPOSIX -E '$a=0.5;
> POSIX::setlocale(POSIX::LC_ALL,"ru_RU.UTF-8"); say "$a";
> POSIX::setlocale(POSIX::LC_ALL,"C"); say "$a"'
> 0,5
> 0,5
>
> Is this desired behaviour?

I would agree that this is a bug. Especially since the locale is only
used for translating NV to PV, but not for PV to NV, so it breaks the
IV/NV/PV transparency:

$ perl -MPOSIX -E 'POSIX::setlocale(POSIX::LC_ALL,"de_DE.UTF-8");
$a=0.2; say "$a"+1; '
1
$ perl -MPOSIX -E 'POSIX::setlocale(POSIX::LC_ALL,"en_EN.UTF-8");
$a=0.2; say "$a"+1; '
1.2

Cheers,
-Jan

[Vladimir Timofeev]

Yes, and that is why I would like to know the original reasoning
behind these changes.
And if it is not vitally important, I would recommend to revert these changes.

>
> Cheers,
> -Jan



--
Vladimir Timofeev <vov...@gmail.com>

[Karl Williamson]

I started looking into this, and ran into a more fundamental question.
It says in perllocale.pod that locales are only paid attention to within
the scope of a 'use locale'. There is no such pragma in the examples
above, therefore it seems to me that the use of setlocale shouldn't
affect anything. The code in the core that I'm familiar with doesn't
look at locales unless under the scope of 'use locale'.

Am I missing something here?

[Jan Dubois]

The code that turns an NV into a PV uses the Gconvert() macro. It will
always use the current locale, regardless of any locale pragma being
in effect or not, AFAICT.

I guess this is the bug then. The problem of course is that saving
the current locale, setting the locale to "C", converting the number,
and then restoring the original locale is quite a bit of overhead for
the regular use case that isn't using locale.

So the simple call to Gconvert() in Perl_sv_2pv_flags() would then become:

#ifdef USE_LOCALE_NUMERIC
char *loc = savepv(setlocale(LC_NUMERIC, NULL));
setlocale(LC_NUMERIC, "C");
#endif
Gconvert(SvNVX(sv), NV_DIG, 0, s);
#ifdef USE_LOCALE_NUMERIC
setlocale(LC_NUMERIC, loc);
Safefree(loc);
#endif

That looks rather expensive to me.

Cheers,
-Jan

[Karl Williamson]

On 04/25/2013 07:07 PM, Jan Dubois wrote:
> The code that turns an NV into a PV uses the Gconvert() macro. It will
> always use the current locale, regardless of any locale pragma being
> in effect or not, AFAICT.
>
> I guess this is the bug then. The problem of course is that saving
> the current locale, setting the locale to "C", converting the number,
> and then restoring the original locale is quite a bit of overhead for
> the regular use case that isn't using locale.
>
> So the simple call to Gconvert() in Perl_sv_2pv_flags() would then become:
>
> #ifdef USE_LOCALE_NUMERIC
> char *loc = savepv(setlocale(LC_NUMERIC, NULL));
> setlocale(LC_NUMERIC, "C");
> #endif
> Gconvert(SvNVX(sv), NV_DIG, 0, s);
> #ifdef USE_LOCALE_NUMERIC
> setlocale(LC_NUMERIC, loc);
> Safefree(loc);
> #endif
>
> That looks rather expensive to me.
>
> Cheers,
> -Jan

Not being familiar at all with the code involved, nor how SVs operate,
and so this could be all wrong, I was thinking that some sort of
TIE-like magic could be applied

[Karl Williamson]

I now realize that I misunderstood your point. Something like what you
said above would be necessary to fix the bug that I raised. But it
wouldn't have to be done when executed in scope of 'use locale', also it
would have to be done only if the current locale is different from
C,POSIX. This could be kept track of; we already keep track of whether
or not the locale is a UTF8 one. Or if we didn't keep track, we could
exclude doing this unless setlocale had ever been called by the user.
This last would exclude all programs that don't use locale; the
performance hit would apply only to those few that do.

It seems to me, with a little more thought, again without doing much
investigation, that the original bug might be solvable simply by not
setting SvPVOK when doing a Gconvert in the scope of locale.

I was unfamiliar with gcvt(). The man page for it on my system doesn't
mention that it is locale dependent. It does say: "Marked as LEGACY in
POSIX.1-2001. POSIX.1-2008 removes the specification of gcvt(),
recommending the use of sprintf(3) instead (though snprintf(3) may be
preferable)."

And I wonder if there are any more of similar locale-dependent functions
that Perl calls?

[Leon Timmermans]

On Fri, Apr 26, 2013 at 4:51 PM, Karl Williamson
<pub...@khwilliamson.com> wrote:
> I now realize that I misunderstood your point. Something like what you said
> above would be necessary to fix the bug that I raised. But it wouldn't have
> to be done when executed in scope of 'use locale', also it would have to be
> done only if the current locale is different from C,POSIX. This could be
> kept track of; we already keep track of whether or not the locale is a UTF8
> one. Or if we didn't keep track, we could exclude doing this unless
> setlocale had ever been called by the user. This last would exclude all
> programs that don't use locale; the performance hit would apply only to
> those few that do.
>

> And I wonder if there are any more of similar locale-dependent functions
> that Perl calls?

strerror is a known issue. Worse yet, $! doesn't know what encoding
sterror returns, so it assumes it's latin-1 :-(

Leon

[Jan Dubois]

On Fri, Apr 26, 2013 at 7:51 AM, Karl Williamson
<pub...@khwilliamson.com> wrote:
> It seems to me, with a little more thought, again without doing much
> investigation, that the original bug might be solvable simply by not setting
> SvPVOK when doing a Gconvert in the scope of locale.

Well, it depends how far you want to go. Once you accept that locale
affects stringification, I don't find it unreasonable to expect that
it should also affect the parsing of numbers, e.g. "1,2" + 1.3 should
be 2.5 if the decimal separator is a "," and not a ".". You don't
achieve that by just preventing the caching of stringified numbers
alone.

But I think using lexical scope for locale is fundamentally at odds
with Perl's polymorphic data types; selectively disabling caching of
representation will not result in a fully consistent model. The
wisdom of chairman Jarkko still applies: "Avoid!"

Furthermore, think about string eval:

$a = 1.2;
$b = eval "$a + 1.5";

This will always break if $a stringifies to "1,2".

I think this is undesirable, and implicit conversion between strings
and numbers should *always* use the "C" locale. Note that perllocale
says:

Category LC_NUMERIC: Numeric Formatting
After a proper POSIX::setlocale() call, Perl obeys the "LC_NUMERIC"
locale information, which controls an application's idea of how numbers
should be formatted for human readability by the printf(), sprintf(),
and write() functions. String-to-numeric conversion by the
POSIX::strtod() function is also affected.

There is no mention that implicit conversion should observe the
LC_NUMERIC setting; only explicit conversions requested by the
programmer are documented to behave that way.

Cheers,
-Jan

[Steffen Mueller]

On 04/26/2013 08:03 PM, Jan Dubois wrote:
> I think this is undesirable, and implicit conversion between strings
> and numbers should *always* use the "C" locale.

Yes, please! Locales are just plain old crazy.

--Steffen

[Nicholas Clark]

On Thu, Apr 25, 2013 at 06:07:11PM -0700, Jan Dubois wrote:
> The code that turns an NV into a PV uses the Gconvert() macro. It will
> always use the current locale, regardless of any locale pragma being
> in effect or not, AFAICT.
>
> I guess this is the bug then. The problem of course is that saving
> the current locale, setting the locale to "C", converting the number,
> and then restoring the original locale is quite a bit of overhead for
> the regular use case that isn't using locale.
>
> So the simple call to Gconvert() in Perl_sv_2pv_flags() would then become:
>
> #ifdef USE_LOCALE_NUMERIC
> char *loc = savepv(setlocale(LC_NUMERIC, NULL));
> setlocale(LC_NUMERIC, "C");
> #endif
> Gconvert(SvNVX(sv), NV_DIG, 0, s);
> #ifdef USE_LOCALE_NUMERIC
> setlocale(LC_NUMERIC, loc);
> Safefree(loc);
> #endif
>
> That looks rather expensive to me.

It turns out that it's not.

Here's the test code for the first 3 examples:

#include <locale.h>
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char **argv) {
int loop = 0;
char buffer[15 + 20];
if (argv[1] && !setlocale(LC_NUMERIC, argv[1])) {
perror("setlocale");
return 0;
}
do {
double d = loop + 0.5;
char *was = setlocale(LC_NUMERIC, NULL);

if (!was) {
perror("setlocale");
return 0;
}

if (was[0] == 'C' && was[1] == '\0') {
gcvt(d, 15, buffer);
} else {
if (!setlocale(LC_NUMERIC, "C")) {
perror("setlocale");
return 0;
}

gcvt(d, 15, buffer);

if (!setlocale(LC_NUMERIC, was)) {
perror("setlocale");
return 0;
}
}
} while (++loop < 10000000);
puts(buffer);
return 0;
}


default (no macros defined) - do what we do now, don't setlocale at all.
This demonstrates that we get different results:
[nicholas@dromedary-001 test]$ ./locale-default hr_HR.iso88592
9999999,5
[nicholas@dromedary-001 test]$ ./locale-default
9999999.5

always (-DALWAYS) - always setlocale to "C", and then back to what it was:
[nicholas@dromedary-001 test]$ ./locale-always
9999999.5
[nicholas@dromedary-001 test]$ ./locale-always hr_HR.iso88592
9999999.5

smarter (-DSMARTER) - always setlocale to "C", but only set it back if it
wasn't C before. Output as before.

smartest (attached code) - setlocale NULL to read the locale, and then
set/restore if it's not C. This one turns out to be most interesting. Output
as before.

dumbbench says

default Rounded run time per iteration: 6.767e+00 +/- 2.3e-02 (0.3%)
default (hr_HR) Rounded run time per iteration: 6.755e+00 +/- 1.0e-02 (0.1%)
always Rounded run time per iteration: 7.030e+00 +/- 1.7e-02 (0.2%)
always (hr_HR) Rounded run time per iteration: 7.039e+00 +/- 1.7e-02 (0.2%)
smarter Rounded run time per iteration: 7.006e+00 +/- 1.8e-02 (0.3%)
smarter (hr_HR) Rounded run time per iteration: 7.020e+00 +/- 1.3e-02 (0.2%)
smartest Rounded run time per iteration: 6.874e+00 +/- 1.8e-02 (0.3%)
smartest (hr_HR) Rounded run time per iteration: 1.9302e+01 +/- 1.9e-02 (0.1%)


So about 4% slower. Apart from the one that looks like it should be more
efficient, which can be 186% slower. I have no clue why.

Note that the numeric conversion is cached. This is only going to be a
performance hit for code which needs to format a lot of numbers. If it's
a problem, we can also get a speedup by using integer formatting if the
value is safely an integer.

If it's *still* a problem we might like to investigate the Grisu3 algorithm
from http://florian.loitsch.com/publications/dtoa-pldi2010.pdf

I believe that that has been coded for V8 and then extracted as
http://code.google.com/p/double-conversion/
It's something like 4 times as fast for the 99% of values which it can cope
with. And we can force it to ignore locales (if the code even deals with
them. I've not looked that hard)

It's in C++, and I believe a 3 clause BSD licence. We could convert it to C.

Also, Python has considered using this, and then rejected it as not worth
the effort: http://bugs.python.org/issue12450

There doesn't seem to be much difference between the two approaches on x86_64
Linux. I'm wondering what would be a useful place to benchmark to see if
there is a difference - Win32? I would have thought that avoiding the
second setlocale() would be a win somewhere. But it is fractionally more code.

Anyway, I think it's worth doing. Just which of the two to use.

Nicholas Clark

[Steffen Mueller]

On 05/12/2013 09:44 PM, Nicholas Clark wrote:
> If it's *still* a problem we might like to investigate the Grisu3 algorithm
> from http://florian.loitsch.com/publications/dtoa-pldi2010.pdf
>
> I believe that that has been coded for V8 and then extracted as
> http://code.google.com/p/double-conversion/
> It's something like 4 times as fast for the 99% of values which it can cope
> with. And we can force it to ignore locales (if the code even deals with
> them. I've not looked that hard)
>
> It's in C++, and I believe a 3 clause BSD licence. We could convert it to C.

It's about 3.4k SLOC C++. I didn't check how much of it we'd need to
convert. Seems like lots of C++ OO code overhead. The paper you cite has
a simple C implementation.

> Also, Python has considered using this, and then rejected it as not worth
> the effort: http://bugs.python.org/issue12450
>
> There doesn't seem to be much difference between the two approaches on x86_64
> Linux. I'm wondering what would be a useful place to benchmark to see if
> there is a difference - Win32? I would have thought that avoiding the
> second setlocale() would be a win somewhere. But it is fractionally more code.
>
> Anyway, I think it's worth doing. Just which of the two to use.

Agreed. Any takers? :)

--Steffen

[Karl Williamson]

It turns out that we already keep track of if we are in the C locale or
not for the numeric radix character. PL_numeric_standard contains that
information. This means that the most common case will only have a
single extra test, of this bit.

The attached draft patch demonstrates this. It also appears to fix
https://rt.perl.org/rt3/Public/Bug/Display.html?id=108378 and
https://rt.perl.org/rt3/Public/Bug/Display.html?id=115800

It does this by not setting SvPOK when in the scope of 'use locale'.
Without this, when upgrading an NV to a PVNV, the stringified version
retains whatever radix character it had at the time it was stringified,
regardless of whatever the current locale calls for. Even implicit
stringification, as in Jan's example:

$a = 1.2;
$b = eval "$a + 1.5";

sets $a to "1,2" if the current locale calls for that radix, where it
remains, potentially causing problems down the road. (#108378 is about
this; the attached patch causes this example to work correctly.)

By not setting SvPOK, the NV remains an NV, and if a stringified version
is required, it must be recalculated each time; thus if the radix
changes by a call to setlocale, we will catch that.

It also has the side effect that the radix in every SVNV will be a dot,
as Steffen suggested, since we won't create an SVNV from an NV unless
outside the scope of locale, and we change to the C locale during each
such creation event, and the C locale uses a dot.

I believe that additionally, within the scope of 'use locale', any PVNV
should be treated as a plain NV, so that the radix won't be output as
what it was set to before the scope was entered.

I'm not familiar enough with this part of Perl to know if there is
something really wrong with this scheme; perhaps you do.

The problem with this patch is that it breaks code that has come to rely
on not needing 'use locale' to get locale behavior. The current
documentation is not totally clear about this. It says that you need
'use locale', but one example given doesn't include it. And the current
behavior is tested for in

lib/version/t/07locale.t

It doesn't do a 'use locale', and even if one is added, one test fails
because it relies on the stringification being retained outside the
scope of the 'use locale'.

Thus, this patch fixes some problems, and I believe makes locales work
more sanely, but breaks code that relies on behavior contrary to what
the documentation says (but in accord with an example in the documentation).

[Aaron Crane]

Nicholas Clark <ni...@ccl4.org> wrote:
> If it's *still* a problem we might like to investigate the Grisu3 algorithm
> from http://florian.loitsch.com/publications/dtoa-pldi2010.pdf
>
> I believe that that has been coded for V8 and then extracted as
> http://code.google.com/p/double-conversion/
> It's something like 4 times as fast for the 99% of values which it can cope
> with.

My general view is that Perl would be well served by having a fast,
accurate, platform-independent floating-point stringification routine.
Our current approach of using the system's float formatter is
admirably simple, and means we don't have to maintain code to do this
tricky task. But the consequence is that, in some cases, Perl users
are exposed to surprising platform differences, some of which are
likely to be perceived as bugs.

As for Grisu3 in particular, I note this line from Loitsch's paper:

"Our algorithms are fast, because they require only fixed-size integer
arithmetic. The sole requirement for the integer type is that it has
at least two more bits than the significand of the floating-point
number."

This is fine for typical systems where NV == IEEE double and there's a
64-bit integer type available. But we also support -Duselongdouble
and -Dusemorebits, which probably give NV at least 64 bits of
significand on typical platforms; and we're relatively unlikely to
have an N-bit integer type for N >= 66. So we'd some non-Grisu3
fallback when building with NV == long double. Then again, Grisu3
itself needs a fallback for the ~0.5% of values which it can't handle,
so maybe that wouldn't be terrible — we'd merely skip the Grisu3
attempt when formatting on such a configuration.

It also occurs to me that one of the ways people are likely to
encounter platform differences in float formatting are for infinities
and NaNs — IIRC, they differ in spelling even on platforms that
support them properly. So if we were looking to mitigate platform
differences even in the non-locale case, merely special-casing
non-finite values somewhere in the guts of Perl_sv_vcatpvfn_flags()
might be a reasonable idea.

> Also, Python has considered using this, and then rejected it as not worth
> the effort: http://bugs.python.org/issue12450

Part of the "not worth the effort" determination for Python seems to
be that, since they already have a cross-platform float-formatting
implementation, switching to a different implementation risks
unacceptable user-visible changes. The premise there doesn't directly
apply to us, though I can imagine situations where the conclusion
would hold regardless — switching away from the platform's formatting
implementation still risks an unacceptable user-visible change.

You'll note that I'm glossing over what sorts of user-visible change
would count as "unacceptable".

> There doesn't seem to be much difference between the two approaches on x86_64
> Linux. I'm wondering what would be a useful place to benchmark to see if
> there is a difference - Win32? I would have thought that avoiding the
> second setlocale() would be a win somewhere. But it is fractionally more code.

It seems plausible that libc implementations would special-case
switching to the C locale, or repeatedly switching between a pair of
locales, or both, to optimise just this sort of use case where a
program running in one locale needs to generate locale-independent
output for some non–user-facing purpose. So, yes, running your locale
benchmarks on the platforms we care about makes sense to me.

--
Aaron Crane ** http://aaroncrane.co.uk/

[bulk88]

Nicholas Clark wrote:
>
> dumbbench says
>
> default Rounded run time per iteration: 6.767e+00 +/- 2.3e-02 (0.3%)
> default (hr_HR) Rounded run time per iteration: 6.755e+00 +/- 1.0e-02 (0.1%)
> always Rounded run time per iteration: 7.030e+00 +/- 1.7e-02 (0.2%)
> always (hr_HR) Rounded run time per iteration: 7.039e+00 +/- 1.7e-02 (0.2%)
> smarter Rounded run time per iteration: 7.006e+00 +/- 1.8e-02 (0.3%)
> smarter (hr_HR) Rounded run time per iteration: 7.020e+00 +/- 1.3e-02 (0.2%)
> smartest Rounded run time per iteration: 6.874e+00 +/- 1.8e-02 (0.3%)
> smartest (hr_HR) Rounded run time per iteration: 1.9302e+01 +/- 1.9e-02 (0.1%)
>
>
> So about 4% slower. Apart from the one that looks like it should be more
> efficient, which can be 186% slower. I have no clue why.
>

.................................

>
> There doesn't seem to be much difference between the two approaches on x86_64
> Linux. I'm wondering what would be a useful place to benchmark to see if
> there is a difference - Win32? I would have thought that avoiding the
> second setlocale() would be a win somewhere. But it is fractionally more code.
>
> Anyway, I think it's worth doing. Just which of the two to use.
>
> Nicholas Clark
>

I got the test program to compile on Win32. How do I benchmark it in an
similar way (dumbbench) to your numbers above?

[Jan Dubois]

Dumbbench is on CPAN. I haven't looked; but can't it be used on Windows too?

Cheers,
-Jan

[bulk88]

Jan Dubois wrote:
> Dumbbench is on CPAN. I haven't looked; but can't it be used on Windows too?
>

A dep of Dumbbench (and Dumbbench has too many deps IHMO),
Statistics-CaseResampling doesn't compile on Visual C due to C99. A
patch is waiting on github for tsee to deal with as of earlier today
that adds VC compatibility.

[Nicholas Clark]

Thanks for your persistence. All the dependencies have built just fine for
me using the CPAN shell, so I tend not to notice how many there are.

If that has got you to the point of being able to run dumbbench, then as
you've probably figured I hand pasted that table together using the output
from about 8 runs, where the runs were like this:

[nicholas@dromedary-001 test]$ dumbbench ./locale-smartest hr_HR.iso88592
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
cmd: Ran 20 iterations (0 outliers).
cmd: Rounded run time per iteration: 1.9302e+01 +/- 1.9e-02 (0.1%)

Also, somewhat strangely, a different x86_64 Linux machine doesn't take 3
times as long for that code.

Nicholas Clark

[bulk88]

Nicholas Clark wrote:
> On Mon, May 13, 2013 at 01:32:54PM -0400, bulk88 wrote:
>> Jan Dubois wrote:
>>> Dumbbench is on CPAN. I haven't looked; but can't it be used on Windows too?
>>>
>> A dep of Dumbbench (and Dumbbench has too many deps IHMO),
>> Statistics-CaseResampling doesn't compile on Visual C due to C99. A
>> patch is waiting on github for tsee to deal with as of earlier today
>> that adds VC compatibility.
>
> Thanks for your persistence. All the dependencies have built just fine for
> me using the CPAN shell, so I tend not to notice how many there are.

on 2nd run I had to cut down dumbbench run count since the first run
took more than 30 minutes
__________________________________________________________________
C:\Documents and Settings\Owner\Desktop>dumbbench main

9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
cmd: Ran 20 iterations (0 outliers).

cmd: Rounded run time per iteration: 3.84383e+001 +/- 5.2e-003 (0.0%)

C:\Documents and Settings\Owner\Desktop>dumbbench main hr_HR.iso88592
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
setlocale: No error
cmd: Ran 29 iterations (9 outliers).
cmd: Rounded run time per iteration: -1.0516e-002 +/- 4.4e-005 (-0.4%)

C:\Documents and Settings\Owner\Desktop>dumbbench -i5 main hun
Number of initial runs is very small (<6). Precision will be off. at
C:\perl517\
bin/dumbbench line 98.

9999999.5
9999999.5
9999999.5
9999999.5
9999999.5
9999999.5

cmd: Ran 5 iterations (0 outliers).
cmd: Rounded run time per iteration: 7.676e+001 +/- 1.1e-001 (0.1%)

C:\Documents and Settings\Owner\Desktop>

__________________________________________________________________

main.exe is
__________________________________________________________________

_______________________________________________________________
I notice the decimal separator didn't change. IDK why.


main.exe was compiled with VC 2003,

cl -c -nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -GL -G7 -DWIN32 -MD -Zi
-DNDEBUG -O1 -GL -G7 main.c

and

link -out:main.exe -nologo -nodefaultlib -debug -opt:ref,icf -ltcg
-machine:x86 main.obj oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib

[Steffen Mueller]

On 05/13/2013 07:32 PM, bulk88 wrote:
> A dep of Dumbbench (and Dumbbench has too many deps IHMO),

*shrug* This dep is for an efficient O(n) quantile calculation. Good
luck with that in Perl.

> Statistics-CaseResampling doesn't compile on Visual C due to C99. A
> patch is waiting on github for tsee to deal with as of earlier today
> that adds VC compatibility.

Will take care of this asap. Sorry for the C99[1].

--Steffen

[1] Also, no cookie for you, Microsoft, for not doing C99 in 2013!

[Steffen Mueller]

[note directed at Daniel]

On 05/13/2013 09:43 PM, Steffen Mueller wrote:
>> Statistics-CaseResampling doesn't compile on Visual C due to C99. A
>> patch is waiting on github for tsee to deal with as of earlier today
>> that adds VC compatibility.

> [1] Also, no cookie for you, Microsoft, for not doing C99 in 2013!

By the way, this clearly shows a lack of Win32 MSVC CPAN smokers since
out of 409 test reports, one is N/A, none is FAIL, which I would expect
due to this.

--Steffen

[Nicholas Clark]

On Mon, May 13, 2013 at 08:59:19AM -0600, Karl Williamson wrote:

> It turns out that we already keep track of if we are in the C locale or
> not for the numeric radix character. PL_numeric_standard contains that
> information. This means that the most common case will only have a
> single extra test, of this bit.

Nice to know.

> The attached draft patch demonstrates this. It also appears to fix
> https://rt.perl.org/rt3/Public/Bug/Display.html?id=108378 and
> https://rt.perl.org/rt3/Public/Bug/Display.html?id=115800
>
> It does this by not setting SvPOK when in the scope of 'use locale'.
> Without this, when upgrading an NV to a PVNV, the stringified version
> retains whatever radix character it had at the time it was stringified,
> regardless of whatever the current locale calls for. Even implicit
> stringification, as in Jan's example:
>
> $a = 1.2;
> $b = eval "$a + 1.5";
>
> sets $a to "1,2" if the current locale calls for that radix, where it
> remains, potentially causing problems down the road. (#108378 is about
> this; the attached patch causes this example to work correctly.)

Yes, this seems like a sane approach. I can't spot any real flaws yet.
(Meaning that I don't trust myself)

> I believe that additionally, within the scope of 'use locale', any PVNV
> should be treated as a plain NV, so that the radix won't be output as
> what it was set to before the scope was entered.
>
> I'm not familiar enough with this part of Perl to know if there is
> something really wrong with this scheme; perhaps you do.

I am not familiar with it either. And, as you say, what we have now leaks
the radix from a PVNV. Right now, C<no locale> leaks in:

$ cat locale.pl
#!/usr/bin/perl -w
use strict;

use POSIX ':locale_h';

$a = 3.14;

$b = $b = "$a" if shift;

setlocale(LC_ALL, "hr_HR");

use locale;

print $a, "\n";

__END__
$ ./locale.pl 0
3,14
$ ./locale.pl 1
3.14


in that if C<use locale> is supposed to be affecting numeric conversions
within its scope, even the implicit conversions, then it should be "3,14"
in both cases, and not happen to be "3.14" just because of the side effects
of an earlier use in a string context.

But that one feels harder to fix. Seems that one has to rid all the SvPV*
macros to ignore the cached string in the scope of C<use locale>

Which feels like it's going to be ugly. To the point that I'm not sure if
it's workable.

> Thus, this patch fixes some problems, and I believe makes locales work
> more sanely, but breaks code that relies on behavior contrary to what
> the documentation says (but in accord with an example in the documentation).

We're allowed to be wrong, and correct our mistakes.

Although generally it helps when doing this that it's clear that the new
model of things is more self-consistent than the old model :-)

Nicholas Clark

[bulk88]

I agree there is a lack of VC smokers on cpantesters. But 1 VC
cpantesters report exists
http://www.cpantesters.org/cpan/report/558acdac-e361-1015-a4a1-18a805085451
. I think the lack of VC smokers is because the 800 lb gorilla is
reliably running VC CPAN smokes at
http://code.activestate.com/ppm/Statistics-CaseResampling/ so there is
no need for volunteer VC smokers really. Meanwhile there is no
equivalent of AS for the GCC/Strawberry Perl platform, so there are many
GCC/Strawberry Perl volunteers. But AS PPM build farm doesn't send
emails on failure like cpantesters does AFAIK. Some tester could also
add "-std=c89" to their Strawberry smoker to create a psuedo-VC. Me
setting up a cpan smoker is low on my priority list.

[Nawglan]

I'll look into setting up a smoke tester using vc2010 sometime this week at work.

[Leon Timmermans]

This all sounds sensible, however

> I believe that additionally, within the scope of 'use locale', any PVNV
> should be treated as a plain NV, so that the radix won't be output as what
> it was set to before the scope was entered.
>
> I'm not familiar enough with this part of Perl to know if there is something
> really wrong with this scheme; perhaps you do.

I think you mean any SvNOK PVNV. PVNV can also contain cached.
non-authoritative NV values. I think that should include SvNOK PVMG's
and PVLV's too. Not sure how workable this is.

Also, you're suggesting that in:

perl -MDevel::Peek -E 'my $num = "1.2"; log $num; say Dump $num;'

the string "1.2" should be printed as a "1,2" because the variable
happens to have been used in numeric context. I'm not sure that's
sane. This is one of those places where not having proper types is
painful.

> The problem with this patch is that it breaks code that has come to rely on
> not needing 'use locale' to get locale behavior. The current documentation
> is not totally clear about this. It says that you need 'use locale', but
> one example given doesn't include it. And the current behavior is tested
> for in
>
> lib/version/t/07locale.t
>
> It doesn't do a 'use locale', and even if one is added, one test fails
> because it relies on the stringification being retained outside the scope of
> the 'use locale'.

That's just wrong IMO.

Leon

[Aristotle Pagaltzis]

* Nicholas Clark <ni...@ccl4.org> [2013-05-13 21:55]:

> And, as you say, what we have now leaks the radix from a PVNV. Right
> now, C<no locale> leaks in:
>
> $ cat locale.pl
> #!/usr/bin/perl -w
> use strict;
>
> use POSIX ':locale_h';
>
> $a = 3.14;
>
> $b = $b = "$a" if shift;
>
> setlocale(LC_ALL, "hr_HR");
>
> use locale;
>
> print $a, "\n";
>
> __END__
> $ ./locale.pl 0
> 3,14
> $ ./locale.pl 1
> 3.14
>
>
> in that if C<use locale> is supposed to be affecting numeric conversions
> within its scope, even the implicit conversions, then it should be "3,14"
> in both cases, and not happen to be "3.14" just because of the side effects
> of an earlier use in a string context.
>
> But that one feels harder to fix. Seems that one has to rid all the SvPV*
> macros to ignore the cached string in the scope of C<use locale>
>
> Which feels like it's going to be ugly. To the point that I'm not sure if
> it's workable.

This is a case where if the initial SV type were marked as a canonical,
a fix would be trivial, no?

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>

[Karl Williamson]

On 05/13/2013 06:25 PM, Aristotle Pagaltzis wrote:
> This is a case where if the initial SV type were marked as a canonical,
> a fix would be trivial, no?

I believe so

[Nicholas Clark]

No.

Certainly not directly. Knowing that something is canonically a number
doesn't solve the problem - that with locales a *number* can have more than
one "correct" stringification, and we have no way to store two
cached stringifications, let alone tag them such that the correct one is
returned in context.

For example

$a = 3.14;
$a =~ /,/;

Should that match?

If locales are affecting stringification, and that is running within the
scope of C<use locale>, then it will depend on the locale's radix character.

But that then means that the call to SvPV() [or equivalent] within the
regular expression engine C code needs to behave differently depending on
circumstances. And that information is not passed down from the OP.

Nicholas Clark

[Dr.Ruud]

I would keep 'stringification' and 'presentation' separated.

I think that the 'presentation' value should always be the (lazy) result
of a format-action.

It would be great :) if that 'presentation' value can be cached with the
value somehow, where it then would follow the status of the
stringification. That (cached) 'presentation' value could even be (the
id of) the opcodes to transform the 'stringification' into the
'presentation'.

Treat the "." radix as we treat "\n".
It could even look like \. where needed.

--
Ruud

[Nicholas Clark]

On Thu, May 16, 2013 at 05:36:54PM +0200, Dr.Ruud wrote:
> On 16/05/2013 17:08, Nicholas Clark wrote:
> > On Thu, May 16, 2013 at 08:58:27AM -0600, Karl Williamson wrote:
> >> On 05/13/2013 06:25 PM, Aristotle Pagaltzis wrote:
>
> >>> This is a case where if the initial SV type were marked as a canonical,
> >>> a fix would be trivial, no?
> >>
> >> I believe so
> >
> > No.
> >
> > Certainly not directly. Knowing that something is canonically a number
> > doesn't solve the problem - that with locales a *number* can have more than
> > one "correct" stringification, and we have no way to store two
> > cached stringifications, let alone tag them such that the correct one is
> > returned in context.
> >
> > For example
> >
> > $a = 3.14;
> > $a =~ /,/;
> >
> > Should that match?
> >
> > If locales are affecting stringification, and that is running within the
> > scope of C<use locale>, then it will depend on the locale's radix character.
> >
> > But that then means that the call to SvPV() [or equivalent] within the
> > regular expression engine C code needs to behave differently depending on
> > circumstances. And that information is not passed down from the OP.
>
> I would keep 'stringification' and 'presentation' separated.

Yes, but "If I were you sir, I wouldn't start from here"

> I think that the 'presentation' value should always be the (lazy) result
> of a format-action.
>
> It would be great :) if that 'presentation' value can be cached with the
> value somehow, where it then would follow the status of the
> stringification. That (cached) 'presentation' value could even be (the
> id of) the opcodes to transform the 'stringification' into the
> 'presentation'.
>
> Treat the "." radix as we treat "\n".
> It could even look like \. where needed.

I don't think that any of that contributes any insight in how to get the
current implementation from where it is to a better place.

I intentionally leave my quoted message in place to show some of the
problems with the current implementation that make what you suggest hard to
implement.

Nicholas Clark

[Karl Williamson]

Actually, I'm not suggesting that. I ran this code with my patch
(adding in locale usage), and it worked. The reason is that forcing it
to be a numeric doesn't affect the stringification that's already in place.

perl -MPOSIX -MDevel::Peek -E 'POSIX::setlocale(POSIX::LC_ALL,
"de_DE.UTF8-8"); my $num = "1.2"; use locale; log $num; say $num; say
Dump $num;'

This is one of those places where not having proper types is
> painful.


>
>> The problem with this patch is that it breaks code that has come to rely on
>> not needing 'use locale' to get locale behavior. The current documentation
>> is not totally clear about this. It says that you need 'use locale', but
>> one example given doesn't include it. And the current behavior is tested
>> for in
>>
>> lib/version/t/07locale.t
>>
>> It doesn't do a 'use locale', and even if one is added, one test fails
>> because it relies on the stringification being retained outside the scope of
>> the 'use locale'.
>
> That's just wrong IMO.

I presume you mean the test is just wrong.

>
> Leon
>

[Karl Williamson]

My patch fixes this.

>
>
> in that if C<use locale> is supposed to be affecting numeric conversions
> within its scope, even the implicit conversions, then it should be "3,14"
> in both cases, and not happen to be "3.14" just because of the side effects
> of an earlier use in a string context.
>
> But that one feels harder to fix. Seems that one has to rid all the SvPV*
> macros to ignore the cached string in the scope of C<use locale>
>
> Which feels like it's going to be ugly. To the point that I'm not sure if
> it's workable.

I believe you are talking about the case
use strict;

$a = 3.14;
$b = $b = "$a" if shift;

POSIX::setlocale(POSIX::LC_ALL,"de_DE.UTF-8");

use locale;
print $a, "\n";

In that case my patch doesn't work, because $a is set POK because it is
stringified outside of the 'use locale' scope.

What I've since done to fix that is to change the eight macros that are
variants of this one:

/* Within locales, an NOK scalar isn't considered POK, thus forcing
* re-stringification. This is because the locale (and hence the current
* proper radix character stringification) can change at any time */
#define SvPOK(sv) ((SvFLAGS(sv) & SVf_POK) \
&& (! SvNOK(sv) || LIKELY(! IN_LOCALE_RUNTIME)))


That forces the SVPV macros to call sv_2pv_flags() each time the
stringification is needed of an NOK scalar, within the scope of 'use
locale'. With that, the code above works.

[Nicholas Clark]

On Thu, May 16, 2013 at 10:26:42AM -0600, Karl Williamson wrote:

> /* Within locales, an NOK scalar isn't considered POK, thus forcing
> * re-stringification. This is because the locale (and hence the current
> * proper radix character stringification) can change at any time */
> #define SvPOK(sv) ((SvFLAGS(sv) & SVf_POK) \
> && (! SvNOK(sv) || LIKELY(! IN_LOCALE_RUNTIME)))

That approach starts to make me feel very uncomfortable.
Whilst it doesn't contradict the documentation for SvPOK()

=for apidoc Am|U32|SvPOK|SV* sv
Returns a U32 value indicating whether the SV contains a character
string.


it does change behaviour since 5.000, and contradicts this comment:

#define SVf_POK 0x00000400 /* has valid public pointer value */

> That forces the SVPV macros to call sv_2pv_flags() each time the
> stringification is needed of an NOK scalar, within the scope of 'use
> locale'. With that, the code above works.

Yes, one has to end up with that approach. The alternative, which at the
moment I think I'm liking better, is to *not* set SvPOK() on stringification
of NVs. (Or at least, NVs which aren't integers). Which would retain the
current meaning of SVf_POK, leave SvPOK() unchanged, and cause the logic
about what to return to be in one place. This also reduces the code size.
However, the cost is more function calls.

I can't see a way of doing this without more complexity.

Nicholas Clark

[Karl Williamson]

Me neither. I'm fine with doing it as you suggest. My scheme was an
attempt to minimize real-time impact on programs that don't use locale.

I do believe, however, that we should issue a statement deprecating any
direct accesses to Perl core data structures. All accesses should go
through some set/get accessor macros or functions; and if any need to be
written, I'll volunteer to do so. We may not be able to enforce this
with warnings, but it's crazy to be constrained to not making changes
because some XS writer is or could be twiddling some internal bits.

[Jan Dubois]

I'm slightly confused by this discussion. AFAICT, there are currently
2 different opinions about the "desired" behavior:

Steffen and me believe that implicit stringification should always use
the "C" locale and ignore the current locale setting. This seems to be
backed by the documentation, and also seems the only way in which
string eval "" can work with interpolated floating point numbers.

Nicholas and Karl seem to be discussing implementation details on how
implicit stringification could observe the lexical effect of the
locale pragma.

I find the implementation discussion premature while there is no
consensus on the desired semantics (or what part of the current
implementation is considered to be buggy). What are the arguments in
favor of using current locale for implicit stringification?

Cheers,
-Jan

[Karl Williamson]

On 04/26/2013 12:03 PM, Jan Dubois wrote:
> On Fri, Apr 26, 2013 at 7:51 AM, Karl Williamson
> <pub...@khwilliamson.com> wrote:
>> It seems to me, with a little more thought, again without doing much
>> investigation, that the original bug might be solvable simply by not setting
>> SvPVOK when doing a Gconvert in the scope of locale.
>
> Well, it depends how far you want to go. Once you accept that locale
> affects stringification, I don't find it unreasonable to expect that
> it should also affect the parsing of numbers, e.g. "1,2" + 1.3 should
> be 2.5 if the decimal separator is a "," and not a ".". You don't
> achieve that by just preventing the caching of stringified numbers
> alone.
>
> But I think using lexical scope for locale is fundamentally at odds
> with Perl's polymorphic data types; selectively disabling caching of
> representation will not result in a fully consistent model. The
> wisdom of chairman Jarkko still applies: "Avoid!"
>
> Furthermore, think about string eval:

>
> $a = 1.2;
> $b = eval "$a + 1.5";
>

> This will always break if $a stringifies to "1,2".

>
> I think this is undesirable, and implicit conversion between strings

> and numbers should *always* use the "C" locale. Note that perllocale
> says:
>
> Category LC_NUMERIC: Numeric Formatting
> After a proper POSIX::setlocale() call, Perl obeys the "LC_NUMERIC"
> locale information, which controls an application's idea of how numbers
> should be formatted for human readability by the printf(), sprintf(),
> and write() functions. String-to-numeric conversion by the
> POSIX::strtod() function is also affected.
>
> There is no mention that implicit conversion should observe the
> LC_NUMERIC setting; only explicit conversions requested by the
> programmer are documented to behave that way.

The documentation is poorly worded, as the next paragraph after the one
you cite does talk about other effects of LC_NUMERIC:

"Output produced by print() is also affected by the current locale: it
corresponds to what you'd get from printf() in the "C" locale. The
same is true for Perl's internal conversions between numeric and
string formats:"

And there follows some examples. The sentence about printf() doesn't
make sense to me, but the examples make it clear that the current
locale's radix character is output. Here are the relevant lines:

$n = 5/2; # Assign numeric 2.5 to $n
$a = " $n"; # Locale-dependent conversion to string
print "half five is $n\n"; # Locale-dependent output
printf "half five is %g\n", $n; # Locale-dependent output

I mentioned in an earlier email that I tested your eval example on a
perl that didn't do the caching, and it works properly, yielding 3.7
internally, and printing as "3,7", all in a locale in which comma is the
radix character. Similarly, your example

"1,2" + 1.3

prints
2,5

[Dr.Ruud]

On 21/05/2013 21:12, Karl Williamson wrote:

> in a locale in which comma is the

> radix character [...]

>
> "1,2" + 1.3
>
> prints
> 2,5

I think it would be good to get rid of that. Leave it to overloaded
operators, so generally don't do it.

--
Ruud

[Jan Dubois]

On Tue, May 21, 2013 at 12:12 PM, Karl Williamson
<pub...@khwilliamson.com> wrote:
> The documentation is poorly worded, as the next paragraph after the one you
> cite does talk about other effects of LC_NUMERIC:
>
> "Output produced by print() is also affected by the current locale: it
> corresponds to what you'd get from printf() in the "C" locale. The
> same is true for Perl's internal conversions between numeric and
> string formats:"
>
> And there follows some examples. The sentence about printf() doesn't make
> sense to me, but the examples make it clear that the current locale's radix
> character is output. Here are the relevant lines:
>
> $n = 5/2; # Assign numeric 2.5 to $n
> $a = " $n"; # Locale-dependent conversion to string
> print "half five is $n\n"; # Locale-dependent output
> printf "half five is %g\n", $n; # Locale-dependent output

Ok, that clearly shows that the intend is that implicit conversations
do observe the current locale.

> I mentioned in an earlier email that I tested your eval example on a perl
> that didn't do the caching, and it works properly, yielding 3.7 internally,
> and printing as "3,7", all in a locale in which comma is the radix
> character. Similarly, your example
>
> "1,2" + 1.3
>
> prints
> 2,5

Note that my example used string eval. I don't think you can get that
to work though, so this would then be a limitation of the locale mode:

$a = 1.2;
$b = eval "$a + 1.3";

because Perl parsing doesn't follow locale, so $b would be set to 3.3
instead of 2.5 under a locale that uses a ',' radix. Unfortunately it
wouldn't even generate a warning.

I still feel that using "C" locale for implicit conversions would be
cleaner, but thanks for pointing out that the current behavior was
intentional and not just a side effect.

Cheers,
-Jan

[Jan Dubois]

On Tue, May 21, 2013 at 12:12 PM, Karl Williamson
<pub...@khwilliamson.com> wrote:

> On 04/26/2013 12:03 PM, Jan Dubois wrote:
>>
>> Furthermore, think about string eval:
>>
>> $a = 1.2;
>> $b = eval "$a + 1.5";
>>
>> This will always break if $a stringifies to "1,2".
>

> I mentioned in an earlier email that I tested your eval example on a perl
> that didn't do the caching, and it works properly, yielding 3.7 internally,
> and printing as "3,7", all in a locale in which comma is the radix
> character.

I didn't notice in my earlier reply that my original example was
quoted above. I don't see how $b could ever be 3.7. It should really
be 2.7, but I can see how it evaluate to 3.5 under the ',' radix. But
I would say that 3.5 would clearly be unexpected.

$ perl -wE '$b=eval "1.2+1.5";say $b'
2.7
$ perl -wE '$b=eval "1,2+1.5";say $b'
3.5

Cheers,
-Jan

[Karl Williamson]

Oops. I probably had confirmation bias when I said that. I did not get
3.7. I got 3.5, and thought I got 2.7, which is what I meant to type.

The comma is being parsed as the comma operator within the eval, and the
1 is thrown away, leaving 2+1.5 = 3.5.
That's how it should be (if unexpected) outside of a comma radix locale.
But within one, ... I start to see your point.

Does (1,0) mean a list with two elements or does it mean a floating point 1?

So I agree, that a string eval requires a dot radix character. But we
have an obligation to try to support those advertised behaviors as well.

[Dr.Ruud]

On 21/05/2013 23:23, Karl Williamson wrote:
> On 05/21/2013 01:51 PM, Jan Dubois wrote:

>> $ perl -wE '$b=eval "1.2+1.5";say $b'
>> 2.7
>> $ perl -wE '$b=eval "1,2+1.5";say $b'
>> 3.5
>

> [...]

> The comma is being parsed as the comma operator within the eval, and the
> 1 is thrown away, leaving 2+1.5 = 3.5.
> That's how it should be (if unexpected) outside of a comma radix locale.

Not only inside a "comma radix locale", just always.
A module can add a localeval() for "the other thing".

> But within one, ... I start to see your point.
>
> Does (1,0) mean a list with two elements or does it mean a floating
> point 1?

Always a list, is what the doctor orders.

> So I agree, that a string eval requires a dot radix character. But we
> have an obligation to try to support those advertised behaviors as well.

Or decide that it was a mistake, and get rid of it.

Should eval q{"\r\n" =~ /\A\n\z/} generally be true on Windows?
(yes, that is not a locale, but it is just as orthogonal)

Move exceptions out, and have modules take care of them.

perllocale:
"Perl is also aware that some character other than "." may be preferred
as a decimal point, and that output date representations may be
language‐specific."

That "may be preferred" promises some support. But don't let it leak out
of the sprintf/scanf "layer", and if already leaking then push it back
in there.

The regex-language can be made to support POSIX::localeconv more,
for example like [:mon_thousands_sep:].

--
Ruud

[Jan Dubois]

On Sun, Jun 9, 2013 at 7:21 AM, Ricardo Signes
<perl...@rjbs.manxome.org> wrote:
> * Jan Dubois <ja...@activestate.com> [2013-05-21T13:41:29]

>> Steffen and me believe that implicit stringification should always use
>> the "C" locale and ignore the current locale setting. This seems to be
>> backed by the documentation, and also seems the only way in which
>> string eval "" can work with interpolated floating point numbers.
>

> Implicit stringification in what sense? With LC_NUMERIC in effect, we promise
> that print($x) will stringify numbers according to locale rules. It seems
> somewhat perverse to then say that print("$x") will do otherwise.
>
> Did I miss a detail somewhere?

Sorry, it was my incomplete reading of perllocale.pod that led me to
believe that only printf() would follow the current locale. After
re-reading it, I find the documentation self-contradictory:

| Output produced by print() is also affected by the current locale: it
| corresponds to what you'd get from printf() in the "C" locale.

If the output from print() corresponds to what you get from printf()
under the "C" locale, then print() would *not* be affected by the
current locale.

But anyways, the next sentence, and the examples further down make it
clear that the intention has been that the current locale does affect
internal conversations between numbers and strings. As I showed in the
other thread about this topic, this has some unpleasant side-effects
for string eval that includes interpolated floating point numbers
(might silently break with incorrect results).

So I continue to consider this a bad language design decision, but
have no further objections to make the implementation match the
documented behavior (and maybe clarify the documentation a bit
regarding print()).

Cheers,
-Jan

PS: string eval with interpolated floating point numbers is already
broken on some platforms for Inf and NaN values, and only works
accidentally on the rest. But then most code doesn't deal with Inf and
NaN anyways, so this is a lesser problem.

[Karl Williamson]

On 06/09/2013 08:27 AM, Ricardo Signes wrote:
> * Karl Williamson <pub...@khwilliamson.com> [2013-05-21T11:43:50]

>> Me neither. I'm fine with doing it as you suggest. My scheme was
>> an attempt to minimize real-time impact on programs that don't use
>> locale.
>

> Nicholas was suggesting not caching locale-stringified strings for numbers.
> You (Karl) and I discussed this a while ago on #p5p and I said it was also what
> I thought we should've been doing. Can we make this happen?

I will work on a patch to do this.

I agree with Jan and Dr. Ruud that this is a poorly designed portion of
Perl. I see no other option but to continue to support it as best we
can. There needs to be cautions about the interactions in eval in both
perlfunc and perllocale. And there should be something in perlfunc
about the NaN and Inf anyway.

>
>> I do believe, however, that we should issue a statement deprecating
>> any direct accesses to Perl core data structures.
>

> Is this something we can plausibly do? I think it's the sort of thing that
> every guts hacker I've known has said we needed to do, but couldn't do without
> breaking tons of stuff. Perhaps we break them by degrees?
>
> At any rate, I am in favor of this to the extent that we don't break everything
> all at once. ;)


What I meant was adding something to perldelta and perhack, and perhaps
other pods (suggestions as to which are welcome) something like:

"Because of limitations of the C language, various undocumented
Perl-core-internal data items are visible to XS modules, for example,
global or per-interpreter variables such as C<PL_curstackinfo>. It is
deprecated to use these directly without going through a documented (in
L<perlapi>) function or macro interface. Starting in Perl v5.24.0, any
such data item can be removed or changed without prior notice. If you
feel you need access to any of these data items, send email to
L<mailto:perl5-...@perl.org>."

I don't see how we can generally make it so that warnings get generated
for illicit uses in the meantime.

[Father Chrysostomos via RT]

On Sun Jun 16 12:16:11 2013, pub...@khwilliamson.com wrote:
> What I meant was adding something to perldelta and perhack, and
> perhaps
> other pods (suggestions as to which are welcome) something like:
>
> "Because of limitations of the C language, various undocumented
> Perl-core-internal data items are visible to XS modules, for example,
> global or per-interpreter variables such as C<PL_curstackinfo>. It is
> deprecated to use these directly without going through a documented
> (in
> L<perlapi>) function or macro interface. Starting in Perl v5.24.0,
> any
> such data item can be removed or changed without prior notice. If you
> feel you need access to any of these data items, send email to
> L<mailto:perl5-...@perl.org>."

per...@perl.org, please. :-)

--

Father Chrysostomos


---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=108378

[James E Keenan via RT]

On Mon Jun 10 15:00:45 2013, perl...@rjbs.manxome.org wrote:
> * Jan Dubois <ja...@activestate.com> [2013-06-10T04:07:32]

> >
> > Sorry, it was my incomplete reading of perllocale.pod that led me to
> > believe that only printf() would follow the current locale. After
> > re-reading it, I find the documentation self-contradictory:

> > [...]
>
> ...and sorry about my reply, to which you're replying, as it was late
> to the
> game and you'd already covered this ground. I had not fully caught up
> in the
> thread when I wrote it.


Is this the same problem as
https://rt.perl.org/rt3/Ticket/Display.html?id=82418 ?

[Father Chrysostomos via RT]

On Sun Jun 16 17:40:05 2013, jkeenan wrote:
> On Mon Jun 10 15:00:45 2013, perl...@rjbs.manxome.org wrote:
> > * Jan Dubois <ja...@activestate.com> [2013-06-10T04:07:32]
> > >
> > > Sorry, it was my incomplete reading of perllocale.pod that led me to
> > > believe that only printf() would follow the current locale. After
> > > re-reading it, I find the documentation self-contradictory:
> > > [...]
> >
> > ...and sorry about my reply, to which you're replying, as it was late
> > to the
> > game and you'd already covered this ground. I had not fully caught up
> > in the
> > thread when I wrote it.
>
>
> Is this the same problem as
> https://rt.perl.org/rt3/Ticket/Display.html?id=82418 ?

No, I don’t think so. That one has to do with constant folding, and
whether the number will be stringified at compile time or run time.

--

Father Chrysostomos

[Karl Williamson]

On 06/16/2013 08:18 PM, Father Chrysostomos via RT wrote:
> On Sun Jun 16 17:40:05 2013, jkeenan wrote:
>> On Mon Jun 10 15:00:45 2013, perl...@rjbs.manxome.org wrote:
>>> * Jan Dubois <ja...@activestate.com> [2013-06-10T04:07:32]
>>>>
>>>> Sorry, it was my incomplete reading of perllocale.pod that led me to
>>>> believe that only printf() would follow the current locale. After
>>>> re-reading it, I find the documentation self-contradictory:
>>>> [...]
>>>
>>> ...and sorry about my reply, to which you're replying, as it was late
>>> to the
>>> game and you'd already covered this ground. I had not fully caught up
>>> in the
>>> thread when I wrote it.
>>
>>
>> Is this the same problem as
>> https://rt.perl.org/rt3/Ticket/Display.html?id=82418 ?
>
> No, I don’t think so. That one has to do with constant folding, and
> whether the number will be stringified at compile time or run time.
>

Also note that the author of #82418 expects that you don't need a 'use
locale' to have locale effects. It is (still) a bug that any of the
examples in this ticket output a comma radix character.