strftime year 2038 issue?

[Marc de Bourget]

It seems strftime encounters the year 2038 problem:
https://en.wikipedia.org/wiki/Year_2038_problem

I've tested with Windows 10, 32bit and this gawk version:
https://sourceforge.net/projects/ezwinports/files/gawk-4.1.3-w32-bin.zip/download

I set the Windows system date to the year 2039 and called with gawk:
print strftime()

The output was Null. Will this be fixed? The solution might be to use 64bit Integers.

A word of caution: After changing the Windows system date to a future date I lost my Antivirus protection due to expired licence, so there might be better ways to test this issue.

[Kaz Kylheku]

On 2016-04-24, Marc de Bourget <marcde...@gmail.com> wrote:
> It seems strftime encounters the year 2038 problem:
> https://en.wikipedia.org/wiki/Year_2038_problem
>
> I've tested with Windows 10, 32bit and this gawk version:
> https://sourceforge.net/projects/ezwinports/files/gawk-4.1.3-w32-bin.zip/download
>
>

> The output was Null. Will this be fixed?

Ask the C library vendor that provides the underlying time_t type.

> I set the Windows system date to the year 2039 and called with gawk:
> print strftime()

[...]

> A word of caution: After changing the Windows system date to a future
> date I lost my Antivirus protection due to expired licence, so there
> might be better ways to test this issue.

Maybe by using these things called arguments, which strftime takes?

$ gawk 'BEGIN { print strftime("%c", 123456789); }'
Thu 29 Nov 1973 01:33:09 PM PST

$ gawk 'BEGIN { print strftime("%c", 1234567890); }'
Fri 13 Feb 2009 03:31:30 PM PST

$ gawk 'BEGIN { print strftime("%c", 1234567890000000); }'
Fri 13 Dec 1901 12:45:52 PM PST

Oops!

Note that the C library strftime doesn't take the time as a simple
integer, but rather a "broken down time" in a "struct tm":

size_t strftime(char *s, size_t max,
const char *format, const struct tm *tm);

"struct tm" has fields like "tm_year". These are robust against
Y2039 even if 32 bit.

The problem is producing "struct tm" from a 32 bit time_t that counts
seconds since the Epoch.

> The solution might be to use 64bit Integers.

Gawk already has arbitrary-precision integers.

One problem is that in the stftime call, it just seems to be truncating
to a 32 bit time silently, leading to a garbage result rather than
an error.

[Kenny McCormack]

In article <9698b695-e961-4541...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:

>It seems strftime encounters the year 2038 problem:
>https://en.wikipedia.org/wiki/Year_2038_problem
>
>I've tested with Windows 10, 32bit and this gawk version:
>https://sourceforge.net/projects/ezwinports/files/gawk-4.1.3-w32-bin.zip/download

I think this is your problem right here------------------------^^

>
>I set the Windows system date to the year 2039 and called with gawk:
>print strftime()
>
>The output was Null. Will this be fixed? The solution might be to use 64bit Integers.
>
>A word of caution: After changing the Windows system date to a future date I lost
>my Antivirus protection due to expired licence, so there might be better ways to
>test this issue.

1) You can test without changing your system time, by passing an argument
to the strftime() call. It is slightly annoying that doing so requires
you to also pass in a "format arg", since the format arg is the first arg.
I always use "%c" as the first arg, while I silently wish (as, no doubt, do
the GAWK maintainers) that it were defined the other way - with the "number
of seconds" arg first, so that the "format" arg could be left to default).

But that ship has sailed...

2) On my 64 bit OSX system (using a self-compiled GAWK), it works fine:

$ gawk4 -l myTecla '{ print strftime("%c",$0) }'
GAWK input: 2000000000
Tue May 17 23:33:20 2033
GAWK input: 2100000000
Fri Jul 18 09:20:00 2036
GAWK input: 2200000000
Sun Sep 18 18:06:40 2039
GAWK input: 2300000000
Wed Nov 19 03:53:20 2042
GAWK input: 22300000000
Mon Aug 28 15:26:40 2676
GAWK input:
$

I would imagine that if you self-compiled it (on Windows) using a 64 bit
compiler, it would "just work".

--
Religion is what keeps the poor from murdering the rich.

- Napoleon Bonaparte -

[Marc de Bourget]

Thank you, you are right: Using arguments is better than changing the system date :-)

Just to illustrate the issue once more:
BEGIN {
print strftime("%c", 2147483647) # OK: => 19.01.2038 04:14:07
print strftime("%c", 2147483648) # WRONG: => NULL
}

I'm new to C programming, just experimenting a bit with the Bloodshed Dev-C++ Compiler. Of course the C language ctime() function has the same issue. If I can figure out how to compile GAWK with a 64bit C Compiler I'll do it.

[Kenny McCormack]

In article <7a942582-8459-415e...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:

...

>I'm new to C programming, just experimenting a bit with the Bloodshed Dev-C++
>Compiler. Of course the C language ctime() function has the same issue. If I can
>figure out how to compile GAWK with a 64bit C Compiler I'll do it.

For what it is worth, I'd recommend using Cygwin. I'm assuming (haven't
checked recently) that Cygwin will compile in 64-bit now. Anyway, Cygwin
has always been the best way to compile for Windows, for these reasons:

1) Easiest. Since bash is builtin and "just works", the Unix
instructions (./configure && make) "just work".

2) All the features "just work" like they do in Unix. I know they've
made strides in terms of making the Unix stuff work under Windows
native compilers, the fact remains that POLA will be with you the
best if you use Cygwin. In particular, I know, at least in the
beginning, the GAWK networking stuff didn't work in Windows-native
compiled versions.

--
To my knowledge, Jacob Navia is not a Christian.

- Rick C Hodgin -

[Marc de Bourget]

I have only basic knowledge in C but I think the actual problem is the time_t data type which uses a signed 32 bit integer or a 64 bit integer.

May it be possible to set it to a 64 bit integer by GAWK developers or does this fail due to compiler limits?

If this works, even 32 bit PCs are not limited to 32 bit boundaries. (My Delphi 32 bit programming environment also uses 64 bit integers for dates, see the UnixToDateTime function: http://docwiki.embarcadero.com/Libraries/Seattle/en/System.DateUtils.UnixToDateTime
This works properly for my 32 bit PC).

[Kenny McCormack]

In article <cf7c0d8b-02ec-4f7f...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:

>I have only basic knowledge in C but I think the actual problem is the time_t
>data type which uses a signed 32 bit integer or a 64 bit integer.

I'm actually curious as to why this is an issue for you at the present
time. (I'm not saying you're wrong if you think that it is; like I said,
I'm just curious)

I think most people assume that it won't really be an issue until it is
upon us, and that over the next 22 years, 64 bit machines will come to
dominate to the point where we will think of 32 bit machines then the same
way we think of 8 and 16 bit machines today.

Anyway, I think it would be easier to just compile it 64 bit than to muck
around with the source code in the way you suggest. The later is certainly
not impossible, but it does come with some risk.

--
For instance, Standard C says that nearly all extensions to C are prohibited. How
silly! GCC implements many extensions, some of which were later adopted as part of
the standard. If you want these constructs to give an error message as
“required” by the standard, you must specify ‘--pedantic’, which was
implemented only so that we can say “GCC is a 100% implementation of the
standard”, not because there is any reason to actually use it.

[Marc de Bourget]

OK, at the moment it is not an urgent problem but I generally don't like unnecessary limits. I have a 64 bit notebook and there are the same limits with these 32 bit binaries. Probably in 2038 no one has 32 bit PCs anymore but then these binaries won't work on 64 bit either. Maybe Eli Zaretskii can compile a 64 bit version from his source or advice how to do this, please? Then I could test if everything works fine on my 64 bit notebook. I can't use Cygwin because I use awk in batch files.

[Kenny McCormack]

In article <2a5e4044-c9f6-4c07...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:

...

>I can't use Cygwin because I use awk in batch files.

False.

You can certainly use Cygwin binaries from within DOS batch files.
I do it all the time.

(Yes, you use the Cygwin interface [i.e., bash] to *build* GAWK, but you can
then use the resulting binary just like normal)

--
- Since the shootings on Friday, the ultra-defensive [maybe wrongly
- hyphenated, but that would be fitting] Roy "News" LIEberman has posted
- at least 90 times, and almost every single post is about his obsessive
- knee-jerk loonball [wacko] gun politics. How much longer before the
- authorities [police] finally disable the trip wires and confiscate the
- arsenal in his St. Louie hovel?

So true. So true.

[Marc de Bourget]

OK. It will work but I don't like to install it because I always prefer native Windows programs if ever possible. This may be a matter of taste.

BTW, as you may know Microsoft has announced bash shell integration in Windows 10 later this year. Then, Cygwin may not be needed anyway (but this is another topic).

[Kaz Kylheku]

On 2016-04-25, Marc de Bourget <marcde...@gmail.com> wrote:
> I have only basic knowledge in C but I think the actual problem is the time_t data type which uses a signed 32 bit integer or a 64 bit integer.
>
> May it be possible to set it to a 64 bit integer by GAWK developers or
> does this fail due to compiler limits?

No, you can't just set time_t to whatever you want, and it won't help,
because the library functions for manipulating time are compiled with
a 32 bit time_t.

> If this works, even 32 bit PCs are not limited to 32 bit boundaries.
> (My Delphi 32 bit programming environment also uses 64 bit integers
> for dates, see the UnixToDateTime function:
> http://docwiki.embarcadero.com/Libraries/Seattle/en/System.DateUtils.UnixToDateTime
> This works properly for my 32 bit PC).

"PC's" are not affected by a 2038 year problem. Only C and C++ programs
that use a 32 bit time_t which is based on the "seconds since January 1,
1970" Unix representation, and some operating systems which use that
internally.

Windows doesn't use this representation at the operating system level.
The Windows API has something called FILETIME:

typedef struct _FILETIME {
DWORD dwLowDateTime;
DWORD dwHighDateTime;
} FILETIME;

This is a structure of two 32-bit words, so effectively a 64 bit
time. This counts in units of hundreds of nanoseconds (0.1
microseconds) since January 1, 1601, UTC.

A C compiler on Windows, and its library, could define time_t as a 64
bit type which maps to FILETIME; ISO C doesn't specify the details of
the time_t representation, only that it's an "arithmetic type
capable of representing times".

However, many C programs (especially ones ported from Unix) will
break if time_t doesn't measure seconds since January 1, 1970.
So for instance Microsoft's Visual C run time library defines time_t
in the Unix way.

Unix-like operating systems actually use time_t at a deeper level;
like in the gettimeofday system call and others. Traditional 32 bit
Unixes have a 2038 year problem affecting their kernels.

[Janis Papanagnou]

What do you mean by "bash shell integration in Windows"?

I recall a colleague having had a couple GNU commands as *.exe files
on Windows, including bash. So there actually seem to be GNU software
available as "native Windows programs".

Janis

[Kenny McCormack]

In article <nfmc5o$750$1...@news.m-online.net>,

Well, the story behind the thing to which Marc refers above is that, like
most recent Microsoft announcements, it is a) mostly hype/hot air and b) too
little, too late (at best).

As you say, there have long been various ports of GNU tools available on
the Windows platform and they haven't needed any help or cooperation from
MS to make them happen. Cygwin is the best and includes the most complete
tool collection, but that isn't really the point. The point of Cygwin is
not the ports of the GNU tools (although it has them), but rather the
compiler (a port of GCC) and, most importantly, a POSIX emulation layer
that makes it possible to compile C programs written for, say, Linux, and
have them run almost identically to how they run on a real Unix/Linux
system. That, my friends, is the point of Cygwin.

So Marc is mistaken above when he states that this latest hype announcement
from MS means that you don't need Cygwin. Even if the bash shell offering
from MS is anything useful (and most indications are that it won't be), it
doesn't provide either a compiler or a POSIX emulation layer.

--
Watching ConservaLoons playing with statistics and facts is like watching a
newborn play with a computer. Endlessly amusing, but totally unproductive.

[Janis Papanagnou]

On 26.04.2016 03:20, Kenny McCormack wrote:
> In article <nfmc5o$750$1...@news.m-online.net>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
>> On 25.04.2016 15:33, Marc de Bourget wrote:
>>> OK. It will work but I don't like to install it because I always prefer
>>> native Windows programs if ever possible. This may be a matter of taste.
>>>
>>> BTW, as you may know Microsoft has announced bash shell integration in
>>> Windows 10 later this year. Then, Cygwin may not be needed anyway (but this
>>> is another topic).
>>
>> What do you mean by "bash shell integration in Windows"?
>>

>> [...]

>
> Well, the story behind the thing to which Marc refers above is that, like
> most recent Microsoft announcements, it is a) mostly hype/hot air and b) too
> little, too late (at best).

I suspected that but wanted a confirmation to be sure. Thanks.

>
> As you say, there have long been various ports of GNU tools available on
> the Windows platform and they haven't needed any help or cooperation from
> MS to make them happen. Cygwin is the best and includes the most complete
> tool collection, but that isn't really the point. The point of Cygwin is
> not the ports of the GNU tools (although it has them), but rather the
> compiler (a port of GCC) and, most importantly, a POSIX emulation layer
> that makes it possible to compile C programs written for, say, Linux, and
> have them run almost identically to how they run on a real Unix/Linux
> system. That, my friends, is the point of Cygwin.

The abstraction layer is the reason why I'd prefer Cygwin (as opposed to
"native" tools; that's why I was a bit astonished, to say the least, why
the OP prefers "native" tools). But a whole Cygwin installation is, OTOH,
also an imposition; just recently made an installation from scratch, and
the setup tool is (still) unspeakable; *that* is a reason I'd understand
why one might abstain from Cygwin.

WRT gcc compiler; I thought (but may be misremembering) that the compiler
was natively available as well.

Janis

> [...]

[Aharon Robbins]

In article <nfmfp7$n1q$1...@news.xmission.com>,

Kenny McCormack <gaz...@shell.xmission.com> wrote:
>Even if the bash shell offering
>from MS is anything useful (and most indications are that it won't be), it
>doesn't provide either a compiler or a POSIX emulation layer.

Totally mistaken.

The new offering is called the "Linux Subsystem for Windows". It lets
you run native Linux binaries on Windows 10; you get them straight from
the Ubuntu repository. It includes a compiler - the regular Linux one,
and a POSIX emulation layer, by definition.

It is a very interesting development. It does not obsolete Cygwin,
especially if you're using any Windows system that isn't bleeding edge
Windows 10. From what I've heard from people using it, things mostly
work, but are not any faster than Cygwin. Also bear in mind that
MSFT themselves are calling it very preliminary.

Google for "Linux Subsystem for Windows" and also see some of the MSFT
videos about it. They're not hard to find.
--
Aharon (Arnold) Robbins arnold AT skeeve DOT com

[Marc de Bourget]

Sounds very promising. Will the most recent gawk version automatically be available in the bash shell for Windows 10?

[Aharon Robbins]

In article <fbe0b7c5-a9ae-44a6...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:

>Sounds very promising. Will the most recent gawk version automatically
>be available in the bash shell for Windows 10?

I don't really know. If they're using the Ubuntu 14.04 repos, then
no. But you can always compile it yourself. Ubuntu 16.04 ships with
gawk 4.1.3, which is the most recently released version.