[AOLSERVER] Fwd: [AOLSERVER] ns_httptime format error in AOL 4.5.1
Jeff Hobbs
clock code) about the state of 8.5 clock:
> Jeff Hobbs wrote:
>>> I haven't looked at Tcl 8.5 source, but has "clock ... -gmt" been
>>> fixed so that it doesn't diddle with env(TZ) any more, which isn't
>>> thread-safe. I remember that there's Tcl mutexes around the
>>> env(TZ) diddling, but that isn't safe when an application embeds
>>> Tcl and modifies env(TZ) as well but doesn't have access to the
>>> mutex that Tcl's "clock" is using.
>
> As of 8.5:
>
> [clock] reads $env(TZ) but no longer needs to modify it. The
> '-gmt' and '-timezone' flags are handled within the Tcl library.
> We don't use the system 'strftime' either. We use 'localtime'
> only as a last resort when we can't determine time zone any
> other way. And that includes using 'localtime_r' where
> appropriate. (Some systems have a thread-safe 'localtime' that
> returns its structure in thread-local storage.)
>
> So you should be safe.
>
> --
> 73 de ke9tv/2, Kevin
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <list...@listserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Tom Jackson
Alexey: you can use the nstclsh shell to test if changing the locale
makes any difference (in same location as your nsd file):
./bin/nstclsh
% set ::env(LANG) en_US.UTF-8
% ns_httptime 0
Thu, 01 Jan 1970 00:00:00 GMT
%
tom jackson
Unfortunately I only have english language locales on my computer, so I
can't test if the locale is causing the problem.
tom jackson
Alexey Pechnikov
On Sunday 10 May 2009 21:15:03 Tom Jackson wrote:
> Alexey: you can use the nstclsh shell to test if changing the locale
> makes any difference (in same location as your nsd file):
>
> ./bin/nstclsh
> % set ::env(LANG) en_US.UTF-8
> % ns_httptime 0
> Thu, 01 Jan 1970 00:00:00 GMT
> %
I'm get wrong result:
$ nstclsh
% set ::env(LANG) en_US.UTF-8
en_US.UTF-8
% ns_httptime 0
Thu, 1 Jan 1970 00:00:00 GMT
%
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
Tom Jackson
> Hello!
>
> On Sunday 10 May 2009 21:15:03 Tom Jackson wrote:
> > Alexey: you can use the nstclsh shell to test if changing the locale
> > makes any difference (in same location as your nsd file):
> >
> > ./bin/nstclsh
> > % set ::env(LANG) en_US.UTF-8
> > % ns_httptime 0
> > Thu, 01 Jan 1970 00:00:00 GMT
> > %
>
> I'm get wrong result:
>
> $ nstclsh
> % set ::env(LANG) en_US.UTF-8
> en_US.UTF-8
> % ns_httptime 0
> Thu, 1 Jan 1970 00:00:00 GMT
> %
I wonder if you have to set this before your start nstclsh?
Anyway, whatever is causing the problem has to do with your setup. At
least you can test it with nstclsh and once solved there should work for
your AOLserver install.
You might also see what my replacement proc produces, or just try in
nstclsh:
% set seconds 0
0
% clock format $seconds -format "%a, %d %b %Y %H:%M:%S %Z" -gmt 1
If that works/doesn't you might also try whatever tclsh came with your
system.
But I just noticed something strange:
% ns_parsehttptime [ns_httptime 0]
invalid time: Thu, 01 Jan 1970 00:00:00 GMT
% ns_parsehttptime "Thu, 01 Jan 1970 00:00:00 GMT"
invalid time: Thu, 01 Jan 1970 00:00:00 GMT
% ns_parsehttptime "Thu, 1 Jan 1970 00:00:00 GMT"
invalid time: Thu, 1 JAn 1970 00:00:00 GMT
On the last one, notice that the month gets converted to JAn.
Strange.
tom jackson
Alexey Pechnikov
On Sunday 10 May 2009 23:42:08 Tom Jackson wrote:
> I wonder if you have to set this before your start nstclsh?
$ LANG="en_US.UTF-8" nstclsh
% ns_httptime 0
Thu, 1 Jan 1970 00:00:00 GMT
> You might also see what my replacement proc produces, or just try in
> nstclsh:
$ nstclsh
% set seconds 0
0
% clock format $seconds -format "%a, %d %b %Y %H:%M:%S %Z" -gmt 3
Thu, 01 Jan 1970 00:00:00 GMT
> But I just noticed something strange:
>
> % ns_parsehttptime [ns_httptime 0]
> invalid time: Thu, 01 Jan 1970 00:00:00 GMT
> % ns_parsehttptime "Thu, 01 Jan 1970 00:00:00 GMT"
> invalid time: Thu, 01 Jan 1970 00:00:00 GMT
> % ns_parsehttptime "Thu, 1 Jan 1970 00:00:00 GMT"
> invalid time: Thu, 1 JAn 1970 00:00:00 GMT
>
> On the last one, notice that the month gets converted to JAn.
Hm, I'm get the same result for my ru_RU.UTF-8 locale.
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
Gustaf Neumann
From the output, i start to believe that the setting of
LANG/LOCALE is not the problem, but i am getting the
suspicion that there is a bug in the c-library of your
operating system. What operating system do you use?
Rationale: aolserver uses strftime(..., ..., "... %d ...", ...)
to format the day of month. Since strftime() is defined
in the C-library, the bug should be there.
-gustaf neumann
Alexey Pechnikov schrieb:
Tom Jackson
that the AOLserver C code passes in the format string, which is clearly
correct. %d is the zero padded day number [01-31]. After the call, the
modified buffer is sent back as-is, so the bug isn't in AOLserver.
The good news, if there is any here is that to set the time formats, you
only need to adjust the LC_TIME database, using localedef or locale-gen.
Here is a somewhat helpful link:
http://www.regatta.cs.msu.su/doc/usr/share/man/info/ru_RU/a_doc_lib/files/aixfiles/LC_TIME.htm
Aside from this issue, the code has obvious limitations built into the
design, at least ns_parsehttptime. For some reason, ns_parsehttptime
uses a return value of 0 to indicate failure. Probably a value of NULL
should have been used instead. However, this will only be a problem if
we need to maintain documents created in 1970.
The other issue is that ns_parsehttptime, which is based upon RFC 1123,
should be somewhat forgiving in parsing the time components. It clearly
fails when the day number is 1-9 with a leading blank space. Probably
the MakeNum API helper should be modified to test the result ranges for
each number, and somehow provide feedback on how much buffer was
consumed.
tom jackson
Alexey Pechnikov
On Monday 11 May 2009 11:02:09 Gustaf Neumann wrote:
> What operating system do you use?
My operation system is debian lenny.
Tom Jackson
> Hello!
>
> On Monday 11 May 2009 11:02:09 Gustaf Neumann wrote:
> > What operating system do you use?
>
> My operation system is debian lenny.
Okay, I have another idea: try python, which probably uses the platform
API where we think the bug exists:
$ python
>>> from time import gmtime, strftime
>>> strftime("%a, %d %b %Y %H:%M:%S GMT", gmtime(0))
'Thu, 01 Jan 1970 00:00:00 GMT'
>>>
tom jackson
Gustaf Neumann
> My operation system is debian lenny.
>
To pin down the problem, compile the progam below and check
the output. It should show the same bug on your system.
I don't have a lenny system around to test, but at least in Ubuntu
8.0.4, everything works as expected.
I don't see, how the locale settings can influence the %d formatting
of stftime. But anyhow, it might be worth to check the output of
locale -k ru_RU.UTF-8 LC_TIME
on your system
hope, we are getting closer
-gustaf neumann
================== time-format.c
#include <time.h>
#include <stdio.h>
int main() {
time_t clock = 0;
struct tm time, *tmPtr=&time;
char buf[40];
tmPtr = gmtime_r(&clock,tmPtr);
strftime(buf, 40, "%a, %d %b %Y %H:%M:%S GMT", tmPtr);
fprintf(stderr,"%s\n",buf);
Alexey Pechnikov
On Tuesday 12 May 2009 01:45:13 Tom Jackson wrote:
> Okay, I have another idea: try python, which probably uses the platform
> API where we think the bug exists:
>
> $ python
> >>> from time import gmtime, strftime
> >>> strftime("%a, %d %b %Y %H:%M:%S GMT", gmtime(0))
> 'Thu, 01 Jan 1970 00:00:00 GMT'
> >>>
This work correct:
$ python
Python 2.5.2 (r252:60911, Jan 4 2009, 17:40:26)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from time import gmtime, strftime
>>> strftime("%a, %d %b %Y %H:%M:%S GMT", gmtime(0))
'Thu, 01 Jan 1970 00:00:00 GMT'
>>>
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
Alexey Pechnikov
On Tuesday 12 May 2009 06:20:57 Gustaf Neumann wrote:
> I don't see, how the locale settings can influence the %d formatting
> of stftime. But anyhow, it might be worth to check the output of
>
> locale -k ru_RU.UTF-8 LC_TIME
>
> on your system
$ locale -k ru_RU.UTF-8 LC_TIME
abday="Вск;Пнд;Втр;Срд;Чтв;Птн;Сбт"
day="Воскресенье;Понедельник;Вторник;Среда;Четверг;Пятница;Суббота"
abmon="Янв;Фев;Мар;Апр;Май;Июн;Июл;Авг;Сен;Окт;Ноя;Дек"
mon="Январь;Февраль;Март;Апрель;Май;Июнь;Июль;Август;Сентябрь;Октябрь;Ноябрь;Декабрь"
am_pm=";"
d_t_fmt="%a %d %b %Y %T"
d_fmt="%d.%m.%Y"
t_fmt="%T"
t_fmt_ampm=""
era=
era_year=""
era_d_fmt=""
alt_digits=
era_d_t_fmt=""
era_t_fmt=""
time-era-num-entries=0
time-era-entries=""
week-ndays=7
week-1stday=19971130
week-1stweek=0
first_weekday=2
first_workday=2
cal_direction=1
timezone=""
date_fmt="%a %b %e %H:%M:%S %Z %Y"
time-codeset="UTF-8"
> hope, we are getting closer
> -gustaf neumann
>
> ================== time-format.c
> #include <time.h>
> #include <stdio.h>
>
> int main() {
> time_t clock = 0;
> struct tm time, *tmPtr=&time;
> char buf[40];
>
> tmPtr = gmtime_r(&clock,tmPtr);
> strftime(buf, 40, "%a, %d %b %Y %H:%M:%S GMT", tmPtr);
> fprintf(stderr,"%s\n",buf);
> }
$ gcc /tmp/time-format.c -o /tmp/time-format
$ /tmp/time-format
Thu, 01 Jan 1970 00:00:00 GMT
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
Gustaf Neumann
What is the output, if you call time-format from aolserver (via exec)
(this way, the external program runs with the same set of
environment variables as the aolserver).
-gustaf neumann
Alexey Pechnikov schrieb:
Alexey Pechnikov
On Tuesday 12 May 2009 12:51:02 Gustaf Neumann wrote:
> Very wierd. So, the behavior must be influenced by the environment.
> What is the output, if you call time-format from aolserver (via exec)
> (this way, the external program runs with the same set of
> environment variables as the aolserver).
>
This return the correct result:
catch {exec /tmp/time-format} msg
ns_log Notice $msg
[12/May/2009:14:56:47][15868.3082901168][-main-] Notice: Thu, 01 Jan 1970 00:00:00 GMT
Gustaf Neumann
We know at least, that the environment is indeed
not the problem. Can you check, whether both programs
use the same strftime function?
such as:
$ nm time-format| fgrep strftime
U strftime@@GLIBC_2.2.5
$ nm /usr/local/src/aolserver-4.5.1/nsd/libnsd.so | fgrep strf
U strftime@@GLIBC_2.2.5
-gustaf neumann
Alexey Pechnikov schrieb:
> This return the correct result:
>
> catch {exec /tmp/time-format} msg
> ns_log Notice $msg
>
> [12/May/2009:14:56:47][15868.3082901168][-main-] Notice: Thu, 01 Jan 1970 00:00:00 GMT
>
> Best regards, Alexey Pechnikov.
> http://pechnikov.tel/
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <list...@listserv.aol.com> with the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
>
--
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA
Alexey Pechnikov
On Tuesday 12 May 2009 17:29:38 Gustaf Neumann wrote:
> Mr Spock would say: fascinating.
>
> We know at least, that the environment is indeed
> not the problem. Can you check, whether both programs
> use the same strftime function?
>
> such as:
>
> $ nm time-format| fgrep strftime
> U strftime@@GLIBC_2.2.5
>
> $ nm /usr/local/src/aolserver-4.5.1/nsd/libnsd.so | fgrep strf
> U strftime@@GLIBC_2.2.5
>
I didn't build AOL with debugging symbols.
Tom Jackson
1. The format passed to sprintf is wrong, or
2. the case 'd' branch could be missing the continue, so that case 'e'
runs, and replaces the result of 'd':
case 'd':
pt = _conv(t->tm_mday, "%02d", pt, ptlim);
continue;
case 'e':
pt = _conv(t->tm_mday, "%2d", pt, ptlim);
continue;
Note that _conv is simply:
static char *_conv(const int n, const char *format, char *pt, const char *ptlim)
{
char buf[32];
sprintf(buf, format, n);
return _add(buf, pt, ptlim);
}
Since other formats rely on %d, you might also look at the results of
formatting %x, %D and %F. But you would have to edit httptime.c with a
new format string:
strftime(buf, 40, "%x", tmPtr);
tom jackson
Alexey Pechnikov
I'm sorry but the problem is produced by patch for non-english locale.
This patch use wrong format string:
snprintf(buf, 40, "%s, %d %s %d %02d:%02d:%02d GMT",
I did fix the bug and I'm think this patch may be added to upstream.
Patch is here:
http://mobigroup.ru/files/aol4.5.1/httptime.c.diff
Thanks for help!
=====================
--- httptime.c.orig 2003-01-18 22:24:20.000000000 +0300
+++ httptime.c 2009-05-12 20:14:24.000000000 +0400
@@ -27,6 +27,9 @@
* version of this file under either the License or the GPL.
*/
+static char *weekdays_names[7] =
+{ "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" };
+
/*
* time.c --
@@ -92,11 +95,14 @@
}
/*
- * This will most likely break if the locale is not an english one.
+ * Using snprintf instead of strftime to always use english names
* The format is RFC 1123: "Sun, 06 Nov 1997 09:12:45 GMT"
*/
- strftime(buf, 40, "%a, %d %b %Y %H:%M:%S GMT", tmPtr);
+ snprintf(buf, 40, "%s, %02d %s %d %02d:%02d:%02d GMT",
+ weekdays_names[tmPtr->tm_wday], tmPtr->tm_mday,
+ month_names[tmPtr->tm_mon], tmPtr->tm_year + 1900,
+ tmPtr->tm_hour, tmPtr->tm_min, tmPtr->tm_sec);
Ns_DStringAppend(pds, buf);
return pds->string;
Tom Jackson
particular case to avoid strftime, since all of the variability in the
format is removed. It is huge overkill, and obviously broken in
non-english languages.
tom jackson
Tom Jackson
>
> I'm sorry but the problem is produced by patch for non-english locale.
> This patch use wrong format string:
> snprintf(buf, 40, "%s, %d %s %d %02d:%02d:%02d GMT",
>
> I did fix the bug and I'm think this patch may be added to upstream.
At first I thought that you were fixing a bad patch. But the diff isn't
a fix of the bug, it is a good replacement for stupidly-generic code
(which we still are using in AOLserver).
We are still left with the question of how the original problem was
created. From my research it appears that the locale would never affect
the numbers (it couldn't be wrong with the day format and right with the
time format), so we could change the locale with no effect on the
output. We have evidence that the locale had no effect.
My best guess is that someone created an almost perfect patch to the
code you were using. It fixed the day-name and month-name problems, but
it screwed up the day-number format. But your patch does not record the
bug, just the original code, which would not produce the output we have
been using.
What gives?
You include the buggy snprintf line above, but this isn't in the patch.
tom jackson
Alexey Pechnikov
On Wednesday 13 May 2009 03:03:07 Tom Jackson wrote:
> You include the buggy snprintf line above, but this isn't in the patch.
Debian package has _wrong_ locale-dependence fixing patch:
=============================
--- aolserver4-4.5.1.orig/nsd/httptime.c
+++ aolserver4-4.5.1/nsd/httptime.c
@@ -27,6 +27,9 @@
* version of this file under either the License or the GPL.
*/
+static char *weekdays_names[7] =
+{ "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" };
+
/*
* time.c --
@@ -92,11 +95,14 @@
}
/*
- * This will most likely break if the locale is not an english one.
+ * Using snprintf instead of strftime to always use english names
* The format is RFC 1123: "Sun, 06 Nov 1997 09:12:45 GMT"
*/
- strftime(buf, 40, "%a, %d %b %Y %H:%M:%S GMT", tmPtr);
+ snprintf(buf, 40, "%s, %d %s %d %02d:%02d:%02d GMT",
+ weekdays_names[tmPtr->tm_wday], tmPtr->tm_mday,
+ month_names[tmPtr->tm_mon], tmPtr->tm_year + 1900,
+ tmPtr->tm_hour, tmPtr->tm_min, tmPtr->tm_sec);
Ns_DStringAppend(pds, buf);
return pds->string;
=============================
This patch has bug in line
snprintf(buf, 40, "%s, %d %s %d %02d:%02d:%02d GMT",
I used this patch because my locale is non-english one but I stupidly
forgot about this.... In my previous mail is attached the fixed patch:
http://mobigroup.ru/files/aol4.5.1/httptime.c.diff
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
Gustaf Neumann
Glad the problem is resolved. i commited a slightly
modified version of the patch (moved the static definition
to the right place in the file, made sure that year is always
4 digits) to CVS head on sourceforge. The patch makes
perfectly sense for non en* locale settings.
It would have saved all of us time if you would have
told us earlier that you are using a patched version.
I have checked the debian patches, there are no
other .c or .h files in the change set, so hopefully
this source of confusion is closed.
Many thanks for your help.
best regards
-gustaf neumann
PS: Tom, can you commit your improvements to queue.c
as well to cvs head?
Alexey Pechnikov schrieb: