Who maintains Tclhttpd?

37 views
Skip to first unread message

EL

unread,
May 3, 2008, 6:20:48 AM5/3/08
to
Hi,

as I see, the last commits to Tclhttpd on sourceforge
(http://tclhttpd.sourceforge.net/) have been done 2 years ago.

The tracker contains several recent bugs (submitted 2007), but no
discussions, and nobody seems to take care about them... otherwise there
would probably be recent commits, or even a release since 2004.

Bug 1446208 (server is open for XXS attacks) is a very prominent and
scary example. Submitted by nobody on 2006/03/09, assigned to nobody and
nobody discussed. Nobody who submitted the bug had even provided a
"clumsy workaround", but nobody who was assigned to the bug has done
nothing (of course).

According to the project site, Brent B. Welch (who sadly seems to have
been disappeared) and Jeff Hobbs are the official project maintainers.
If Brent and Jeff don't have time or too many other assignments,
wouldn't it make sense to find another maintainer?

It would be sad, if this great piece of software would die...


--
Eckhard

George Peter Staplin

unread,
May 3, 2008, 12:11:43 PM5/3/08
to


I think a lot of people have moved to Wub. The Tcler's Wiki is powered
by Wub now. The creator of Wub used to do more than a few commits to
tclhttpd.

http://code.google.com/p/wub/

I've been frustrated by the [gets] security bug. You can crash a
tclhttpd server (due to an out of memory condition) quite easily with:

puts -nonewline $sock [string repeat "GETSTHIS" $someLargeNumber]

You can also do it over a period of time, so long as you avoid the
timeout that tclhttpd uses.

Another bug was with Tcl's [fcopy]. I have noticed seemingly random
data truncation with tclhttpd that I believe (after studying the code
paths) was caused by fcopy. tclhttpd uses fcopy. fcopy was buggy, and
it only seems to have been fixed recently. We literally had multiple
bugs in the Tcl tracker for fcopy that went unfixed for years, until
Alexandre Ferrieux recently tracked them down and fixed them. Tcl 8.6
will have the changes.

The XXS potential attack seems to be rather serious, if I understand
what is being implied by that.

I think Jeff Hobbs and Brent Welch are just too busy with life and work
to do much of anything with tclhttpd. It has served its purpose. It's
probably still useful behind a good firewall, unexposed to the outer
WAN.


George

EL

unread,
May 3, 2008, 12:40:40 PM5/3/08
to
George Peter Staplin schrieb:

> It has served its purpose. It's
> probably still useful behind a good firewall, unexposed to the outer
> WAN.

I don't think that this should be it's only purpose. Sure, the server
has a few flaws that have been discovered over the years, but it's worth
to track them down and fix them.

I am quite sure that this is possible. The software is written in a very
useful and good, modular style... even I understand the source code. It
should be easy to maintain and to squash out security holes and bugs. If
only someone did it.


--
Eckhard

Message has been deleted

Donal K. Fellows

unread,
May 3, 2008, 5:21:28 PM5/3/08
to
EL wrote:
> I don't think that this should be it's only purpose. Sure, the server
> has a few flaws that have been discovered over the years, but it's worth
> to track them down and fix them.

Are you volunteering? :-)

Donal.

blacksqr

unread,
May 3, 2008, 7:59:48 PM5/3/08
to
If you made a decent pitch to Jeff Hobbs, he might well hand the
project off to you.

EL

unread,
May 4, 2008, 7:26:54 AM5/4/08
to
Donal K. Fellows schrieb:

Yes, I would like to volunteer. However, it involves a bit of
responsibility and work ;-).
Given the other projects that I have, I could almost do all of them
together as a full time job. I have another full time job, so I don't
know how much time will be left for Tclhttpd.

But let's start simple: I would like to co work on the maintenance. When
I managed to fix some bugs, I would love to commit this to the
repository, or submit a patch to Jeff Hobbs for review and commit. As
far as I see he is the only one who is kindof involved in the project at
the moment.
I could also revamp the website http://tclhttpd.sourceforge.net/ and
make it look similar to the one of Tloona, if desired. I think this is
also important part of the maintenance (it's the resource that new and
returning users see probably first).

So, one step after the other I will maybe take the whole ownership...
but first let's see how it works out.


--
Eckhard

Sean Woods

unread,
May 4, 2008, 8:01:52 AM5/4/08
to
Funny, I've been using Tclhttpd on public webservers for years. I have
a
a few patches for that uberlargequery bug and a new mechanism to hand
off
directUrls to objects.

I just assumed that everyone who used it had their own little bag of
tricks
they use. If someone can get me CVS access to the Sourceforge website
I'll
start feeding back my fixes.

--The Hypnotoad

EL

unread,
May 4, 2008, 9:04:33 AM5/4/08
to
Sean Woods schrieb:

> Funny, I've been using Tclhttpd on public webservers for years. I have
> a
> a few patches for that uberlargequery bug and a new mechanism to hand
> off
> directUrls to objects.

Definitely interresting...


I have also done a little bit here and there:

- Most notably on httpd::session: I've worked out how to process .tml
templates in the (safe) session interpreter, rather than the global
interpreter. This makes it much easier to provide session specific
content, and additionally keeps the global interpreter clean from
unwanted side effects.

- A module for pooled Postgresql connections and an easy way to do
queries on configured databases: you just write "dbname select * from
table ..." in your code, whenever you need something from dbname, or
"dbname insert into ..." etc.

- A random image module: creates gd image from a random string, sends it
via a Url_Prefix, and allows for validation.

All things that may be useful for others too, but need some
documentation and especially a place to live (like the Tclhttpd SF page)


--
Eckhard

Ron Fox

unread,
May 4, 2008, 10:07:54 AM5/4/08
to
My problem with wub? nonexistent documentation.
RF

vitic

unread,
May 4, 2008, 1:27:57 PM5/4/08
to
On May 3, 9:11 am, George Peter Staplin

If I understand the problem correctly, it could probably be solved by
the new TCL 8.5 [chan pending] command, as in:

set data [gets $sock]
if {[chan pending $sock] > 1024} {
close $sock
}


> You can also do it over a period of time, so long as you avoid the
> timeout that tclhttpd uses.
>
> Another bug was with Tcl's [fcopy]. I have noticed seemingly random
> data truncation with tclhttpd that I believe (after studying the code
> paths) was caused by fcopy. tclhttpd uses fcopy. fcopy was buggy, and
> it only seems to have been fixed recently. We literally had multiple
> bugs in the Tcl tracker for fcopy that went unfixed for years, until
> Alexandre Ferrieux recently tracked them down and fixed them. Tcl 8.6
> will have the changes.
>
> The XXS potential attack seems to be rather serious, if I understand
> what is being implied by that.

There seem to be never ending new ways to exploit the XSS attack.
I would rather deal with it on the application level
(write safe code, reject bad data). Otherwise, IMHO, the server might
soon become bloated with all that security code.
Also, if I understand it correctly, most of the XSS security problems
happen on the client (browser) side.


---Victor

Jan Kandziora

unread,
May 4, 2008, 4:04:31 PM5/4/08
to
George Peter Staplin schrieb:

>
> Another bug was with Tcl's [fcopy]. I have noticed seemingly random
> data truncation with tclhttpd that I believe (after studying the code
> paths) was caused by fcopy.
>
Other was caused by having tclhttpd calculating the "Content-Width" header
incorrectly because of wrong LF->CRLF translation. I have fallen into that
trap recently.

I've still using tclhttpd for my embedded device, though. Developed some
code for it (gzip compression) and worked around that and other bugs.
The "gets" bug is nasty, you are right...

Kind regards

Jan

EL

unread,
May 4, 2008, 4:52:15 PM5/4/08
to
Jan Kandziora schrieb:

> Other was caused by having tclhttpd calculating the "Content-Width" header
> incorrectly because of wrong LF->CRLF translation.

...


> The "gets" bug is nasty, you are right...

Has anyone fixed that and can provide a patch? Jan, maybe you can?


--
Eckhard

Jeff Hobbs

unread,
May 4, 2008, 11:22:59 PM5/4/08
to

Send me your SF id (by private email) and I can add you to the admin
crew for tclhttpd.

Jeff

Alexandre Ferrieux

unread,
May 5, 2008, 2:43:12 AM5/5/08
to
On May 4, 10:52 pm, EL <eckhardnos...@gmx.de> wrote:
> Jan Kandziora schrieb:
>
>
> > Other was caused by having tclhttpd calculating the "Content-Width" header
> > incorrectly because of wrong LF->CRLF translation.
>
> Has anyone fixed that and can provide a patch? Jan, maybe you can?

Maybe you mean "Content-Length" ?
If yes, it is traced to tracker id 1929403:

https://sourceforge.net/tracker/index.php?func=detail&aid=1929403&group_id=10894&atid=110894

in this case, it is rather the http package's fault, doing a simple
[tell] on the channel after writing (hence subject to CRLF stuff)
instead of keeping track of actual counts. Thanks for adding any
insight there, I've gotten no answer to the questino "why" ?

-Alex

Jan Kandziora

unread,
May 5, 2008, 4:10:03 AM5/5/08
to
Alexandre Ferrieux schrieb:

>
> Maybe you mean "Content-Length" ?
>
Yes. Damn' brain-typo.


> If yes, it is traced to tracker id 1929403:
>
>
https://sourceforge.net/tracker/index.php?func=detail&aid=1929403&group_id=10894&atid=110894
>
> in this case, it is rather the http package's fault, doing a simple
> [tell] on the channel after writing (hence subject to CRLF stuff)
> instead of keeping track of actual counts. Thanks for adding any
> insight there, I've gotten no answer to the questino "why" ?
>

I get truncation-errors only with error and authentication pages.

The culprit is (in version 3.5.1, around line 1560 in httpd.tcl)
---------
if [catch {
HdRespondHeader $sock text/html 1 [expr {[string length $message] + 4}]
$code
puts $sock ""
puts $sock $message
} err] {
Log $sock LostSocket $data(url) $err
}
Httpd_SockClose $sock $close
---------
This is clearly wrong, as it implies <puts $sock $message>, which is part of
the HTTP *data* will insert *exactly one* CRLF into the output. For the
above case, this is only true if $message has no \n (which is then
translated to CRLF!) inside. Second, it must be "string bytelength" instead
of "string length", as the output page may be encoded in utf-8 or other
multi-byte encoding.

Instead, it should read
---------
if [catch {
HttpdRespondHeader $sock text/html 1 [string bytelength $message] $code
puts $sock ""
fconfigure $sock -translation lf
puts -nonewline $sock $message
} err] {
Log $sock LostSocket $data(url) $err
}
Httpd_SockClose $sock $close
---------


Same with authentication pages (in version 3.5.1, around line 1882 in
httpd.tcl)
---------
HttpdRespondHeader $sock text/html $close [string length
$HttpdAuthorizationFormat] 401
puts $sock "Www-Authenticate: $type realm=\"$realm\" $additional"
puts $sock ""
puts -nonewline $sock $HttpdAuthorizationFormat
Httpd_SockClose $sock $close
---------
This should read
---------
HttpdRespondHeader $sock text/html $close [string bytelength
$HttpdAuthorizationFormat] 401
puts $sock "Www-Authenticate: $type realm=\"$realm\" $additional"
puts $sock ""
fconfigure $sock -translation lf
puts -nonewline $sock $HttpdAuthorizationFormat
Httpd_SockClose $sock $close
---------


If your error is only with these pages, too, we could patch the bug as above
and have one less.

Kind regards

Jan

Fredderic

unread,
May 5, 2008, 4:09:37 AM5/5/08
to
On Sun, 4 May 2008 10:27:57 -0700 (PDT),
vitic <vit...@gmail.com> wrote:

>> puts -nonewline $sock [string repeat "GETSTHIS" $someLargeNumber]
>
> If I understand the problem correctly, it could probably be solved by
> the new TCL 8.5 [chan pending] command, as in:
>
> set data [gets $sock]
> if {[chan pending $sock] > 1024} {
> close $sock
> }

Wouldn't that be the other way around...? Check [chan pending] and
THEN use [gets]? I presume [chan pending] would buffer up whatever
data is available...


Fredderic

Jan Kandziora

unread,
May 5, 2008, 4:15:57 AM5/5/08
to
Jeff Hobbs schrieb:

>
> Send me your SF id (by private email) and I can add you to the admin
> crew for tclhttpd.
>
If Eckard is willing to do the boring maintainer job, i'd like to re-send in
some patches I developed over the last few years. Otherwise you could add
me up to the project, too, I'm willing to do some housekeeping on it.

Kind regards

Jan

Fredderic

unread,
May 5, 2008, 4:30:50 AM5/5/08
to

Okay... I typed that way too quickly. In non-blocking mode, [gets]
will grab what's available, return an empty string because it can't
find an eol, and then [chan pending] can throw it out because of
over-buffering.

In blocking mode, you're still stuffed.


Fredderic

minu...@hotmail.fr

unread,
May 5, 2008, 5:36:26 AM5/5/08
to
On 4 mai, 16:07, Ron Fox <f...@nscl.msu.edu> wrote:
> My problem with wub? nonexistent documentation.
> RF
>

My problem witl tcl/tk 8.5? nonexistant book.

M

Alexandre Ferrieux

unread,
May 5, 2008, 8:42:53 AM5/5/08
to
On May 5, 10:10 am, Jan Kandziora <j...@gmx.de> wrote:
> Alexandre Ferrieux schrieb:
>
> > Maybe you mean "Content-Length" ?
>
> Yes. Damn' brain-typo.
>
> > If yes, it is traced to tracker id 1929403:
>
> https://sourceforge.net/tracker/index.php?func=detail&aid=1929403&gro...

Oh, I think I overestimated the similarity :-)
Right, so we do have two completely independent size-mismatch
situations:

- the one you found and fixed in tclhttpd (please go to the
tracker and post your patch there).

- the one mentioned in 1929403, which is in the client package,
and occurs only with [http::geturl -channel] with a non-binary-
translation channel (the bad [tell] trick). It is in the tracker
already, we're just awaiting Jeff's reaction.

Sorry for the false hope ;-)

-Alex

George Peter Staplin

unread,
May 5, 2008, 8:58:27 AM5/5/08
to


It sounds like the fconfigure probably is right.

I wonder if this is any better:
set utf8msg [encoding convertto utf-8 $message]

HttpdRespondHeader $sock text/html 1 [string length $utf8msg] $code
...
puts -nonewline $sock $utf8msg


The [string length $utf8msg] should be the same as [string bytelength
$message] as I understand it, and my testing confirms that with
multibyte characters.

With Tcl's internal encoding being very similar to utf-8 it may
not matter though.


George

vitic

unread,
May 5, 2008, 2:21:18 PM5/5/08
to
On May 5, 5:58 am, George Peter Staplin

<georgepsSPAMME...@xmission.com> wrote:
> Jan Kandziora wrote:
> > Alexandre Ferrieux schrieb:
>
> >> Maybe you mean "Content-Length" ?
>
> > Yes. Damn' brain-typo.
>
> >> If yes, it is traced to tracker id 1929403:
>
> >https://sourceforge.net/tracker/index.php?func=detail&aid=1929403&gro...

I my little server code (not tclhttpd), I also noticed the problem
with wrong Content-Lenght. As a quick solution I just default all text
data to utf-8 and no translation of newlines.
Otherwise, if you use character translation on Linux (from LF to
CRLF), the length of data changes at the time it is being sent out. As
a fix you could count all newlines in your data and add that number
(accounting for an extra byte of each CR being added to LF) to the
Content-Length. Since, in my case, I haven't seen any issues with the
web browsers not liking the LF without CR, I don't do any translation
at all (fconfigure -translation binary -encoding binary). But that
might not work on Mac (I am on Linux).

---Victor

---Victor

vitic

unread,
May 5, 2008, 2:28:27 PM5/5/08
to

I am not sure as far as exact usage of [chan pending] but the
documentation specifically states that it can be used to prevent DOS
(Denial Of Service) attacks.

---Victor

Jeff Hobbs

unread,
May 5, 2008, 2:47:49 PM5/5/08
to
On May 5, 5:42 am, Alexandre Ferrieux <alexandre.ferri...@gmail.com>
wrote:

> On May 5, 10:10 am, Jan Kandziora <j...@gmx.de> wrote:
> > If your error is only with these pages, too, we could patch the bug as above
> > and have one less.
>
> Right, so we do have two completely independent size-mismatch
> situations:
...

>      - the one mentioned in 1929403, which is in the client package,
> and occurs only with [http::geturl -channel] with a non-binary-
> translation channel (the bad [tell] trick). It is in the tracker
> already, we're just awaiting Jeff's reaction.

Ah, if you are waiting to hope that I'll be able to look at every bug
that gets assigned to me, you might look at just how many that is and
realize I can't get to them all. ;)

FWIW, I do vaguely recall some odd behaviors related to the mixing of
eol types when doing Tcl http stuff on Windows. This may well be
correctly identifying the issue ... just don't have time to
investigate myself this week.

Jeff

USCode

unread,
May 5, 2008, 3:19:25 PM5/5/08
to
George Peter Staplin wrote:
...

>
> Another bug was with Tcl's [fcopy]. I have noticed seemingly random
> data truncation with tclhttpd that I believe (after studying the code
> paths) was caused by fcopy. tclhttpd uses fcopy. fcopy was buggy, and
> it only seems to have been fixed recently. We literally had multiple
> bugs in the Tcl tracker for fcopy that went unfixed for years, until
> Alexandre Ferrieux recently tracked them down and fixed them. Tcl 8.6
> will have the changes.
...
Thanks for fixing that Alexandre! But why do we need to wait for 8.6 to
get the fix?
Thanks!

Jan Kandziora

unread,
May 5, 2008, 4:19:38 PM5/5/08
to
Alexandre Ferrieux schrieb:

>
> - the one you found and fixed in tclhttpd (please go to the
> tracker and post your patch there).
>
I was a bit lazy with that in the past, as noone seemed interested on my
gzip patch. But ok, if we'll have a maintainer with more spare time for
this project than Jeff has, I will put the patch onto the tracker.

Kind regards

Jan

EL

unread,
May 5, 2008, 5:30:03 PM5/5/08
to
Jan Kandziora schrieb:

Just wanted to let you know that I am one of the maintainers now. I will
start in this role step by step, get an overview and look at the patches
and bugs in the tracker. Then, again step by step, review and apply them
to the CVS head.

It would be good to have as many fixes as possible in the tracker ;).
But of course also as many discovered bugs as possible...


--
Eckhard

Alexandre Ferrieux

unread,
May 5, 2008, 5:48:01 PM5/5/08
to

Hmm, I believe these 4 or 5 tiny patches, rendered industrial-strength
by Andreas, made it into 8.4.19 and upcoming 8.5.3 too.

-Alex

Jeff Hobbs

unread,
May 5, 2008, 5:52:26 PM5/5/08
to
On May 5, 2:48 pm, Alexandre Ferrieux <alexandre.ferri...@gmail.com>
wrote:

That is correct - they should be in the released 8.4.19 (via source or
ActiveTcl on our site).

Jeff

Sean Woods

unread,
May 6, 2008, 7:31:06 AM5/6/08
to

I've added my object module to the CVS tree. I'm working
on some example code the doesn't depend on my library of
Tao code. But essentially I jimmy a hook into the Direct
mechanism that tells it to call a special method in an
object to change Urls and Cgi lists into a command that
is executed.

[object] httpMarshallArguments [suffix] [cgiList]

Basically, because you can't count on an object having
a namespace or procs hanging off of it, I had to add
this extra little mechanism.

Some other things I have in my little toolkit is an
sqlite based session manager, an uploader that stores
files by md5 hash, and an autoconfigure mechanism for
virtual hosts. Basically my httpd.tcl looks for
subdirectories with a hosts.rc file (containing a list
of hostnames the virtual host responds to) accompanied
by a tclhttpd.rc file specific to the virtual hosts.

Reply all
Reply to author
Forward
0 new messages