Who maintains Tclhttpd?
EL
as I see, the last commits to Tclhttpd on sourceforge
(http://tclhttpd.sourceforge.net/) have been done 2 years ago.
The tracker contains several recent bugs (submitted 2007), but no
discussions, and nobody seems to take care about them... otherwise there
would probably be recent commits, or even a release since 2004.
Bug 1446208 (server is open for XXS attacks) is a very prominent and
scary example. Submitted by nobody on 2006/03/09, assigned to nobody and
nobody discussed. Nobody who submitted the bug had even provided a
"clumsy workaround", but nobody who was assigned to the bug has done
nothing (of course).
According to the project site, Brent B. Welch (who sadly seems to have
been disappeared) and Jeff Hobbs are the official project maintainers.
If Brent and Jeff don't have time or too many other assignments,
wouldn't it make sense to find another maintainer?
It would be sad, if this great piece of software would die...
--
Eckhard
George Peter Staplin
I think a lot of people have moved to Wub. The Tcler's Wiki is powered
by Wub now. The creator of Wub used to do more than a few commits to
tclhttpd.
I've been frustrated by the [gets] security bug. You can crash a
tclhttpd server (due to an out of memory condition) quite easily with:
puts -nonewline $sock [string repeat "GETSTHIS" $someLargeNumber]
You can also do it over a period of time, so long as you avoid the
timeout that tclhttpd uses.
Another bug was with Tcl's [fcopy]. I have noticed seemingly random
data truncation with tclhttpd that I believe (after studying the code
paths) was caused by fcopy. tclhttpd uses fcopy. fcopy was buggy, and
it only seems to have been fixed recently. We literally had multiple
bugs in the Tcl tracker for fcopy that went unfixed for years, until
Alexandre Ferrieux recently tracked them down and fixed them. Tcl 8.6
will have the changes.
The XXS potential attack seems to be rather serious, if I understand
what is being implied by that.
I think Jeff Hobbs and Brent Welch are just too busy with life and work
to do much of anything with tclhttpd. It has served its purpose. It's
probably still useful behind a good firewall, unexposed to the outer
WAN.
George
EL
> It has served its purpose. It's
> probably still useful behind a good firewall, unexposed to the outer
> WAN.
I don't think that this should be it's only purpose. Sure, the server
has a few flaws that have been discovered over the years, but it's worth
to track them down and fix them.
I am quite sure that this is possible. The software is written in a very
useful and good, modular style... even I understand the source code. It
should be easy to maintain and to squash out security holes and bugs. If
only someone did it.
--
Eckhard
Donal K. Fellows
> I don't think that this should be it's only purpose. Sure, the server
> has a few flaws that have been discovered over the years, but it's worth
> to track them down and fix them.
Are you volunteering? :-)
Donal.
blacksqr
project off to you.
EL
Yes, I would like to volunteer. However, it involves a bit of
responsibility and work ;-).
Given the other projects that I have, I could almost do all of them
together as a full time job. I have another full time job, so I don't
know how much time will be left for Tclhttpd.
But let's start simple: I would like to co work on the maintenance. When
I managed to fix some bugs, I would love to commit this to the
repository, or submit a patch to Jeff Hobbs for review and commit. As
far as I see he is the only one who is kindof involved in the project at
the moment.
I could also revamp the website http://tclhttpd.sourceforge.net/ and
make it look similar to the one of Tloona, if desired. I think this is
also important part of the maintenance (it's the resource that new and
returning users see probably first).
So, one step after the other I will maybe take the whole ownership...
but first let's see how it works out.
--
Eckhard
Sean Woods
a
a few patches for that uberlargequery bug and a new mechanism to hand
off
directUrls to objects.
I just assumed that everyone who used it had their own little bag of
tricks
they use. If someone can get me CVS access to the Sourceforge website
I'll
start feeding back my fixes.
--The Hypnotoad
EL
> Funny, I've been using Tclhttpd on public webservers for years. I have
> a
> a few patches for that uberlargequery bug and a new mechanism to hand
> off
> directUrls to objects.
Definitely interresting...
I have also done a little bit here and there:
- Most notably on httpd::session: I've worked out how to process .tml
templates in the (safe) session interpreter, rather than the global
interpreter. This makes it much easier to provide session specific
content, and additionally keeps the global interpreter clean from
unwanted side effects.
- A module for pooled Postgresql connections and an easy way to do
queries on configured databases: you just write "dbname select * from
table ..." in your code, whenever you need something from dbname, or
"dbname insert into ..." etc.
- A random image module: creates gd image from a random string, sends it
via a Url_Prefix, and allows for validation.
All things that may be useful for others too, but need some
documentation and especially a place to live (like the Tclhttpd SF page)
--
Eckhard
Ron Fox
RF
vitic
If I understand the problem correctly, it could probably be solved by
the new TCL 8.5 [chan pending] command, as in:
set data [gets $sock]
if {[chan pending $sock] > 1024} {
close $sock
}
> You can also do it over a period of time, so long as you avoid the
> timeout that tclhttpd uses.
>
> Another bug was with Tcl's [fcopy]. I have noticed seemingly random
> data truncation with tclhttpd that I believe (after studying the code
> paths) was caused by fcopy. tclhttpd uses fcopy. fcopy was buggy, and
> it only seems to have been fixed recently. We literally had multiple
> bugs in the Tcl tracker for fcopy that went unfixed for years, until
> Alexandre Ferrieux recently tracked them down and fixed them. Tcl 8.6
> will have the changes.
>
> The XXS potential attack seems to be rather serious, if I understand
> what is being implied by that.
There seem to be never ending new ways to exploit the XSS attack.
I would rather deal with it on the application level
(write safe code, reject bad data). Otherwise, IMHO, the server might
soon become bloated with all that security code.
Also, if I understand it correctly, most of the XSS security problems
happen on the client (browser) side.
---Victor
Jan Kandziora
>
> Another bug was with Tcl's [fcopy]. I have noticed seemingly random
> data truncation with tclhttpd that I believe (after studying the code
> paths) was caused by fcopy.
>
incorrectly because of wrong LF->CRLF translation. I have fallen into that
trap recently.
I've still using tclhttpd for my embedded device, though. Developed some
code for it (gzip compression) and worked around that and other bugs.
The "gets" bug is nasty, you are right...
Kind regards
Jan
EL
> Other was caused by having tclhttpd calculating the "Content-Width" header
> incorrectly because of wrong LF->CRLF translation.
...
> The "gets" bug is nasty, you are right...
Has anyone fixed that and can provide a patch? Jan, maybe you can?
--
Eckhard
Jeff Hobbs
Send me your SF id (by private email) and I can add you to the admin
crew for tclhttpd.
Jeff
Alexandre Ferrieux
> Jan Kandziora schrieb:
>
>
> > Other was caused by having tclhttpd calculating the "Content-Width" header
> > incorrectly because of wrong LF->CRLF translation.
>
Maybe you mean "Content-Length" ?
If yes, it is traced to tracker id 1929403:
https://sourceforge.net/tracker/index.php?func=detail&aid=1929403&group_id=10894&atid=110894
in this case, it is rather the http package's fault, doing a simple
[tell] on the channel after writing (hence subject to CRLF stuff)
instead of keeping track of actual counts. Thanks for adding any
insight there, I've gotten no answer to the questino "why" ?
-Alex
Jan Kandziora
>
> Maybe you mean "Content-Length" ?
>
> If yes, it is traced to tracker id 1929403:
>
>
https://sourceforge.net/tracker/index.php?func=detail&aid=1929403&group_id=10894&atid=110894
>
> in this case, it is rather the http package's fault, doing a simple
> [tell] on the channel after writing (hence subject to CRLF stuff)
> instead of keeping track of actual counts. Thanks for adding any
> insight there, I've gotten no answer to the questino "why" ?
>
I get truncation-errors only with error and authentication pages.
The culprit is (in version 3.5.1, around line 1560 in httpd.tcl)
---------
if [catch {
HdRespondHeader $sock text/html 1 [expr {[string length $message] + 4}]
$code
puts $sock ""
puts $sock $message
} err] {
Log $sock LostSocket $data(url) $err
}
Httpd_SockClose $sock $close
---------
This is clearly wrong, as it implies <puts $sock $message>, which is part of
the HTTP *data* will insert *exactly one* CRLF into the output. For the
above case, this is only true if $message has no \n (which is then
translated to CRLF!) inside. Second, it must be "string bytelength" instead
of "string length", as the output page may be encoded in utf-8 or other
multi-byte encoding.
Instead, it should read
---------
if [catch {
HttpdRespondHeader $sock text/html 1 [string bytelength $message] $code
puts $sock ""
fconfigure $sock -translation lf
puts -nonewline $sock $message
} err] {
Log $sock LostSocket $data(url) $err
}
Httpd_SockClose $sock $close
---------
Same with authentication pages (in version 3.5.1, around line 1882 in
httpd.tcl)
---------
HttpdRespondHeader $sock text/html $close [string length
$HttpdAuthorizationFormat] 401
puts $sock "Www-Authenticate: $type realm=\"$realm\" $additional"
puts $sock ""
puts -nonewline $sock $HttpdAuthorizationFormat
Httpd_SockClose $sock $close
---------
This should read
---------
HttpdRespondHeader $sock text/html $close [string bytelength
$HttpdAuthorizationFormat] 401
puts $sock "Www-Authenticate: $type realm=\"$realm\" $additional"
puts $sock ""
fconfigure $sock -translation lf
puts -nonewline $sock $HttpdAuthorizationFormat
Httpd_SockClose $sock $close
---------
If your error is only with these pages, too, we could patch the bug as above
and have one less.
Kind regards
Jan
Fredderic
vitic <vit...@gmail.com> wrote:
>> puts -nonewline $sock [string repeat "GETSTHIS" $someLargeNumber]
>
> If I understand the problem correctly, it could probably be solved by
> the new TCL 8.5 [chan pending] command, as in:
>
> set data [gets $sock]
> if {[chan pending $sock] > 1024} {
> close $sock
> }
Wouldn't that be the other way around...? Check [chan pending] and
THEN use [gets]? I presume [chan pending] would buffer up whatever
data is available...
Fredderic
Jan Kandziora
>
> Send me your SF id (by private email) and I can add you to the admin
> crew for tclhttpd.
>
some patches I developed over the last few years. Otherwise you could add
me up to the project, too, I'm willing to do some housekeeping on it.
Kind regards
Jan
Fredderic
Okay... I typed that way too quickly. In non-blocking mode, [gets]
will grab what's available, return an empty string because it can't
find an eol, and then [chan pending] can throw it out because of
over-buffering.
In blocking mode, you're still stuffed.
Fredderic
minu...@hotmail.fr
> My problem with wub? nonexistent documentation.
> RF
>
My problem witl tcl/tk 8.5? nonexistant book.
M
Alexandre Ferrieux
> Alexandre Ferrieux schrieb:
>
> > Maybe you mean "Content-Length" ?
>
> Yes. Damn' brain-typo.
>
> > If yes, it is traced to tracker id 1929403:
>
Oh, I think I overestimated the similarity :-)
Right, so we do have two completely independent size-mismatch
situations:
- the one you found and fixed in tclhttpd (please go to the
tracker and post your patch there).
- the one mentioned in 1929403, which is in the client package,
and occurs only with [http::geturl -channel] with a non-binary-
translation channel (the bad [tell] trick). It is in the tracker
already, we're just awaiting Jeff's reaction.
Sorry for the false hope ;-)
-Alex
George Peter Staplin
It sounds like the fconfigure probably is right.
I wonder if this is any better:
set utf8msg [encoding convertto utf-8 $message]
HttpdRespondHeader $sock text/html 1 [string length $utf8msg] $code
...
puts -nonewline $sock $utf8msg
The [string length $utf8msg] should be the same as [string bytelength
$message] as I understand it, and my testing confirms that with
multibyte characters.
With Tcl's internal encoding being very similar to utf-8 it may
not matter though.
George
vitic
<georgepsSPAMME...@xmission.com> wrote:
> Jan Kandziora wrote:
> > Alexandre Ferrieux schrieb:
>
> >> Maybe you mean "Content-Length" ?
>
> > Yes. Damn' brain-typo.
>
> >> If yes, it is traced to tracker id 1929403:
>
I my little server code (not tclhttpd), I also noticed the problem
with wrong Content-Lenght. As a quick solution I just default all text
data to utf-8 and no translation of newlines.
Otherwise, if you use character translation on Linux (from LF to
CRLF), the length of data changes at the time it is being sent out. As
a fix you could count all newlines in your data and add that number
(accounting for an extra byte of each CR being added to LF) to the
Content-Length. Since, in my case, I haven't seen any issues with the
web browsers not liking the LF without CR, I don't do any translation
at all (fconfigure -translation binary -encoding binary). But that
might not work on Mac (I am on Linux).
---Victor
---Victor
vitic
I am not sure as far as exact usage of [chan pending] but the
documentation specifically states that it can be used to prevent DOS
(Denial Of Service) attacks.
---Victor
Jeff Hobbs
wrote:
> On May 5, 10:10 am, Jan Kandziora <j...@gmx.de> wrote:
> > If your error is only with these pages, too, we could patch the bug as above
> > and have one less.
>
> situations:
> - the one mentioned in 1929403, which is in the client package,
> and occurs only with [http::geturl -channel] with a non-binary-
> translation channel (the bad [tell] trick). It is in the tracker
> already, we're just awaiting Jeff's reaction.
Ah, if you are waiting to hope that I'll be able to look at every bug
that gets assigned to me, you might look at just how many that is and
realize I can't get to them all. ;)
FWIW, I do vaguely recall some odd behaviors related to the mixing of
eol types when doing Tcl http stuff on Windows. This may well be
correctly identifying the issue ... just don't have time to
investigate myself this week.
Jeff
USCode
...
>
> Another bug was with Tcl's [fcopy]. I have noticed seemingly random
> data truncation with tclhttpd that I believe (after studying the code
> paths) was caused by fcopy. tclhttpd uses fcopy. fcopy was buggy, and
> it only seems to have been fixed recently. We literally had multiple
> bugs in the Tcl tracker for fcopy that went unfixed for years, until
> Alexandre Ferrieux recently tracked them down and fixed them. Tcl 8.6
> will have the changes.
Thanks for fixing that Alexandre! But why do we need to wait for 8.6 to
get the fix?
Thanks!
Jan Kandziora
>
> - the one you found and fixed in tclhttpd (please go to the
> tracker and post your patch there).
>
gzip patch. But ok, if we'll have a maintainer with more spare time for
this project than Jeff has, I will put the patch onto the tracker.
Kind regards
Jan
EL
Just wanted to let you know that I am one of the maintainers now. I will
start in this role step by step, get an overview and look at the patches
and bugs in the tracker. Then, again step by step, review and apply them
to the CVS head.
It would be good to have as many fixes as possible in the tracker ;).
But of course also as many discovered bugs as possible...
--
Eckhard
Alexandre Ferrieux
Hmm, I believe these 4 or 5 tiny patches, rendered industrial-strength
by Andreas, made it into 8.4.19 and upcoming 8.5.3 too.
-Alex
Jeff Hobbs
wrote:
That is correct - they should be in the released 8.4.19 (via source or
ActiveTcl on our site).
Jeff
Sean Woods
I've added my object module to the CVS tree. I'm working
on some example code the doesn't depend on my library of
Tao code. But essentially I jimmy a hook into the Direct
mechanism that tells it to call a special method in an
object to change Urls and Cgi lists into a command that
is executed.
[object] httpMarshallArguments [suffix] [cgiList]
Basically, because you can't count on an object having
a namespace or procs hanging off of it, I had to add
this extra little mechanism.
Some other things I have in my little toolkit is an
sqlite based session manager, an uploader that stores
files by md5 hash, and an autoconfigure mechanism for
virtual hosts. Basically my httpd.tcl looks for
subdirectories with a hosts.rc file (containing a list
of hostnames the virtual host responds to) accompanied
by a tclhttpd.rc file specific to the virtual hosts.