[AOLSERVER] recent updates
Jeff Rogers
For those of you who don't monitor the commits list I wanted to share a
few changes I've recently made as well as a few I'm still thinking about.
- implemented native decoding of strings in ns_returnfile. This allows
filenames that are not utf-8 to be passed, similar to tcl core functions.
- committed a long-outstanding patch to address a fastpath caching bug
where incorrect file content could be delivered in a system when inodes
are rapidly being reused. Added a new config option "cacheminage" to
control this behavior.
- implemented a suggestion to re-run pre-auth and post-auth filters on
internal redirects. As this could break assumptions in unusual
situations, there is a config option "filterredirect" to disable it.
Among other things, this addresses an issue in openacs that made it
complicated to handle 404 errors. Also fixed a server crash when there
is an error in the configured error page.
- added flags to insert a filter at the beginning of the list of
registered filters instead of only at the end, and to order filters by
priority.
- exposed the gzip flag to tcl scripts (ns_conn gzip)
- added another new filter, "pre-write" which runs after content has
been returned but before it has been sent over the network. To make
this useful, there is a new method to ns_conn to examine and set the
response content that is about to be sent. This could be useful for
implementing something like apache's mod_pagespeed, which rewrites
content to do things like merge javascript and css files to reduce
external requests, or just strip extraneous whitespace from a page.
- added a tuning knob "maxaccept" to let the driver accept more than one
connection per spin. The SF RFE for this referenced a paper that found
this type of change could improve performance and reduce dropped
connections under load; I unfortunately don't have a network capable of
simulating enough load to stress this so I cannot claim any measured
improvement.
I also have a few other ideas I haven't gotten around to adding (or even
necessarily thinking through completely) yet:
- removing encoding and gzip compression from the normal response
writing codepath (Ns_ConnFlush) and making them into pre-write filters.
- adding an api to name C-coded filters and handler procs such that they
could be used by the tcl ns_register_filter/ns_register_proc commands.
This could eliminate a few special-purpose commands that do this same
thing for particular operations (ns_register_fastpath and
ns_register_filter_shortcut).
- add support for chunked encoding for requests, digest authentication,
and Expect: 100-continue. I've heard that these are already supported
in NaviServer, so this may be a fairly simple cut&paste enhancement.
- replace c-coded versions of many file operations with equivalent
tcl-coded versions. Many of these functions (e.g., ns_mkdir, ns_cp)
come from before tcl had equivalent functionality, but tcl now has that
functionality, and it's better (native decoding).
- replace ns_share with tcl version built around nsv, if not eliminating
it entirely.
- remove other really old code like tclx keylist support. I honestly
don't know if this is still being used, perhaps it could be kept around
in a "nscompat" module along with other things like ns_var and
ns_share). I'm a bit reluctant to break compatibility without an easy
switch to un-break it.
- make content sending event based, by pushing it back to the driver
thread, a task thread, or a completely new writer thread, rather than
blocking a conn thread if the content is big and the network is slow.
There is support for doing this at the tcl level, but it would be nice
to have it implicitly happen everywhere.
- let queue-waits be set up from a conn thread and the conn re-queued in
the driver thread. This would let you do database operations before
making network calls and still wait for those calls to complete without
tying up conn threads. This would mean a rather esoteric coding style
but I think it would make for a really powerful mashup engine.
Some of these are pretty disruptive (especially the last two), and
introducing incompatibilities (even ones with an easy off switch) is not
something to be taken lightly; does anyone else think a new 4.5.2
release with the recent bugfixes and non-disruptive new features could
be done, and then slightly disruptive stuff (i.e., moving c-coded
commands to tcl) could be targeted at 4.6?
Naturally, I'm also interested in any feedback on anything I've done or
suggested also.
Cheers,
-J
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <list...@listserv.aol.com> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Fenton, Brian
congrats on all the great work. That all sounds super. A new AOLserver release would be amazing, but maybe seeing as 4.0 was first released 8 years ago, we could consider that it's about time for a move to 5.0? Not wanting to get too Firefox-y about it, but I think it would be good for the project.
best wishes
Brian
________________________________________
From: AOLserver Discussion [AOLS...@LISTSERV.AOL.COM] On Behalf Of Jeff Rogers [dv...@DIPHI.COM]
Sent: 19 October 2011 06:45
To: AOLS...@LISTSERV.AOL.COM
Subject: [AOLSERVER] recent updates
Brett Schwarz
From: Jeff Rogers <dv...@DIPHI.COM>
To: AOLS...@LISTSERV.AOL.COM
Sent: Tuesday, October 18, 2011 10:45 PM
Subject: [AOLSERVER] recent updates
Andrew Piskorski
> A new AOLserver release would be amazing, but maybe seeing as 4.0
> was first released 8 years ago, we could consider that it's about
> time for a move to 5.0?
I guess he should just bump the version number up to 9.0 then, that'd
look really swell!
When Jeff releases a new version he should pick the version number
based on the usual proper criteria; how extensive the changes or new
features are, whether backwardness compatibility is preserved or
broken, etc. How many years the codebase sat there unchanged is
irrelevant.
--
Andrew Piskorski <a...@piskorski.com>
http://www.piskorski.com/
Rusty Brooks
> On Wed, Oct 19, 2011 at 05:09:48PM +0100, Fenton, Brian wrote:
>
>> A new AOLserver release would be amazing, but maybe seeing as 4.0
>> was first released 8 years ago, we could consider that it's about
>> time for a move to 5.0?
>
> I guess he should just bump the version number up to 9.0 then, that'd
> look really swell!
Also known as the firefox version numbering method.
Jeff Rogers
> Hi Jeff
>
> congrats on all the great work. That all sounds super. A new
> AOLserver release would be amazing, but maybe seeing as 4.0 was first
> released 8 years ago, we could consider that it's about time for a
> move to 5.0? Not wanting to get too Firefox-y about it, but I think
> it would be good for the project.
I think that any new release would be benificial; but I'm a bit
old-school about version numbers. The hairs on the back of my neck
stand up every time I get the monthly popup message for a new major
version of FireFox.
Major versions = can break functionality in a non-backwards compatible way
Minor versions = can have fairly sizable changes and new features, but
things can generally be expected to keep working
Point versions = small new features, bugfixes, no incompatible changes
I realize at least one of my changes can violate that, I'm hoping it's
enough of an edge case to slide.
IMO, a 5.0 release is a chance to incorporate major changes, such as to
the way that interps are iinitialized (there's some ideas from Jim on
this somwhere). The stuff I've done so far I think is still in the
point release realm.
Jeff Rogers
> I'll grab the latest and greatest from SF and start messing around with
> your changes. Is SF the main repo for aolserver? What about the modules
> (specifically nspostgres)?
Yep, SF is still the place for aolserver and nspostgres (which looks
like it has been updated to work with 9.x).
> RE: TclX...can't that be loaded as an extension, so that those who still
> need it can load it?
Good question .. from what I can gather from old threads the main users
of this was internal AOL users who are probably all gone now, or at
least not upgrading any longer. So this might be a pretty unimportant
thing to maintain compatibility on from several angles.
-J
>
> Thanks for your effort.
> --brett
>
Jim Davidson
Cool! Nice updates :)
On the version # question a few days back, I agree this is 4.x update. For a 5.x release, in addition to what you listed below, I'd add:
-- integrate SSL support directly (comm driver, api's)
-- integrate the comm drivers
-- figure out some better build environment for windows
-- ensure 64bit works everywhere
-- finally re-factor the Tcl management
-- make aolserver really just a loaded Tcl module
The last two would be disruptive but interesting or maybe just dumb on closer inspection. Basically, aolserver could use the ns_itcl stuff along with Tcl package to better leverage existing extensions. As a module overall, the basic startup and event loop stuff could be exposed as commands instead of special case startup code in a custom binary. I think that could get aolserver/tcl closer to command-line PHP type flexibility which enables, e.g., "crush" in drupal. May be too much effort but something to explore.
-Jim
Daniël Mantione
Op Tue, 25 Oct 2011, schreef Jim Davidson:
> Hi,
>
> Cool! Nice updates :)
>
> On the version # question a few days back, I agree this is 4.x update. For a 5.x release, in addition to what you listed below, I'd add:
>
> -- integrate SSL support directly (comm driver, api's)
> -- integrate the comm drivers
> -- figure out some better build environment for windows
> -- ensure 64bit works everywhere
> -- finally re-factor the Tcl management
> -- make aolserver really just a loaded Tcl module
>
> The last two would be disruptive but interesting or maybe just dumb on closer inspection. Basically, aolserver could use the ns_itcl stuff along with Tcl package to better leverage existing extensions. As a module overall, the basic startup and event loop stuff could be exposed as commands instead of special case startup code in a custom binary. I think that could get aolserver/tcl closer to command-line PHP type flexibility which enables, e.g., "crush" in drupal. May be too much effort but something to explore.
Maybe I could point again at the ipv6 patch I wrote in 2008. Back there
was even not even a response on this list, but maybe today, now the world
has run out of ipv4 addresses, the interrest in serving ipv6 is a bit
higher.
Patch still available for download here:
http://www.freepascal.org/~daniel/aolserver-ipv6support.diff
Best regards,
Dani�l Mantione
Jim Davidson
-Jim
Jeff Rogers
> Maybe I could point again at the ipv6 patch I wrote in 2008. Back there
> was even not even a response on this list, but maybe today, now the
> world has run out of ipv4 addresses, the interrest in serving ipv6 is a
> bit higher.
>
> Patch still available for download here:
>
> http://www.freepascal.org/~daniel/aolserver-ipv6support.diff
>
> Best regards,
>
> Dani�l Mantione
I'll take a swing at getting this integrated sometime soon (available
time waxes and wanes, as I'm sure it does for all of us). This patch is
a bigger piece of functionality that the others I put in; I think 4.6 is
the right target for this.
I think now is the right time to mark a 4.5.2 cvs branch; I'll try to
remember how to do that and tag it.
Sorry if I missed this (now as then); I was mostly going off the SF
RFEs looking for stuff to add. Actually, that's another thing I'd like
to see happen tho I can't do it - clean up the bug reports and RFEs.
There's a bunch of stuff sitting around from 2004 or earlier that's no
longer relevant and much as I dislike closing bugs as "old version" I
think that's what's needed for many of them.
-J
Daniël Mantione
Op Tue, 25 Oct 2011, schreef Jeff Rogers:
> I'll take a swing at getting this integrated sometime soon (available time
> waxes and wanes, as I'm sure it does for all of us). This patch is a bigger
> piece of functionality that the others I put in; I think 4.6 is the right
> target for this.
Yup, after the patch all ip-adresses internally in the webserver are ipv6
addresses, so it's really doing brain surgery :) On the other hand, I was
surprised the patch is this small.
> I think now is the right time to mark a 4.5.2 cvs branch; I'll try to
> remember how to do that and tag it.
>
> Sorry if I missed this (now as then);
Heheh... let's say we now have years of experience about the reliability
:) At least my webservers reliably serving both ipv4 and ipv6.
Dani�l