apache + mod_lisp + cmucl + clsql +...
Chris Beggy
+ cmucl + clsql + lml + cl-ppcre. I *think* that's all of the
components and libraries I used, but maybe I am overlooking some!
Oh, right. uffi + mysql too:
http://lisp.t2100cdt.kippona.net/lispy/home
There's a little advocacy, but it's mostly a nuts and bolts
explanation. I've geared it to people familiar with php,
mod_perl, eperl, etc.
I will move it over to a production box running sbcl when I
get a round toit. I welcome comments and feedback.
Chris
Eduardo Muñoz
> I've done a simple website with a database using apache + mod_lisp
> + cmucl + clsql + lml + cl-ppcre. I *think* that's all of the
> components and libraries I used, but maybe I am overlooking some!
> Oh, right. uffi + mysql too:
>
> http://lisp.t2100cdt.kippona.net/lispy/home
>
You can see something similar here (my first webserver :)
Most of the content is static html, generated using Tim
Bradshaw's htout. The lisp section is dinamic (htout
again). Keep in mind that the site is still under
construction. I'm using Debian+Apache+mod_lisp+CMUCL+htout
for the website and Emacs+ilisp+tramp+detachtty for
interactive sessions from a Win2K box to change the page at
runtime.
BTW, your transfer rate for lml only is impressive (2196
Kb/s) are you using some kind of caching?
--
Eduardo Muñoz
Chris Beggy
Eduardo,
Nice web site.
> BTW, your transfer rate for lml only is impressive (2196
> Kb/s) are you using some kind of caching?
No, no caching that I know of.
Chris
Vlad S.
vulnereable to the "chunk handling bug", amongst other things. I just
built 1.3.27 yesterday (to act as a proxy for a simple
Portable-Allegroserve based search, which by itself fell victim to a
DOS attack last week), and it seems to work pretty well. That, and on
my "deployment platform" (ie - the rescued-from-the-trash Pentium 90
overclocked to 133 sitting in my basement) Apache throws around images
and such much faster than Paserve+CMUCL. Also, if you ever want to try
running a Lisp webserver behind an Apache proxy (like Cliki does), I
believe that HTTP 1.1 proxying was only added in 1.3.26.
Chris Beggy
Daniel Barlow
Yeah, although note that 1.1 proxying is only really any use if you
are using chunked encoding on your Lisp server, or you know the
correct Content-length when you write the headers (which is less
likely for dynamic content). Without some indication of content
length, it won't know where the boundaries between responses are.
One attraction of mod_proxy (I don't know if mod_lisp also does this)
is that if you implement conditional GET, and and you do Last-modified
and Expires headers correctly on oyur origin server, you can turn on
Apache's proxy cache to get significant speedups on your static and
nearly-static content, without the administrative hassle of moving it
all to another virtual host.
Chunked encoding for Araneida (the server used by CLiki) is on my list
of things to get around to eventually, but absent clamouring hordes of
paying users, I think that "taking an axe to SBCL's file-streams code"
may be a little higher up the priority list.
-dan
--
http://www.cliki.net/ - Link farm for free CL-on-Unix resources
Rob Warnock
+---------------
| Chris Beggy <chr...@kippona.com> writes:
| > I've done a simple website with a database using apache + mod_lisp
| > Oh, right. uffi + mysql too:
| > http://lisp.t2100cdt.kippona.net/lispy/home
|
| You can see something similar here (my first webserver :)
| http://213.97.131.125/lisp
| Most of the content is static html, generated using Tim
| Bradshaw's htout. The lisp section is dinamic (htout
| again). Keep in mind that the site is still under
| construction. I'm using Debian+Apache+mod_lisp+CMUCL+htout
+---------------
I can't show off the database stuff yet (since the data is proprietary),
but I've been having lots of fun hacking some web-based database apps
with FreeBSD + Apache + CMUCL + a tiny CGI library I wrote + Tim's
HTOUT + Eric Marsden's PG + PostgreSQL. Virtually no problems at all
with integration (except my own learning curve with CL). Development
has been quite pleasant!
Because the anticipated traffic is quite low (and because I won't have
complete freedom with the Apache server on the machine that'll be used
in production), I haven't bothered with mod_lisp yet (sorry, Marc!).
Instead, I simply built a saved image of CMUCL with all the above stuff
in it, plus a customized init function which includes a readmacro for #!
which lets plain ol' CGI scripts start with:
#!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
;;;; Lisp code here...
This fortunately works on FreeBSD, despite the two "words" after the
initial #! path. [CMUCL is documented to accept "-core=/path/to/core",
which would allow this to work on other OSes, but it doesn't. When I
get time to learn how to build CMUCL's "lisp" from sources (my first
attempt died with some kind of version skew), I'll fix that. It's a
really simple patch to "lisp/lisp.c:main()".]
As a side benefit, that same "cgi.core" is usable for running all
kinds of non-CGI scripts as well, and FreeBSD's VM system does a
very good job of cacheing the 20+ MB core file (which is mostly
read-only, after all), so that after it's been used one time,
subsequent scripts run *very* fast. E.g., the following script
runs in under 20ms on a 1.4 GHz Athlon, and under 250ms on an
ancient 133 MHz Pentium (with only 48 MB RAM):
% cat foo
#!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
(format t "Hello, world! Args = ~s~%" *script-args*)
% time foo bar baz
Hello, world! Args = ("bar" "baz")
0.011u 0.005s 0:00.01 100.0% 264+5144k 0+0io 0pf+0w
%
So much for complaining about "startup time" for CL. I've just
about given up using anything else for general-purpose scripting!
-Rob
p.s. As long as I'm sharing grotesque hacks, the following is offered
without further explanation or apology: ;-} ;-}
#!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
;;;; Hack to allow CMUCL FASL files to run as shell scripts.
;;;; Usage: cat {this_file} foo1.x86f ... fooN.x86f > foo ; chmod +x foo
;;;; The last "fooN.x86f" should end with a top-level form which runs the app.
(let ((stream (open *script-name* :element-type '(unsigned-byte 8))))
(dotimes (i 8) (read-line stream)) ; Eat header text from stream, then
(load stream :contents :binary) ; pass rest to FASLOAD.
(unix:unix-exit 0)) ; Keep LOAD of script from falling into FASL file(s).
-----
Rob Warnock, PP-ASEL-IA <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Vlad S.
Oh, man, you aren't going to let me procrastinate, are you =].
http://voodoohut.homeunix.net/iz-search
It's a search of the Izware forums (
http://www.izware.com/cgi-bin/Ultimate.cgi?action=intro ), which run
on the free UBB version, which doesn't have one, and makes it annoying
to find all the goodies that have accumulated there over the past
several years. You may experience highly erratic search times as the
thread titles are initially pulled from a (very slow, and usually
asleep) disk, but once pulled they are memoized. Besides that it has
everything it needs in memory, so the actual page generation takes a
few fractions of a second. Blame the rest on my cable connection.
Right now Paserve is running naked, as the Apache 1.3.27 proxy won't
talk to it (the actual Apache server does, however, work pretty well).
I'll have to fix that later.
Vlad
Ingvar Mattsson
> voodo...@hotmail.com (Vlad S.) writes:
>
> > You might want to upgrade your version of Apache. 1.3.22 is
> > vulnereable to the "chunk handling bug", amongst other things. I just
> > built 1.3.27 yesterday (to act as a proxy for a simple
> > Portable-Allegroserve based search, which by itself fell victim to a
> > DOS attack last week), and it seems to work pretty well. That, and on
> > my "deployment platform" (ie - the rescued-from-the-trash Pentium 90
> > overclocked to 133 sitting in my basement) Apache throws around images
> > and such much faster than Paserve+CMUCL. Also, if you ever want to try
> > running a Lisp webserver behind an Apache proxy (like Cliki does), I
> > believe that HTTP 1.1 proxying was only added in 1.3.26.
>
> Yeah, although note that 1.1 proxying is only really any use if you
> are using chunked encoding on your Lisp server, or you know the
> correct Content-length when you write the headers (which is less
> likely for dynamic content). Without some indication of content
> length, it won't know where the boundaries between responses are.
One could cheat and do something similar to what I did for my C CGI
library. One had to do some minor changes to how one handled output
(use an emit function, obeying printf format strings) and finish with
a finaliser function. The CGI library then cached all output data and
generated the header as the very last thing it did.
Possibly not ideal, though, since there will be nothing arriving while
eth script thinks, but taht wasn't a problem for the stuff I needed it
for.
//Ingvar
--
Ingvar Mattsson; gr...@algonet.se;
You can get further with a kind word and a 2x4
than you can with just a kind word. Among others, Marcus Cole
Daniel Barlow
> One could cheat and do something similar to what I did for my C CGI
> library. One had to do some minor changes to how one handled output
> (use an emit function, obeying printf format strings) and finish with
> a finaliser function. The CGI library then cached all output data and
> generated the header as the very last thing it did.
Sure, and it's even easier in Lisp because you have string-streams, so
no need to change (the majority of) existing code.
> Possibly not ideal, though, since there will be nothing arriving while
> eth script thinks, but taht wasn't a problem for the stuff I needed it
> for.
Yeah, it's not nice if you're sucking 200 records out of a database to
display or something like that.
Håkon Alstadheim
> without further explanation or apology: ;-} ;-}
>
> #!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
> ;;;; Hack to allow CMUCL FASL files to run as shell scripts.
> ;;;; Usage: cat {this_file} foo1.x86f ... fooN.x86f > foo ; chmod +x foo
> ;;;; The last "fooN.x86f" should end with a top-level form which runs the app.
> (let ((stream (open *script-name* :element-type '(unsigned-byte 8))))
> (dotimes (i 8) (read-line stream)) ; Eat header text from stream, then
> (load stream :contents :binary) ; pass rest to FASLOAD.
> (unix:unix-exit 0)) ; Keep LOAD of script from falling into FASL file(s).
Here is an alternative for CMUCL on linux.
Put this in a boot script (e.g. /etc/init.d./boot.local ):
--
echo ':lisp-magic:M::FASL FILE output::/usr/bin/lisp-start:' > /proc/sys/fs/binfmt_misc/register
--
This will recognise fasl files by the first few bytes. You might
need to modprobe binfmt_misc first.
Then, in /usr/bin/lisp-start I have:
--
#!/bin/sh
# se /etc/init.d/boot.local
exec /usr/bin/lisp \
-core /home/hakon/cl/cmucl/18dlocal/output/release/lib/lisp-start.core \
-quiet -batch \
-eval '(setf EXTENSIONS::*complain-about-illegal-switches* nil )' \
-load "$@" \
-eval '(quit)'
--
You could of course make the lisp-start command distinguish
different versions of fasl (just open a fasl file in emacs and have
a look). You could also use file-type recognition like:
--
echo ':lisp-src:E::lisp::/usr/bin/lisp-start:' > /proc/sys/fs/binfmt_misc/register
echo ':lisp:E::x86f::/usr/bin/lisp-start:' > /proc/sys/fs/binfmt_misc/register
--
The only thing special about the lisp-start.core is that it is saved
with ext:*load-verbose* set to nil, because the -quiet switch is
processed after the *init.lisp files have been loaded, which messes
up the output.
Remember that you can rename your fasl files without the extension,
and set the executable bit on them. This will make fasl files blend
in seamlessly with regular linux commands. You should also check out
file(1) and teach it about fasl files, especially if you remove the
x86f extension.
The thing you say about startup times being good after the first one
also holds for linux, I have no timings though.
--
Håkon Alstadheim, hjemmepappa.
Ingvar Mattsson
> Ingvar Mattsson <ing...@cathouse.bofh.se> writes:
>
> > One could cheat and do something similar to what I did for my C CGI
> > library. One had to do some minor changes to how one handled output
> > (use an emit function, obeying printf format strings) and finish with
> > a finaliser function. The CGI library then cached all output data and
> > generated the header as the very last thing it did.
>
> Sure, and it's even easier in Lisp because you have string-streams, so
> no need to change (the majority of) existing code.
Yep. Darned handy they are, too. I'm doodling on a package to
implement as much as I can of emacs-lisp semantics[1] as macrology on top
of CL and actually use that in one place (mostly because I'm not quite
sure of to load-time define macros depending on the state of the
system at load-time, if it was functions, I could probably have used
(SETF (FUNCTION ...) ...) ). Though it cave me a perfect opportunity
to combine that hack with some name surgery.
Basically, iterate through the symbols provided in one package and
make sure to generate wrapper-macros for everything in one. I do this
by writing to a string-stream and then load said string-stream.
> > Possibly not ideal, though, since there will be nothing arriving while
> > eth script thinks, but taht wasn't a problem for the stuff I needed it
> > for.
>
> Yeah, it's not nice if you're sucking 200 records out of a database to
> display or something like that.
200 records should be quite fast, shouldn't it? Depends on the query,
but...
//Ingvar
[1] I am *not* aiming at case-preservation, once it's closer to
completion, I'll see how much of the built-ins are needed to
provide something that can load some elisp and I'll then notice
how much code breaks.
--
When the SysAdmin answers the phone politely, say "sorry", hang up and
run awaaaaay!
Informal advice to users at Karolinska Institutet, 1993-1994
Rob Warnock
+---------------
| > p.s. As long as I'm sharing grotesque hacks, the following is offered
| > without further explanation or apology: ;-} ;-}
| >
| > #!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
|
| Here is an alternative for CMUCL on linux.
| echo ':lisp-magic:M::FASL FILE output::/usr/bin/lisp-start:' >
| /proc/sys/fs/binfmt_misc/register
| exec /usr/bin/lisp ...
+---------------
Yes, that's also described in <URL:http://www.cliki.net/CMUCL%20Hints>.
The only problem with this, of course, is that it fork/execs an additional
/bin/sh trampoline process (my hack doesn't), so it's no more efficient
than having a separate wrapper script that explicitly loads the FASL file.
The upside to the "binfmt_misc" approach, of course, is that you don't
have to put any special header in front of the FASL file, nor do you
need to use a special core image.
(Choices, choices, so many...)
-Rob
Daniel Barlow
> p.s. As long as I'm sharing grotesque hacks, the following is offered
> without further explanation or apology: ;-} ;-}
>
> #!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
> ;;;; Hack to allow CMUCL FASL files to run as shell scripts.
> ;;;; Usage: cat {this_file} foo1.x86f ... fooN.x86f > foo ; chmod +x foo
> ;;;; The last "fooN.x86f" should end with a top-level form which runs the app.
> (let ((stream (open *script-name* :element-type '(unsigned-byte 8))))
> (dotimes (i 8) (read-line stream)) ; Eat header text from stream, then
> (load stream :contents :binary) ; pass rest to FASLOAD.
> (unix:unix-exit 0)) ; Keep LOAD of script from falling into FASL file(s).
I came up with the approximate equivalent for SBCL this morning
(before reading this thread). I used an sh, which could probably be
skipped on an OS that supported enough #! argumentation, but I'm not
trying to do CGI with it so I don't worry too much about that. The
upside is that I don't need a custom core, so I could in theory
distribute "binaries" this way and they'd run with the standard sbcl
package from Debian or whoever.
#!/bin/sh --
exec sbcl --noinform --disable-debugger --userinit /dev/null --sysinit /dev/null --eval "(with-open-file (i \"$0\" :element-type '(unsigned-byte 8)) (read-line i) (read-line i) (load i) (quit))" --end-toplevel-options ${1+"$@"}
Roland Kaufmann
[[snip]]
Ingvar> I'm doodling on a package to implement as much as I can of
Ingvar> emacs-lisp semantics[1] as macrology on top of CL and
Ingvar> actually use that in one place (mostly because I'm not
Ingvar> quite sure of to load-time define macros depending on the
Ingvar> state of the system at load-time, if it was functions, I
Ingvar> could probably have used (SETF (FUNCTION ...) ...) ).
Ingvar> Though it cave me a perfect opportunity to combine that
Ingvar> hack with some name surgery.
[[snip]]
Ingvar> //Ingvar
Ingvar> [1] I am *not* aiming at case-preservation, once it's closer to
Ingvar> completion, I'll see how much of the built-ins are needed to
Ingvar> provide something that can load some elisp and I'll then notice
Ingvar> how much code breaks.
Are you aware of Sam Steingolds Emacs Lisp in CL code (in CLOCC cllib)?
HTH.
Roland
Ingvar Mattsson
[SNIP]
> Are you aware of Sam Steingolds Emacs Lisp in CL code (in CLOCC cllib)?
> HTH.
I wasn't, no. Ah, well, I'll continue to doodle on mine, so far it's
fun. Managed to create something that's amazingly similar to
buffer-local variables this morning, but there's a few small bugs to
iron out.
//Ingvar
--
A routing decision is made at every routing point, making local hacks
hard to permeate the network with.
Chris Beggy
> Håkon Alstadheim <haa...@online.no> wrote:
> +---------------
> | > p.s. As long as I'm sharing grotesque hacks, the following is offered
> | > without further explanation or apology: ;-} ;-}
> | >
> | > #!/usr/local/bin/cmucl -core /usr/local/lib/cmucl/lib/cgi.core
> | > ;;;; Hack to allow CMUCL FASL files to run as shell scripts...
> |
> | Here is an alternative for CMUCL on linux.
> ...
> | echo ':lisp-magic:M::FASL FILE output::/usr/bin/lisp-start:' >
> | /proc/sys/fs/binfmt_misc/register
> | Then, in /usr/bin/lisp-start I have:
> | #!/bin/sh
> | exec /usr/bin/lisp ...
> +---------------
>
> Yes, that's also described in <URL:http://www.cliki.net/CMUCL%20Hints>.
> The only problem with this, of course, is that it fork/execs an additional
> /bin/sh trampoline process (my hack doesn't), so it's no more efficient
> than having a separate wrapper script that explicitly loads the FASL file.
>
> The upside to the "binfmt_misc" approach, of course, is that you don't
> have to put any special header in front of the FASL file, nor do you
> need to use a special core image.
>
> (Choices, choices, so many...)
I found that neither the binfmt_misc nor the trampoline methods
have worked for me in cases where lisp opens a tcp
connection and listens on a port. For this case, detachtty was
the only way I could properly create a daemon. Thank heavens for
detachtty.
Chris
Marc Battyani
"Chris Beggy" <chr...@kippona.com> wrote
> I've done a simple website with a database using apache + mod_lisp
> + cmucl + clsql + lml + cl-ppcre. I *think* that's all of the
> components and libraries I used, but maybe I am overlooking some!
> Oh, right. uffi + mysql too:
>
> http://lisp.t2100cdt.kippona.net/lispy/home
>
> There's a little advocacy, but it's mostly a nuts and bolts
> explanation. I've geared it to people familiar with php,
> mod_perl, eperl, etc.
Very cool! I've put links to it on the mod_lisp web site.
Marc
Marc Battyani
"Rob Warnock" <rp...@rpw3.org> wrote in
> Because the anticipated traffic is quite low (and because I won't have
> complete freedom with the Apache server on the machine that'll be used
> in production), I haven't bothered with mod_lisp yet (sorry, Marc!).
No problem, it's just that you don't know what you are missing ... ;-)
It's less bothering to use mod_lisp and a persistent lisp process at least
to develop and debug.
When there is an error in the lisp code processing a request, LW fires the
debugger so I can look at the problem, correct it and go on without any
trouble. I can also use things like trace, break etc.
That alone is a good point for mod_lisp and other persistent ways to serve
web pages (aserve, cl-http, aranedia, etc.)
Marc
Daniel Barlow
> When there is an error in the lisp code processing a request, LW fires the
> debugger so I can look at the problem, correct it and go on without any
> trouble. I can also use things like trace, break etc.
> That alone is a good point for mod_lisp and other persistent ways to serve
> web pages (aserve, cl-http, aranedia, etc.)
Yup. When an end-user has an obscure problem and can't describe it
accurately (which is the case for a lot of end-user's bug reports) I
can ssh into the server and trace any functions I like while they
attempt the operation again. This is a big help if they're using a
browser or OS version I don't have (keeping n versions of IE around
pretty much requires keeping n versions of Windows around) or are
stuck behind a firewall or proxy that is translating the request
somehow, or any one of a number of other scenarios.
Rob Warnock
+---------------
| "Rob Warnock" <rp...@rpw3.org> wrote in
| > Because the anticipated traffic is quite low (and because I won't have
| > complete freedom with the Apache server on the machine that'll be used
| > in production), I haven't bothered with mod_lisp yet (sorry, Marc!).
|
| No problem, it's just that you don't know what you are missing ... ;-)
Actually, I think I *do* know, but am under some (mostly non-technical)
constraints that don't let me do anything about it. Fortunately, CMUCL's
startup time on modern CPUs/OSs is fast enough that I can still use Lisp
even given those constraints.
+---------------
| It's less bothering to use mod_lisp and a persistent lisp process at
| least to develop and debug. When there is an error in the lisp code
| processing a request, LW fires the debugger so I can look at the problem,
| correct it and go on without any trouble...
+---------------
Yes. (*sigh*)
But with a "tail -f" aimed at the Apache error log on the one hand
and a wrapper that lets me simulate the CGI environment on the other
(so I can "run" a CGI from a REPL, catching errors in the debugger),
development in Lisp is *still* a lot easier/faster than doing it with
/bin/sh and Perl scripts was before.
One step at a time...
-Rob