1) Is there any bechmark to comparing the performance between Apache with mod_wsgi and Aspen? It would be very insteresting to have any benchamrk using any Django/TurboGears application.
> 1) Is there any bechmark to comparing the performance between > Apache with mod_wsgi and Aspen? It would be very insteresting > to have any benchmark using any Django/TurboGears application. > > 2) Is Aspen ready to scale?
With any framework like Django/TG, one would expect bottlenecks to be with the framework, not the server.
That said, I've looked at mod_wsgi, and I think it might be a good way to scale Aspen. Aspen is simply a process manager around a WSGI app, so I don't think there's a reason we can't wire the WSGI app to also run with mod_wsgi as its process manager.
This would give you some great flexibility:
o Aspen/Aspen for local dev o Aspen/Aspen for small-scale deployment o mod_wsgi/Aspen for large-scale deployment o clustered Aspen/Aspen for large-large-scale deployment
Theoretically, your site wouldn't have to change at all on the filesystem to work for any of these patterns. Supporting mod_wsgi would also make Aspen more accessible to the commodity hosting market, assuming that mod_wsgi picks up steam there.
> > 1) Is there any bechmark to comparing the performance between > > Apache with mod_wsgi and Aspen? It would be very insteresting > > to have any benchmark using any Django/TurboGears application.
> > 2) Is Aspen ready to scale?
> With any framework like Django/TG, one would expect bottlenecks > to be with the framework, not the server.
> That said, I've looked at mod_wsgi, and I think it might be a > good way to scale Aspen. Aspen is simply a process manager around > a WSGI app, so I don't think there's a reason we can't wire the > WSGI app to also run with mod_wsgi as its process manager.
> This would give you some great flexibility:
> o Aspen/Aspen for local dev > o Aspen/Aspen for small-scale deployment > o mod_wsgi/Aspen for large-scale deployment > o clustered Aspen/Aspen for large-large-scale deployment
Pardon my ignorance, but what does Aspen have that would make it the option for 'clustered Aspen/Aspen for large-large-scale deployment'? How would that be better than using multiple Apache instances (using mod_wsgi) on a cluster of machines with a load balancing proxy in front?
> Pardon my ignorance, but what does Aspen have that would make > it the option for 'clustered Aspen/Aspen for large-large-scale > deployment'? How would that be better than using multiple > Apache instances (using mod_wsgi) on a cluster of machines with > a load balancing proxy in front?
Sure, you could do that too. I guess I think of Aspen as easier to install and configure than Apache+mod_wsgi, but by the time you have problems that clustering will solve, you don't need anyone telling you how to do it.
> > Pardon my ignorance, but what does Aspen have that would make > > it the option for 'clustered Aspen/Aspen for large-large-scale > > deployment'? How would that be better than using multiple > > Apache instances (using mod_wsgi) on a cluster of machines with > > a load balancing proxy in front?
> Sure, you could do that too. I guess I think of Aspen as easier > to install and configure than Apache+mod_wsgi, but by the time > you have problems that clustering will solve, you don't need > anyone telling you how to do it.
> Thanks for keeping me honest. :)
It isn't a question of keeping you honest. I'm genuinely interested in how it would be setup for that sort of scenario. I had a quick glance at the documentation but couldn't really see anything about deployment scenarios. If there is something in the documentation then at least just point me in the right direction.
> I had a quick glance at the documentation but couldn't really > see anything about deployment scenarios.
Well, on one level Aspen is simply a general-purpose web server, just like Apache, lighttpd, nginx, etc. So deployment scenarios needn't look any different than with those tools. Regarding the clustering scenario specifically, Aspen can fill the same role as any backend web app server set up on multiple machines behind a load balancing HTTP proxy.
Now, Aspen's core functionality--its app, handler, and middleware hooks[1]--is implemented internally as a WSGI application. This means that Aspen-the-WSGI-app could be wired to any WSGI server, e.g., mod_wsgi. And if I haven't communicate this yet, let me say here that I see such a connector as a great boon to Aspen, because it makes Aspen's core functionality available to the wider Apache audience. I hope to get a connector together before too long.
> > I had a quick glance at the documentation but couldn't really > > see anything about deployment scenarios.
> Well, on one level Aspen is simply a general-purpose web server, > just like Apache, lighttpd, nginx, etc. So deployment scenarios > needn't look any different than with those tools. Regarding the > clustering scenario specifically, Aspen can fill the same role as > any backend web app server set up on multiple machines behind a > load balancing HTTP proxy.
> Now, Aspen's core functionality--its app, handler, and middleware > hooks[1]--is implemented internally as a WSGI application. This > means that Aspen-the-WSGI-app could be wired to any WSGI server, > e.g., mod_wsgi. And if I haven't communicate this yet, let me say > here that I see such a connector as a great boon to Aspen, > because it makes Aspen's core functionality available to the > wider Apache audience. I hope to get a connector together before > too long.
> Am I answering your question at all?
Yeah. Just thought that since you had mentioned distinctly the clustering scenario that Aspen itself may have provided something which was useful in that context.
Related question about inner WSGI application. The question in another post about global config module/setup, does that only apply to the Aspen web server or is that global information depended upon by the Aspen WSGI application component.
Wanting to know so I know if two instances of the Aspen WSGI application can be mounted at two different mount points under the same WSGI adapter for a server. Ie., slightly different scenario to other poster where they wanted two instances of the web server itself running on different ports.
Have come across a number of WSGI frameworks which depend on global configuration (Django, web.py etc) and as such the only way to host two distinct instances of the application is by process separation or by delegation to different Python sub interpreters in the same process.
Since mod_wsgi has the latter ability, like to know about whether different WSGI frameworks support multiple instances or not in same interpreter space. That way can advise people using mod_wsgi as to what they can and cant do when they have problems. If I get some time and get Aspen working on mod_wsgi and decide to put up a page about how to do it, it is also issues like this that I like to document if relevant.
FWIW, I actually have an experimental Python extension module which allows one in normal Python program to create additional sub interpreters and make calls between sub interpreters. It is arguably a bit dangerous to use in its current form so have never made it available. Feasibly though, by restricting further how it works so that all one is doing when creating a new sub interpreter is giving it a module to import like a main code file and execute it, one could make it reasonably safe. With this one could actually solve that other posters problem of running multiple distinct instances of the web server on different ports. Ie., each would run in its own sub interpreter.
One of these days I will have to revisit the code and release it. :-)
> I hope to get a connector together before too long.
Ok, I've added an initial 'aspen.mod_wsgi' script to the Aspen trunk, r235.
The trick is configuring Aspen's root directory. Aspen only exposes this to the command line, not the aspen.conf file. Since you can't pass command line arguments to your mod_wsgi script (right?), the result is that the current working directory of your Apache or WSGIDaemonProcess process will be the Aspen root. I'm not sure if/how you can do this for Apache (ServerRoot?), but mod_wsgi exposes this with the 'home' option to WSGIDaemonProcess.
(BTW, I'm still having problems with mod_wsgi trunk r427, Apache 2.0.59, and FreeBSD 6.1-RELEASE. It works with httpd -X and a WSGIScriptAlias, but I haven't gotten WSGIDaemonProcess to work, or even Apache's own background mode. Sorry I don't have a better problem report at the moment. :^( )
On 16/08/07, Chad Whitacre <c...@zetaweb.com> wrote:
> Graham, et al.
> > I hope to get a connector together before too long.
> Ok, I've added an initial 'aspen.mod_wsgi' script to the Aspen > trunk, r235.
I'll try and schedule some time to look at it. :-)
> The trick is configuring Aspen's root directory.
Hmmm, does this mean that Aspen WSGI component is like Karigell in that it will not work if the current working directory is not the site directory?
> Aspen only > exposes this to the command line, not the aspen.conf file. Since > you can't pass command line arguments to your mod_wsgi script > (right?),
Technically no, but you can do stuff at global scope within the script. Provided script file isn't changed, or WSGIScriptReloading set to Off, it only gets loaded once the first time application is accessed. If one had to you could fiddle sys.argv if you wanted to, although for a WSGI application to need that to be done means one could say it isn't necessarily a well behaved WSGI application. ;-)
> the result is that the current working directory of > your Apache or WSGIDaemonProcess process will be the Aspen root. > I'm not sure if/how you can do this for Apache (ServerRoot?), but > mod_wsgi exposes this with the 'home' option to WSGIDaemonProcess.
If using embedded mode of mod_wsgi changing the working directory and depending on it being a specific location is not a good idea in case some other non Python stuff wants to also change it.
Karrigell has such issues and why for it the only option is to run it with daemon mode. For details see:
This includes showing how Karrigell needs to change the working directory, although as you identify, could also be done using 'home' option to WSGIDaemonProcess.
> (BTW, I'm still having problems with mod_wsgi trunk r427, Apache > 2.0.59, and FreeBSD 6.1-RELEASE. It works with httpd -X and a > WSGIScriptAlias, but I haven't gotten WSGIDaemonProcess to work, > or even Apache's own background mode. Sorry I don't have a better > problem report at the moment. :^( )
What do mean by 'Apache's own background mode'? Do you mean that only httpd -X works even when mod_wsgi isn't being loaded? If that is the case, sounds like threaded Apache on FreeBSD still has issues.
> Yeah. Just thought that since you had mentioned distinctly the > clustering scenario that Aspen itself may have provided > something which was useful in that context.
Nope, nothing like mongrel_cluster, e.g., though not on principle, just lack of itch/scratcher. Contributions welcome. :)
> [I]s that global information depended upon by the Aspen WSGI > application component?
Yes. It will take some significant refactoring to change, so hearing another use case here is valuable, thanks.
> Have come across a number of WSGI frameworks which depend on > global configuration (Django, web.py etc) and as such the only > way to host two distinct instances of the application is by > process separation or by delegation to different Python sub > interpreters in the same process.
Aspen does enable a related use-case, where the same framework instance is mounted at a few different URL paths. For example, I usually mount django on /admin/, and other stuff elsewhere. If I wanted to use django at a second path I could do this in Aspen's apps.conf:
Does this at all address the use case you mention? (It doesn't address Ori's problem, where he actually wants listeners on two separate ports).
> If I get some time and get Aspen working on mod_wsgi and decide > to put up a page about how to do it, it is also issues like > this that I like to document if relevant.
Understood. Check out the connector I just checked in and let me know what else I can help with.
> FWIW, I actually have an experimental Python extension module > which allows one in normal Python program to create additional > sub interpreters and make calls between sub interpreters.
> > [I]s that global information depended upon by the Aspen WSGI > > application component?
> Yes. It will take some significant refactoring to change, so > hearing another use case here is valuable, thanks.
> > Have come across a number of WSGI frameworks which depend on > > global configuration (Django, web.py etc) and as such the only > > way to host two distinct instances of the application is by > > process separation or by delegation to different Python sub > > interpreters in the same process.
> Aspen does enable a related use-case, where the same framework > instance is mounted at a few different URL paths. For example, I > usually mount django on /admin/, and other stuff elsewhere. If I > wanted to use django at a second path I could do this in Aspen's > apps.conf:
> Does this at all address the use case you mention?
Not sure that it does as not talking about the same application instance. Understanding how Django works, you would be relying on the one Django instance with it supporting the different URLs sets you are passing it. With Django, you couldn't within the same interpreter have two Django instances each with their own /admin pages.
Imagine now for example a scenario where someone used Aspen to create a wiki application and that it added its own sections in to the global configuration. Now, someone totally separate created a blog application which just so happened to need to add its own sections into the global configuration and there was a clash.
In this scenario, if you are forcing all application components to use the one configuration file, there may be issues when a third party wants to try and combine these two distinct application components together in the one overall application. To get it to work it would seem that it may be required to modify one or the other rather than it just being a process of dropping them in and mounting them at URL you want them to be available.
Also now consider that someone wanted to add two distinct instances of the wiki component running in the same overall application, each with its own data area. The idea with WSGI components is that the application entry point be able in some way to be given different configuration based on its mount point. At the simplest this could be a middleware wrapper that passes it through the WSGI environment. Alternatively, the configuration could be passed to a constructor for the application object when it is being created and mounted at required location in URL mapping mechanism. If a frameworks native configuration mechanism depends on a single global configuration source, this might be difficult, or certainly would require the component writer to specifically design for it.
I do admit that I am quite ignorant about how things are done with Aspen, so it may be the case that there are easy ways of dealing with this. :-)
> I'll try and schedule some time to look at it. :-)
Thank you. :) It actually turned out to be quite simple.
> Hmmm, does this mean that Aspen WSGI component is like Karigell > in that it will not work if the current working directory is > not the site directory?
No, it's not as bad as that. The Aspen root defaults to the cwd, but it can be changed ... but only from the command line. If mod_wsgi allowed one to pass command line arguments to the wsgi script, then you could change the Aspen root to whatever you wanted.
I've had it in mind to enable one to set the Aspen root from a conf file, and this gives me another reason to do so.
> Technically no, but you can do stuff at global scope within the > script. Provided script file isn't changed, or > WSGIScriptReloading set to Off, it only gets loaded once the > first time application is accessed. If one had to you could > fiddle sys.argv if you wanted to, although for a WSGI > application to need that to be done means one could say it > isn't necessarily a well behaved WSGI application. ;-)
Now that you mention it, you could also make your own mod_wsgi script using Aspen's API. I've just checked in a change (r236) so you can pass your own argv to aspen.website_factory. A hack to be sure, but better than nothing. Thanks for the idea. :)
So you can change the Aspen root by using a script like this instead of the aspen.mod_wsgi script bundled with Aspen:
#!/usr/bin/env python from aspen import website_factory application = website_factory(['--root', '/usr/home/whit537'])
> If using embedded mode of mod_wsgi changing the working > directory and depending on it being a specific location is not > a good idea in case some other non Python stuff wants to also > change it.
Agreed.
> Do you mean that only httpd -X works even when mod_wsgi isn't > being loaded? If that is the case, sounds like threaded Apache > on FreeBSD still has issues.
Yes and yes. ACK!!!!!!! >:^(
Ok, bedtime here. Thanks for all the feedback Graham!
> Imagine now for example a scenario where someone used Aspen to > create a wiki application and that it added its own sections in > to the global configuration. Now, someone totally separate > created a blog application which just so happened to need to > add its own sections into the global configuration and there > was a clash.
Aspen's global conf file is __/etc/aspen.conf (where __ is a protected directory under your website's root), and it's in standard INI format. So the wiki and blog config would only clash if both used the same section name, which I suggest is unlikely.
However, neither need to use __/etc/aspen.conf at all. Back to the Django example: it uses it's own configuration mechanism (settings.py) within Aspen, and both wiki and blog app could do the same.
> In this scenario, if you are forcing all application components > to use the one configuration file [...]
As mentioned, Aspen is designed for multiple configuration files. It's the Unix way. :) This is the reason for the __/etc directory within an Aspen website. Applications are welcome both to use __/etc/aspen.conf and/or to use their own particular configuration mechanisms.
> Also now consider that someone wanted to add two distinct > instances of the wiki component running in the same overall > application, each with its own data area.
Ok ...
> The idea with WSGI components is that the application entry > point be able in some way to be given different configuration > based on its mount point.
This is something I hadn't fully taken on board yet, thanks for bringing it to my attention.
> At the simplest this could be a middleware wrapper that passes > it through the WSGI environment.
So you would write your middleware wrapper in your project package under __/lib/python, and wire that up in __/etc/middleware.conf.
> Alternatively, the configuration could be passed to a > constructor for the application object when it is being created > and mounted at required location in URL mapping mechanism.
Under Aspen, this would mean writing a shim within your project package that configured the objects as needed, and then wire the shim in Aspen's __/etc/apps.conf. This is the basis for glue code I've already got for Django and Trac:
And this is parallel to what you've got on the wiki for mod_wsgi+frameworks, no?
> If a frameworks native configuration mechanism depends on a > single global configuration source, this might be difficult, or > certainly would require the component writer to specifically > design for it.
Calling Aspen a framework is probably too strong. It's basically just a middleware conf file, an app-mounting conf file, and a (more complex) handler association conf file. At this level, I'm not sure two Aspen instances in your site are any better than one.
That is, the primary value of Aspen-as-WSGI-app is precisely in wiring up other WSGI components. If Aspen-WSGI isn't your primary WSGI router, then it doesn't really have much to do, and you probably shouldn't be using it.
(Ori's problem is at a lower level. He actually wants different server/listener instances, not just WSGI app instances. That's another story.)
> I do admit that I am quite ignorant about how things are done > with Aspen, so it may be the case that there are easy ways of > dealing with this. :-)
If I've understood you right, I think Aspen does in fact account for these use cases. Let me know if not.
> > Imagine now for example a scenario where someone used Aspen to > > create a wiki application and that it added its own sections in > > to the global configuration. Now, someone totally separate > > created a blog application which just so happened to need to > > add its own sections into the global configuration and there > > was a clash.
> Aspen's global conf file is __/etc/aspen.conf (where __ is a > protected directory under your website's root), and it's in > standard INI format. So the wiki and blog config would only clash > if both used the same section name, which I suggest is unlikely.
> However, neither need to use __/etc/aspen.conf at all. Back to > the Django example: it uses it's own configuration mechanism > (settings.py) within Aspen, and both wiki and blog app could do > the same.
The problem in the Django case though is that Django expects to pick up the name of the settings file from os.environ and there is only one of them. This is what makes it tricky with Django to host more than one instance in an interpreter without doing fiddles.
I'll try and digest everything else later and comment then if I feel the need. Thanks for explaining a bit more about Aspen. Always interesting to see and learn from how different people solve these sorts of problems.
On 16/08/07, Chad Whitacre <c...@zetaweb.com> wrote:
> Now that you mention it, you could also make your own mod_wsgi > script using Aspen's API. I've just checked in a change (r236) so > you can pass your own argv to aspen.website_factory. A hack to be > sure, but better than nothing. Thanks for the idea. :)
> So you can change the Aspen root by using a script like this > instead of the aspen.mod_wsgi script bundled with Aspen:
