Storing settings on module level -- bad idea?
Tom Lazar
i was getting tired of either a) having to pass in the request all over the place (mainly from views down to helper methods) in order for the latter to have access to the settings via request.settings or b) having to call get_current_registry() all the time (which would mean my tests require some additional setup foo) so i tried this in the myapp/__init__.py:
from pyramid.config import Configurator
settings = dict()
def main(global_config, **_settings):
config = Configurator(settings=_settings)
import myapp
myapp.settings = _settings
config.scan()
return config.make_wsgi_app()
and henceforth wherever i need access to the settings:
from myapp import settings
...
settings['myapp.foo']
is this a bad idea? it seems 'dirty' but it would keep my tests and code significantly leaner...
any opinions?
cheers,
tom
Chris McDonough
I think it might be significantly better to just create a settings.py
module in your app and put stuff in there instead of in your config file
if you want globals, because this is effectively a monkeypatch, and
makes bootstrap code very timing-dependent. Or you could do something
like what's suggested in
<http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/configuration.html#django-style-settings-py-configuration> I personally find that global settings makes my test code much more brittle, because it means I have to clean up the settings after changing them within test code instead of just saying "request.settings = {}". But at the end of the day, it's really up to you.
- C
Tom Lazar
> I think it might be significantly better to just create a settings.py
> module in your app and put stuff in there instead of in your config file
hm, i don't want to *maintain* the settings in python code, just have easy access. or else i end up with duplicate entries that i need to keep in sync.
> if you want globals, because this is effectively a monkeypatch, and
> makes bootstrap code very timing-dependent.
right.
> Or you could do something
> like what's suggested in
> <http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/configuration.html#django-style-settings-py-configuration>
wow, that approach seems almost ridiculous :) following it you end up with having to pass in/configure the location of the ini file *twice* and parse the whole thing manually *again*. i don't think i'll go down that road.
> I personally find that global settings makes my test code much more brittle, because it means I have to clean up the settings after changing them within test code instead of just saying "request.settings = {}". But at the end of the day, it's really up to you.
ah, i just give each test a new instance of the settings, so there's no need to clean up.
i really would like to have a clean, convenient and supported way to access global settings. i guess, the answer is that pyramid currently doesn't have this on offer.
generically exposing the settings that were used to invoke a pyramid instance on module level would be nice, but i infer from what you've said the the convenience that would add would be outweighed by the stuff that could possibly go wrong. correct?
maybe i could just mock get_current_registry() in my test harness and simply use that in the few places of my code that does not have convenient access to a request.
sorry for rambling, i guess, i'm (ab-)using the list for rubberducking... :)
tom
>
> - C
>
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To post to this group, send email to pylons-...@googlegroups.com.
> To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
>
Robert Forkel
the request all the time. But it turned out to be the right (and
transparent) thing to do. In my case this is because I often need
request-specific information like the locale name basically
everywhere. Once you get used to this (and to creating DummyRequests
all the time for tests), you will come to like the registry as the
place to go for global settings.
regards
robert
Chris McDonough
> On 31.01.2012, at 05:48, Chris McDonough wrote:
>
> > I think it might be significantly better to just create a settings.py
> > module in your app and put stuff in there instead of in your config file
>
> hm, i don't want to *maintain* the settings in python code, just have easy access. or else i end up with duplicate entries that i need to keep in sync.
You'd just do the work at global scope in a module to read them from an
ini file. You wouldn't maintain them twice.
> > if you want globals, because this is effectively a monkeypatch, and
> > makes bootstrap code very timing-dependent.
>
> right.
>
> > Or you could do something
> > like what's suggested in
> > <http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/configuration.html#django-style-settings-py-configuration>
>
> wow, that approach seems almost ridiculous :) following it you end up with having to pass in/configure the location of the ini file *twice* and parse the whole thing manually *again*. i don't think i'll go down that road.
If you did what that thing suggested you'd likely write a small wrapper
to run your app instead of using paster serve, so there'd be no "twice".
Or at least you can arrange to do that with enough imagination.
> > I personally find that global settings makes my test code much more brittle, because it means I have to clean up the settings after changing them within test code instead of just saying "request.settings = {}". But at the end of the day, it's really up to you.
>
> ah, i just give each test a new instance of the settings, so there's no need to clean up.
>
> i really would like to have a clean, convenient and supported way to access global settings. i guess, the answer is that pyramid currently doesn't have this on offer.
No, it doesn't, or at least it leaves it up to you. Individuals can
always make nonglobal things global, but a framework can't make things
nonglobal once it makes them global.
> generically exposing the settings that were used to invoke a pyramid instance on module level would be nice, but i infer from what you've said the the convenience that would add would be outweighed by the stuff that could possibly go wrong. correct?
The Django core devs I've heard always lament using global settings.
The zope.testing.setUp stuff that clears a module scope registry is
another antipattern.
> maybe i could just mock get_current_registry() in my test harness and simply use that in the few places of my code that does not have convenient access to a request.
I dunno. If you want global stuff it's easy enough to make things
global if you use a little thought. Or just pass the request around.
- C
Tom Lazar
On Jan 31, 2012, at 1:43 PM, Robert Forkel wrote:
> fwiw: when I started using pyramid I was also reluctant to pass around
> the request all the time. But it turned out to be the right (and
> transparent) thing to do. In my case this is because I often need
> request-specific information like the locale name basically
> everywhere. Once you get used to this (and to creating DummyRequests
> all the time for tests), you will come to like the registry as the
> place to go for global settings.
thanks robert, that was helpful. since posting the original message to this list i've also realized that my real problem is that i'm partially trying to do the right things at the wrong places.
for example, by splitting the setting (a filesystem path) into a root part (deployment specific) and the remaining relative path (application specific) the whole problem vanished.
thanks for the feedback,
tom
Tom Lazar
> If you did what that thing suggested you'd likely write a small wrapper
> to run your app instead of using paster serve, so there'd be no "twice".
> Or at least you can arrange to do that with enough imagination.
true.
>
>>> I personally find that global settings makes my test code much more brittle, because it means I have to clean up the settings after changing them within test code instead of just saying "request.settings = {}". But at the end of the day, it's really up to you.
>>
>> ah, i just give each test a new instance of the settings, so there's no need to clean up.
>>
>> i really would like to have a clean, convenient and supported way to access global settings. i guess, the answer is that pyramid currently doesn't have this on offer.
>
> No, it doesn't, or at least it leaves it up to you. Individuals can
> always make nonglobal things global, but a framework can't make things
> nonglobal once it makes them global.
makes sense.
>
>> generically exposing the settings that were used to invoke a pyramid instance on module level would be nice, but i infer from what you've said the the convenience that would add would be outweighed by the stuff that could possibly go wrong. correct?
>
> The Django core devs I've heard always lament using global settings.
> The zope.testing.setUp stuff that clears a module scope registry is
> another antipattern.
>
>> maybe i could just mock get_current_registry() in my test harness and simply use that in the few places of my code that does not have convenient access to a request.
>
> I dunno. If you want global stuff it's easy enough to make things
> global if you use a little thought. Or just pass the request around.
like i mentioned, i've meanwhile realized that my problem was in poor (or no) design of what needs to be done where. once i re-arranged the furniture, the problem went away.
passing the request around seems the right thing™ to do. at least now i know *why* instead of just blindly cargo-culting :)
cheers,
Malthe Borch
> fwiw: when I started using pyramid I was also reluctant to pass around
> the request all the time. But it turned out to be the right (and
> transparent) thing to do. In my case this is because I often need
> request-specific information like the locale name basically
> everywhere. Once you get used to this (and to creating DummyRequests
> all the time for tests), you will come to like the registry as the
> place to go for global settings.
Another option – in terms of "the right thing" – is to derive an
object that carries just to things you need from the request, e.g.
"locale" and "logged in user", so that you don't pass around an HTTP
request everywhere (and hence need to stub it out in tests).
That object might be an "environment" in which code executes.
\malthe
Mike Orr
> i really would like to have a clean, convenient and supported way to access global settings. i guess, the answer is that pyramid currently doesn't have this on offer.
>
> generically exposing the settings that were used to invoke a pyramid instance on module level would be nice, but i infer from what you've said the the convenience that would add would be outweighed by the stuff that could possibly go wrong. correct?
Pylons had magic globals ('config' -- akin to settings, 'request',
'response', 'session', etc), and they failed in situations like unit
tests or maintenance utilities where you're not in a view called by
the router. Plus they're always a source of potential failure and
hard-to-trace bugs even when they do work most of the time. In short,
it's harder to *guarantee* your application will always be correct
when you're not passing the objects as arguments through to all the
routines that need them. If you pass them all the way through or
attach them to 'self', it leaves a visible trace of how the object got
from one place to another, and who's responsible for modifying it.
With magic globals, this is taken out of your control and you have to
depend on black-box code in the framework, which is harder to test and
guarantee.
A major goal for Pylons 2 was to eliminate the magic globals. Pylons
merged into Pyramid because it fulfilled that goal.
> maybe i could just mock get_current_registry() in my test harness and simply use that in the few places of my code that does not have convenient access to a request.
That's the back door. The 'threadlocals' module gives you the state
objects when you can't access them any other way; e.g., in 'pshell'.
But as the manual says, you should try to use it as little as
possible, because it is essentially a magic global.
--
Mike Orr <slugg...@gmail.com>
Tom Lazar
> Another option – in terms of "the right thing" – is to derive an
> object that carries just to things you need from the request, e.g.
> "locale" and "logged in user", so that you don't pass around an HTTP
> request everywhere (and hence need to stub it out in tests).
FWIW that's what i ended up with, actually. the view extracts the filesystem root setting from the request and then just passes that particular setting to the helper methods that require it. in my testing i create a sandbox and pass *its* location to the helpers. presto, no request involved, no wsgi app instantiated!
> That object might be an "environment" in which code executes.
in my case that's an instance of a 'dropbox' class that receives a root path in its init.
i'm actually rather happy with how it turned out :)
tom
>
> \malthe
Jonathan Vanasco
dearly.
The Pyramid/Repoze group seems very much into the idea of passing a
request around all the time. I don't like that, mostly from the 'mass
appeal route' , in that its something that just about every other
framework has, and can make Pyramid a bit odd/hard to new people
considering it.
mod_perl had an interesting approach to globals.
- it was often recommended that globals were 'read-only' variables, so
you couldn't change them
- the mp architecture was a little weird. the Apache Master server
started up and had the true 'global' vars. Requests would be handled
by spawned children, up to max-requests, which had their own copy of
globals. Anything they altered would stay global only within that
child ( not affect the master or the other children ). because of
tis , applications built on it tended to do a lot of caching and pre-
compiling on startup to build up a true 'global' datastore... but then
basically never think about writing to the globals -- as there was
little efficiency to be had.
Anyways, if you need to do this again:
The Akhet scaffold has an event subscriber that creates a 'g' globals
object to emulate the pylons environment
https://bitbucket.org/sluggo/akhet/src/6c984e2e24cf/akhet/paster_templates/akhet/%2Bpackage%2B/subscribers.py_tmpl
or you could also just have a little convenience method that just
wraps get_current_registry()
def get_current_settings(field=None):
if field:
return get_current_registry().settings[field]
return get_current_registry().settings
Chris McDonough
> I come from Pylons, where we had the g globals object. I miss it
> dearly.
>
> The Pyramid/Repoze group seems very much into the idea of passing a
> request around all the time. I don't like that, mostly from the 'mass
> appeal route' , in that its something that just about every other
> framework has, and can make Pyramid a bit odd/hard to new people
> considering it.
The framework does not require that your application use globals. This
makes it possible to run more than one Pyramid application in the same
Python process, and makes it more likely that Pyramid can be used "like
a library", which makes creating things like a development environment
on top of Pyramid more pleasant.
*You* can use globals if you like, because any nonglobal can be turned
into a global. The inverse is not true, however.
I'd put some well-tested well-understood turn-nonglobals-into-globals
thing in the cookbook if folks agreed on it (that was the intent of
putting the Django-settings-like recipe in there). Pyramid, however,
will never require that any framework user depend upon globals.
- C
Mike Orr
> I come from Pylons, where we had the g globals object. I miss it
> dearly.
>
> The Pyramid/Repoze group seems very much into the idea of passing a
> request around all the time. I don't like that, mostly from the 'mass
> appeal route' , in that its something that just about every other
> framework has, and can make Pyramid a bit odd/hard to new people
> considering it.
Some frameworks have it and some don't. I've already explained why it
caused so much trouble in Pylons.
> Anyways, if you need to do this again:
>
> The Akhet scaffold has an event subscriber that creates a 'g' globals
> object to emulate the pylons environment
> https://bitbucket.org/sluggo/akhet/src/6c984e2e24cf/akhet/paster_templates/akhet/%2Bpackage%2B/subscribers.py_tmpl
>
> or you could also just have a little convenience method that just
> wraps get_current_registry()
> def get_current_settings(field=None):
> if field:
> return get_current_registry().settings[field]
> return get_current_registry().settings
Akhet doesn't have 'g', but you can inject 'g =
request.registry.settings' into the template namespace if you want.
I'm not sure if you misunderstood the file you linked to; the
'renderer_globals' variable there is the template namespace, not a 'g'
variable.
Chris McDonough wrote:
> The framework does not require that your application use globals. This
makes it possible to run more than one Pyramid application in the same
Python process, and makes it more likely that Pyramid can be used "like
a library", which makes creating things like a development environment
on top of Pyramid more pleasant.
Pylons bent over backwards to support multiple Pylons applications in
the same process, and even multiple instances of the same Pylons
application. That's what really turned the globals into deep voodo,
and required StackedObjectProxies to ensure the correct global values
were visible to each request. I thought you were trying to discourage
multiple Pyramid applications in the same process, and that Pyramid
was intended to run as one extensible application rather than multiple
applications side by site.
--
Mike Orr <slugg...@gmail.com>
Chris McDonough
Pyramid also has the same concept of a stack; it's just not implemented
like Pylons' SOPs were. The problem with SOPs was that they were
*importable*. When you use the request or registry in Pyramid as a ,
you have to request it via get_current_request() or
get_current_registry(), and if you try to do that at module scope, you
get to keep both pieces when it breaks. In Pylons, because SOPs were
importable, people had the expectation that they were useful at module
scope, and got wrapped around the axle when they were sometimes not.
> I thought you were trying to discourage
> multiple Pyramid applications in the same process, and that Pyramid
> was intended to run as one extensible application rather than multiple
> applications side by site.
Nah. There's no logical binary distinction there. Currently you can do
it either way, or both. There'd be no reason to not have globals
(except for maybe some idea of cleanliness) if you didn't want to make
it possible to run more than one application per process. I'm not
suggesting it's a "99%" use case, but for what it's worth, my company
already uses it like this in production to save RAM where we have to
support multiple (small) applications.
- C
Jonathan Vanasco
Chris McDonough
It's fine. Nobody can take away your right to rant! That said, nothing
is going to change in Pyramid itself with respect to making things that
are currently local into globals while I still walk upright and breathe.
Locals can be turned into globals. Globals cannot be turned into
locals. Cope! ;-)
- C
Rob Miller
trying to be the same thing that either Django or Rails is trying to be.
Those are what we call "opinionated" frameworks, which make things
easier on less experienced developers by making many choices for them.
This usually comes at a cost; you lose flexibility, and you have to
wrestle w/ the framework when you realize that the choice you want to
make isn't in fact the one your tool has made.
Pyramid has some opinions, sure, but far fewer of them. It's intended to
strike the right balance between flexibility, performance, and ease of
use for more experienced developers. Some people complain about certain
features b/c "I'm never gonna need this, and it confuses me, can't we
just rip it out?" But in every case that feature is there b/c SOMEone
had to solve a hard problem and that was the best way to do it. Ripping
it out would make those hard problems harder to solve. But Pyramid
targets those hard problems, and thus the features stay.
This case w/ globals is similar, although slightly inverted. In many
cases module-level global settings are fine; they make the developers
life a little bit easier, and they don't cause any problems. But in some
cases they *do* cause problems. Real, honest to god developer pain.
Django and Pylons have both hit this in the real world. For Django it's
fine, it's a trade-off that makes sense given their philosophy and their
target audience. For Pyramid it doesn't, b/c Pyramid is just as
interested in reducing developer friction in the hard cases as it is the
easy cases, and so they make a different set of trade-offs.
Does this mean that Pyramid will never be as widely used as Django?
Probably. Does that matter? Nope.
-r
> --
> You received this message because you are subscribed to the Google
> Groups "pylons-discuss" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/pylons-discuss/-/ddSQnMKjZZ4J.
Tom Lazar
Sent from a phone, please excuse the brevity.
Mike Orr
This is a great summary, Rob. Blaise, that second paragraph has good
material for the website or a FAQ entry -- the part about how all the
features are there because somebody needed them, and you may also
someday work on a site that needs them. That's why I'm excited about
all the additional capabilities Pyramid has given to the Pylons world,
capabilities that don't exist in Django or most WSGI frameworks. It
will take time for people to realize how much power Pyramid gives you,
but when they do -- or at least when they have a need for those
capabilities -- I think you'll see Pyramid's "market share" rise
significantly.
I used PHP back when it was transitioning from global scalars to
global dicts. Originally, it created a global scalar for every query
parameter or POST variable. That was ultra-convenient but they
discovered it was a major security hole because users could override
any variable in the site. So they went to a global dicts of params.
Later they found that wasn't convenient enough so they went to
super-globals. Each of these transitions caused programmers to have to
learn a new technique. Or if they didn't, it forced others to read and
maintain their old-style code.
But the lesson for us is that PHP did some horrible things in the
beginning. Yes, that gave a lot of new web programmers an opening, but
was that the best it could have done? Would those progammers have come
in if PHP had been just slightly stricter? Or should they learn a
structured development environment from the get-go? I lean toward the
latter but there's room for multiple views on this.
The point of Pyramid is that it's both a framework and a set of tools.
If you think a framework should have globals and other easy-looking
things, by all means make a framework on top of Pyramid that does
this. Then people can start with the "easy" environment, and drop
down to regular Pyramid when they need extended features.
I'd like to see a Pyramid port of Django. Maybe not 100% compatible,
but enough to be acceptable to a large percentage of Django users.
Because my main goal is interoperability, so that people aren't stuck
in silos. If we can get to a point where people agree to use the core
Pyramid infrastructure, even if they put a totally different front end
on it, that will help the Python web community in the same way that
WSGI and Paste and Setuptools helped it: by providing a common set of
packages that people used, which inherently helped compatibility.
Of course, Twisted will never join Pyramid because it has a different
paradigm, which I don't think can be made compatible. But I may be
wrong about that. I did not foresee the Zope discontents coming back
to Zope technologies, or the Zope fans adopting to the things the
discontents brought with them, but it has happened.
--
Mike Orr <slugg...@gmail.com>