Is pylons.config available during tests?
Evgeny
as in upgrade manual.
I'm using pylons.config in one of my functions in lib/helpers.py
And testing it with nosetests.
Just noticed that pylons.config is an empty structure during test.
Is it a bug? Or any way to refer pylons config during tests?
Wyatt Baldwin
Evgeny
and pylons.test.pylonsapp.config is available.
But pylons.config is empty.
Wyatt Baldwin
>
> > On May 29, 5:57 pm, Evgeny <evgeny.tu...@gmail.com> wrote:
>
> > > I'm switching from 0.9.7 to 1.0 and have a question. Changed my code
> > > as in upgrade manual.
>
> > > I'm using pylons.config in one of my functions in lib/helpers.py
> > > And testing it with nosetests.
>
> > > Just noticed that pylons.config is an empty structure during test.
>
> > > Is it a bug? Or any way to refer pylons config during tests?
>
> > Are you using Nose's --with-pylons option?
>
> All functional tests go well, so sure yes.
> and pylons.test.pylonsapp.config is available.
> But pylons.config is empty.
is how I'm doing things. It's a bit different from what's recommended
in the upgrade manual, but avoids certain issues I've had with the
"official" method.
# environment.py
from paste.deploy.converters import asbool
import pylons
def load_environment(global_conf, app_conf):
# ...
# I put the 'testing = true' in my test.ini
config['testing'] = asbool(config.get('testing', 'false'))
if config['testing']:
test_environ = {
# put stuff relevant to your test environ here, if you
need to
}
pylons.config._push_object(config)
url._push_object(URLGenerator(config['routes.map'],
test_environ))
config['test_environ'] = test_environ
return config
# tests/__init__.py
from unittest import TestCase
from pylons import config
import pylons.test
from webtest import TestApp
class BaseTestController(TestCase):
def __init__(self, *args, **kwargs):
wsgiapp = pylons.test.pylonsapp
self.app = TestApp(wsgiapp,
extra_environ=config['test_environ'])
TestCase.__init__(self, *args, **kwargs)
# middleware.py
def make_app(global_conf, full_stack=True, **app_conf):
# ...
# Do NOT add this line right before 'return app':
app.config = config
Mike Orr
> All functional tests go well, so sure yes.
> and pylons.test.pylonsapp.config is available.
> But pylons.config is empty.
Use pylons.test.pylonsapp.config, or self.app.config (in
TestController). The problem is that the special globals are normally
active only during a request.
pylons.url is explicitly pushed in TestController.__init__ to make it
available, and I suppose you could do the same for pylons.config .
--Ben-- This needs to be documented. Everybody has trouble remembering
how to get the config during tests.
You can also execute the null request, "/_test_vars", which updates
the globals and does nothing else.
See http://pylonshq.com/docs/en/1.0/testing/ .
--
Mike Orr <slugg...@gmail.com>
Eugueny Kontsevoy
To work around the bug I have add the following line to environment.py:
pylons.config.update(config)
See the full file here: http://pastie.org/984565
--
Cheers,
ev
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To post to this group, send email to pylons-...@googlegroups.com.
To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Ben Bangert
pylons.config is broken in 1.0 (although some folks on IRC would argue that it's a feature).
To work around the bug I have add the following line to environment.py:
pylons.config.update(config)
See the full file here: http://pastie.org/984565
Eugueny Kontsevoy
Well, anyway - the workaround posted here works fine, but in the future (1.1?) we should consider never leaving pylons.config in this "half-constructed" stage after setup_app().
Cheers,
Ev
The instructions documented here don't work?I can pull config, app_globals, etc. off the response as shown there just fine. That isn't working for other folks?In particular I'm referring to:Which specifically indicates how to get to the Pylons globals for the purpose of getting app_globals or config.I plan on making the test_app setup process also populate the app_globals and config global objects, but that didn't make it for 1.0, most likely it'll be in for 1.0.1 or 1.1. So in the meantime I did try to fully document how to access these things.Cheers,Ben
--
Wyatt Baldwin
> They work, but we needed to have our existing 0.9.7 Pylons applications to
> work and they don't access config objects via TestResponse. In fact 90% of
> our tests are units, i.e. they don't issue request/response, they just do
> "from pylons import config" and test business logic in isolation from
> request/response machinery.
generally shouldn't rely on Web app configuration.
Hans Lellelid
"business logic" can easily depend on configurable parameters. E.g.
I'd like a test to ensure my account gets locked out after X failed
login attempts. X is configurable.
I've griped about this before, but I think the biggest (only?) problem
that comes up repeatedly for us w/ Pylons is the inaccessibility of
these "special" globals from tests (without using response/request
objects). Stacked object proxies. I dream of a day (or a fork) when
these are either a thing of the past, or disabled by default. (Mike
has given me some pointers in the past, but I still maintain that
writing tests shouldn't be so hard. -- and one shouldn't be forced to
use the WebTest framework, which IMO is inadequate for services
applications.)
Hans
Wyatt Baldwin
> On May 30, 5:40 pm, Wyatt Baldwin <wyatt.lee.bald...@gmail.com> wrote:
>
> > On May 30, 2:31 pm, Eugueny Kontsevoy <eugu...@gmail.com> wrote:
>
> > > They work, but we needed to have our existing 0.9.7 Pylons applications to
> > > work and they don't access config objects via TestResponse. In fact 90% of
> > > our tests are units, i.e. they don't issue request/response, they just do
> > > "from pylons import config" and test business logic in isolation from
> > > request/response machinery.
>
> > Going a little off-topic here, but in my view, business logic
> > generally shouldn't rely on Web app configuration.
>
> That's both unhelpful to the discussion and nonsensical.
differently. The way that it might be helpful is that when you run
unit tests against your business logic, if you don't try use
pylons.config, that particular issue would go away (since, presumably,
any necessary config would be pulled from elsewhere).
Depending on the app, this level of separation might not be necessary,
or it might be too complex, but it's not nonsensical.
> In practice
> "business logic" can easily depend on configurable parameters. E.g.
> I'd like a test to ensure my account gets locked out after X failed
> login attempts. X is configurable.
rely on *Web* configuration. Now, what gets defined as "business
logic" versus "Web logic" is certainly debatable (to an extent).
Personally, I wouldn't refer to logging in to a Web app as "business
logic".
That said, I don't really know what Eugeney meant by "business logic",
which is why I just threw out my suggestion as a possible avenue for
exploration.
Ben Bangert
> I've griped about this before, but I think the biggest (only?) problem
> that comes up repeatedly for us w/ Pylons is the inaccessibility of
> these "special" globals from tests (without using response/request
> objects). Stacked object proxies. I dream of a day (or a fork) when
> these are either a thing of the past, or disabled by default. (Mike
> has given me some pointers in the past, but I still maintain that
> writing tests shouldn't be so hard. -- and one shouldn't be forced to
> use the WebTest framework, which IMO is inadequate for services
> applications.)
I've already put a changeset for this for 1.0.1 into the repo. app_globals and config will be completely available during unit tests when using the nose extension (as all Pylons projects do by default). Barring any other bugs reported, I'll release 1.0.1 to address this within the next week.
Cheers,
Ben
Ben Bangert
> I've griped about this before, but I think the biggest (only?) problem
> that comes up repeatedly for us w/ Pylons is the inaccessibility of
> these "special" globals from tests (without using response/request
> objects). Stacked object proxies. I dream of a day (or a fork) when
> these are either a thing of the past, or disabled by default. (Mike
> has given me some pointers in the past, but I still maintain that
> writing tests shouldn't be so hard. -- and one shouldn't be forced to
> use the WebTest framework, which IMO is inadequate for services
> applications.)
While my latest changeset does address the testing of these, there is still the issue of how to handle reading configuration information. Of course, the other real issue is that the INI format is rather limited. I don't think anyone is thrilled with the thought of writing XML files, so I've been looking at YAML as an *optional* configuration format. That is, you could choose to configure your application in pure Python, as it is now, or ask the configuration object to read in a YAML file and configure things appropriately. This will also help Pylons extensibility as extensions that have their own basic config can let you read it into the active Pylons app with just one command.
So, there's the question of loading configuration, which I think I just addressed, and then there's the stacked object proxies (SOP's) themselves. In Pylons 1.1, I plan on making the SOP's fully optional. Until then though, I would definitely like to make it much easier to test Pylons applications. It'd also help for making it easier to run models and such in other non-web scripts.
Cheers,
Ben
Eugueny Kontsevoy
Besides, I find those things (request/response) to be a secondary concern. I like to start every project as a simple console/ncurses app with full test coverage. Request/response and other UI is just a last minute snap-on, so I don't like bending my test code over to carry around UI-specific objects. "pylons.config" is vastly superior.
Just my $0.02
--
ev
Ben Bangert
> Besides, I find those things (request/response) to be a secondary concern. I like to start every project as a simple console/ncurses app with full test coverage. Request/response and other UI is just a last minute snap-on, so I don't like bending my test code over to carry around UI-specific objects. "pylons.config" is vastly superior.
So the question... it obviously is damn useful to have them available. So even in the future when they're optional, I'd still like the option of using them to be fairly 'sane'. I can (and have now) added the setup of these objects to the Pylons nose extension, so at least app_globals and config will be present in tests from 1.0.1 and on.
Of course, after looking at it, I realize that it would've been nice to know when the configuration period has *ended*, since then I know that I can populate the globals. At the moment, there is no definitive way to know when the configuration period of a Pylons app has actually completed. config.init_app is called, but then things, like app_globals, are added afterwards. So while my nose extension should help for testing purposes with the 1.0.1 release, going forward Pylons needs a way to more explicitly declare the *end* of the configuration stage.
This would also help with making the global's more optional, which is part of my Pylons 1.1 intention. With the introduction of events in Pylons 1.1, it'll be fairly easy to make the globals optional, and when using the 'globals extension' or whatever it is named, it can just listen for the 'configuration ended' event, and register the globals for use with scripts/tests.
Cheers,
Ben
Mike Orr
<wyatt.le...@gmail.com> wrote:
> On May 30, 5:43 pm, Hans Lellelid <h...@velum.net> wrote:
>> On May 30, 5:40 pm, Wyatt Baldwin <wyatt.lee.bald...@gmail.com> wrote:
>>
>> > On May 30, 2:31 pm, Eugueny Kontsevoy <eugu...@gmail.com> wrote:
>>
>> > > They work, but we needed to have our existing 0.9.7 Pylons applications to
>> > > work and they don't access config objects via TestResponse. In fact 90% of
>> > > our tests are units, i.e. they don't issue request/response, they just do
>> > > "from pylons import config" and test business logic in isolation from
>> > > request/response machinery.
>>
>> > Going a little off-topic here, but in my view, business logic
>> > generally shouldn't rely on Web app configuration.
>>
>> That's both unhelpful to the discussion and nonsensical.
>
> Well, that's, like, your opinion, man, but I happen to think
> differently. The way that it might be helpful is that when you run
> unit tests against your business logic,
Different people define "business logic" more widely than others, and
I think the individual application also plays a role. Complex
applications, or cases where multiple applications share some code or
data, are examples.
I define business logic as the same as the model, plus whatever extra
code is in lib or in shared modules. I generally don't use 'config'
or any of the magic globals in these places. That's partly because
I've had to port a lot of code from one framework to another over the
years, and it's easier if the business logic doesn't have
framework-dependent parts. So my model never depends on other parts of
Pylons, so that it can be imported standalone for utilities
(maintenance scripts or cron jobs). Of course, the top-level script
has to initialize the model with an engine.
For things like a second database connection for a shared module
(e.g., a stats database), or a search index or such, I put under
'app_globals' for the controllers/templates, but I also try to make it
not dependent on 'app_globals' for testing.
I haven't figured out a way to do comprehensive testing within the
Pylons structure. For instance, how to hide tests that should only be
run on special occasions, or multiple levels of "unit" tests. I find
myself declaring everything in a TestController even if it's a unit
test, in order to have convenient access to the config. Sometimes I
recast my tests as standalone scripts to hide them from the regular
"nosetests". And 'websetup' is not really flexible enough to deal with
all the data situations you might want to test against -- while
remaining compatible with its primary role of setting up the databases
initially. So I'm not really satisfied with the existing test
infrastructure, but I haven't found anything better either. (Although
'twill' works great to quickly test a few URLs.)
>> I've griped about this before, but I think the biggest (only?) problem
>> that comes up repeatedly for us w/ Pylons is the inaccessibility of
>> these "special" globals from tests (without using response/request
>> objects). Stacked object proxies. I dream of a day (or a fork) when
>> these are either a thing of the past, or disabled by default. (Mike
>> has given me some pointers in the past, but I still maintain that
>> writing tests shouldn't be so hard. -- and one shouldn't be forced to
>> use the WebTest framework, which IMO is inadequate for services
>> applications.)
As Ben said, SOPless usage is coming, and I keep bugging him about it.
In frameworks in general, there are three general ways to access the
request's context. One is via a 'request' argument or 'self'
attributes. Quixote 1 had a request argument, while Quixote 2 has
global functions (get_request(), get_response(), get_publisher() [the
framework object]). CherryPy prefers threadlocal globals, which are a
precursor to Pylons' StackedObjectProxies. They are convenient, but
they're also magical, and it ties your code to the framework if you
use them everywhere (including in lib functions and the model).
--
Mike Orr <slugg...@gmail.com>
Evgeny
I've just thought what if I'm unable to get global request/config out
of every piece of my code.
Today I have a helper function that utilizes config, environ, and I
call it from templates
just like ${h.help_me('arg')}.
So do I supposed to call it in 1.1 like
${h.help_me('arg', self.request, self.config,
self.whatever_request_context_I_need_there)}?
instead of importing globals in lib/help_me.py?
I believe the beauty of Pylons and Python, is that it allows a minor
magic in favor of nice readability.
As a framework user, I don't need a pure OOP design of it. I just need
a tool to do the things.
So globals are good in many situations, and request/response/config is
really of nothing magic in them.
They just need to be accessible out of everywhere in app.
Instead, it is much worse to add request/response accessor to every
pylons object that needs access to them.
It adds a real complexity that the framework needs to avoid. Few
simple global objects are preferable to me.
Python is not Java, not "everything is object".
This is why I love Python, and switched to it. It has nice compromises
in its design that helps a lot in daily life.
On May 31, 9:30 am, Mike Orr <sluggos...@gmail.com> wrote:
> On Sun, May 30, 2010 at 6:19 PM, Wyatt Baldwin
>
>
>
Mike Orr
> OK.
> I've just thought what if I'm unable to get global request/config out
> of every piece of my code.
>
> Today I have a helper function that utilizes config, environ, and I
> call it from templates
> just like ${h.help_me('arg')}.
> So do I supposed to call it in 1.1 like
> ${h.help_me('arg', self.request, self.config,
> self.whatever_request_context_I_need_there)}?
> instead of importing globals in lib/help_me.py?
Your function probably doesn't need all of config, but one or two
specific variables. So you'd pass those. And in a template, 'request'
and 'config' are already in the template namespace (that's what the
render function does) so you don't need 'self'.
${paginate.Page(c.records, items_per_page=config["items_per_page"])}
Well, that's a bit contrived because you'd normally pass
items_per_page as a 'c' variable. But that's how you'd pass a config
var to a library routine.
> I believe the beauty of Pylons and Python, is that it allows a minor
> magic in favor of nice readability.
> As a framework user, I don't need a pure OOP design of it. I just need
> a tool to do the things.
That's the counter-argument, and why the SOPs exist.
> So globals are good in many situations, and request/response/config is
> really of nothing magic in them.
The last part is untrue. They're magic because they don't follow the
normal rules of Python, that when you set an attribute, it keeps that
value until you reassign it. The SOPs depend on Pylons pushing and
popping values onto the attributes to match the current request. An
accessor function (such as get_request()) would be non-magical because
it would be obvious to the user that some sort of calculation is being
performed. Instance properties are not equivalent because again, it's
documented that any instance attribute might be a property. But
people do not expect that for module attributes. Indeed, that's why
people are dumbfounded when they get a "no object has been registered
in this thread" error, because it's not a standard part of Python.
The problems with SOPs are precisely what people have described in
this thread: the fact that they don't always work in some edge cases,
and so you sometimes have to get the config another way in certain
situations.
> Python is not Java, not "everything is object".
> This is why I love Python, and switched to it. It has nice compromises
> in its design that helps a lot in daily life.
But Python also believes that simple is better than complex, explicit
is better than implicit, and intuitive is better than magical.
--
Mike Orr <slugg...@gmail.com>
Evgeny
> ${paginate.Page(c.records, items_per_page=config["items_per_page"])}
if it requires several config/request/environ values to be passed.
For real world example of mine.
There's a function to render static content URLs with $
{h.static_url('img.png')}.
This function depends on
config['debug']
config['pylons.paths']['static_files']
environ['HTTP_HOST']
The function is used pretty widely across templates and controllers.
Will the 1.1 version of Pylons force me to pass all those globals as
parameters of static_url?
That is a huge amount of repeating code just to make things look
obvious. Even if I use "c" instead.
I believe there are always should be at least two kinds of well known
globals:
1. Application instance with application-wide globals (app, config,
app_globals).
2. Something with HTTP request (thread) lifecycle (request/response/
c/...).
This dramatically reduces code mess in certain cases.
> But Python also believes that simple is better than complex
--
Evgeny
Ben Bangert
> The function is used pretty widely across templates and controllers.
> Will the 1.1 version of Pylons force me to pass all those globals as
> parameters of static_url?
A radical change like removing globals, is not going to happen to Pylons. Especially not in a minor version increase like 1.0 -> 1.1. The change I plan is merely to make the globals *optional*. That is, for those who don't want to use them, they don't need to be enabled.
They can actually be disabled now by removing the RegistryMiddleware from the middleware.py config, but getting to some things isn't very clean at the moment when this is done. Making it a bit nicer in that case, and consolidating some of the globals is all that is going to occur.
Cheers,
Ben