Testing Pyramid Traversal App with ZODB

[Gerhard Schmidt]

Hi,

I'm writing a pyramid app using traversal and ZODB. Right now i writing
unit tests and try to test if the _p_changed attribute is set after a
method that changes a persistent object.

The problem is that _p_changed is always False when running in the
testrunner. Even after I do obj._p_changed = True obj._p_changed is
still False.

In running environment everything works fine. But I often forget the
self._p_changed = True after modifying lists or dicts inside a
persistent object. So i really like to test if it's get set.

Is there a way to test if a persistent object would be saved when the
transaction is committed.

Regards
Estartu

[Thierry Florac]

Hi,

Don't know for the testing environment...

But why don't you use PersistentList / PersistentMapping classes instead of classis list/dict for your inner properties?

Regards,

Thierry

-- 

  https://www.ulthar.net -- http://pyams.readthedocs.io

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/5b2f3caa-0c8e-dabb-bee0-d16868229771%40augusta.de.
For more options, visit https://groups.google.com/d/optout.

[Jens W. Klein]

On 24.02.19 08:48, Gerhard Schmidt wrote:> Hi,

_p_changed is meant as a trigger, but it is not meant to read a status.
Triggering _p_changed joins the object to a transaction manager which
then on commit saves the object.

What your're looking for is probably _p_status which is "saved" or
"unsaved".

For dicts/lists Thierry is right, better use
PersistentList/PersistentDict in order to not to have to trigger
_p_changed manually.

hth Jens
--
Klein & Partner KG, member of BlueDynamics Alliance

[Gerhard Schmidt]

Am 25.02.2019 um 10:51 schrieb Jens W. Klein:
> On 24.02.19 08:48, Gerhard Schmidt wrote:> Hi,
>>
>> I'm writing a pyramid app using traversal and ZODB. Right now i writing
>> unit tests and try to test if the _p_changed attribute is set after a
>> method that changes a persistent object.
>>
>> The problem is that _p_changed is always False when running in the
>> testrunner. Even after I do obj._p_changed = True obj._p_changed is
>> still False.
>>
>> In running environment everything works fine. But I often forget the
>> self._p_changed = True after modifying lists or dicts inside a
>> persistent object. So i really like to test if it's get set.
>>
>> Is there a way to test if a persistent object would be saved when the
>> transaction is committed.
>
> _p_changed is meant as a trigger, but it is not meant to read a status.
> Triggering _p_changed joins the object to a transaction manager which
> then on commit saves the object.

In the normal code I don't read it, but for testing purposes it would be
quite handy.

> What your're looking for is probably _p_status which is "saved" or
> "unsaved".

it's always 'unsaved' while testing, because all objects are brand new
as the database is new for every test. Even after calling
transaction.commit() it's still unsaved. Is there a way to query the
transaction manager if a object is in the to be saved list.

> For dicts/lists Thierry is right, better use
> PersistentList/PersistentDict in order to not to have to trigger
> _p_changed manually.

For most purposes I use PersistenList/Dict, but first there is no
PersistentSet. PersistenList and PeristentDict (especial PeristentDicts
in a PersistentList) tend to fill the ZODB with many small objects and
remove other from the cache. So I use the not persistent ones when they
don't change often (most of the Time never) but even code that called
once a month or once a year has to be tested. And when _p_changed is not
used often the chance of forgetting increases and testing it is rather
useful.

Regards
Estartu

[Jonathan Vanasco]

We use SqlAlchemy but perhaps this will still help:

We test anything dealing with transactions via Functional Tests or sometimes Integrated Tests, not Unit Tests.

For Functional Tests, look for examples in the docs that are based on `from webtest import TestApp`.  I think I have an open source project or two that does this, and can check.

Basically, we  construct the test harness to create the database on setUp, making it available until tearDown.  Then you invoke the logic by a route which calls the function, so a test looks something like this...

    def test_edit(self):

          res_new = self.testapp.get('/item_new', status=200)

          res_item_id = 1  # or regex the item id off the response

          dbItem = self.ctx.dbSession.query(model.Foo).order_by(model.Foo.id.desc()).first()

          assert item_id == dbItem.id

if you're doing an edit flow, then you need to use a new dbSession (or clear the existing one) so the stale data doesn't persist.  This works fine with testing via sqlite, mysql and postgres

            

the general concept though is...

    * test harness creates a new database

    * the app uses the new database
    * queries in the test harness use the new database

we also have some Integrated tests that spin up an instance of the Pyramid app, then make queries against it using a headless browser.

to the best of my knowledge, those are the only ways to really test anything involving the transaction package 

[Jens W. Klein]

On 26.02.19 23:28, Gerhard Schmidt wrote:
> Am 25.02.2019 um 10:51 schrieb Jens W. Klein:
>> On 24.02.19 08:48, Gerhard Schmidt wrote:> Hi,

[...]>>> Is there a way to test if a persistent object would be saved

when the
>>> transaction is committed.
>>
>> _p_changed is meant as a trigger, but it is not meant to read a status.
>> Triggering _p_changed joins the object to a transaction manager which
>> then on commit saves the object.
>
> In the normal code I don't read it, but for testing purposes it would be
> quite handy.
>
>> What your're looking for is probably _p_status which is "saved" or
>> "unsaved".
>
> it's always 'unsaved' while testing, because all objects are brand new
> as the database is new for every test. Even after calling
> transaction.commit() it's still unsaved. Is there a way to query the
> transaction manager if a object is in the to be saved list.

Sorry, I am not sure what it means in testing and if the bahvior here is
different.

>> For dicts/lists Thierry is right, better use
>> PersistentList/PersistentDict in order to not to have to trigger
>> _p_changed manually.
>
> For most purposes I use PersistenList/Dict, but first there is no
> PersistentSet. PersistenList and PeristentDict (especial PeristentDicts
> in a PersistentList) tend to fill the ZODB with many small objects and
> remove other from the cache. So I use the not persistent ones when they
> don't change often (most of the Time never) but even code that called
> once a month or once a year has to be tested. And when _p_changed is not
> used often the chance of forgetting increases and testing it is rather
> useful.

You may want touse the BTrees package https://pypi.org/project/BTrees/
where different kinds of persistent BTrees and persistent Sets are
implemented in a very efficient way (also using C-Extension).
Documentation at https://btrees.readthedocs.io/en/latest/