ZODB transaction callback problem

[Robin Harms Oredsson]

Hi everyone,

I'm encountering problems with transactions when using ZODB. On very rare occasions the primary connection seems to have dumped the transaction manager before running the request callbacks. As far as I understand the code, transaction_manager on the primary connection is set to None once it's finished.

Since I haven't seen this problem before and I've been running ZODB instances with roughly the same code for 5 years I'm guessing something else is new.

Does anyone have any advice on what could be going on?

Some relevant(?) versions:

pyramid_zodbconn 0.7

pyramid_tm 1.0.2

ZODB 4.4.4

transaction 1.7.0

Pyramid 1.7.3

The only pattern I've really found seems to be:

- It always happens on aborts

- It seems to be during some  server load

- It only seems to happen during requests that only read data

If anyone have some insight in this I'd really appreciate it.

All the best,

Robin

ps

Basically logged exceptions will look something like this:

<...>

  File "./eggs/pyramid-1.7.3-py2.7.egg/pyramid/request.py", line 147, in _process_finished_callbacks

    callback(self)

  File "./eggs/pyramid_zodbconn-0.7-py2.7.egg/pyramid_zodbconn/__init__.py", line 56, in finished

    primary_conn.transaction_manager.abort()

AttributeError: 'NoneType' object has no attribute 'abort'

[Mikko Ohtamaa]

Some relevant(?) versions:

pyramid_zodbconn 0.7

pyramid_tm 1.0.2

ZODB 4.4.4

transaction 1.7.0

Pyramid 1.7.3

Are you using gevent?

-Mikko

The only pattern I've really found seems to be:

- It always happens on aborts

- It seems to be during some  server load

- It only seems to happen during requests that only read data

If anyone have some insight in this I'd really appreciate it.

All the best,

Robin

ps

Basically logged exceptions will look something like this:

<...>

  File "./eggs/pyramid-1.7.3-py2.7.egg/pyramid/request.py", line 147, in _process_finished_callbacks

    callback(self)

  File "./eggs/pyramid_zodbconn-0.7-py2.7.egg/pyramid_zodbconn/__init__.py", line 56, in finished

    primary_conn.transaction_manager.abort()

AttributeError: 'NoneType' object has no attribute 'abort'

--
You received this message because you are subscribed to the Google Groups "pylons-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-devel+unsubscribe@googlegroups.com.
To post to this group, send email to pylons...@googlegroups.com.
Visit this group at https://groups.google.com/group/pylons-devel.
For more options, visit https://groups.google.com/d/optout.

--

Mikko Ohtamaa
http://opensourcehacker.com
http://twitter.com/moo9000

[Robin Harms Oredsson]

Nope, and nothing async, longpoll, sockets etc...

Robin

[Jonathan Vanasco]

On Saturday, March 11, 2017 at 10:14:11 AM UTC-5, Robin Harms Oredsson wrote:

The only pattern I've really found seems to be:

- It always happens on aborts

- It seems to be during some  server load

- It only seems to happen during requests that only read data

I don't use ZODB, but I may have an idea on where to look / possible ways to create a repeatable test-case.

I think I've seen similar situations caused by transaction deadlocks or timeouts on readers high-concurrency situations when there is an extended write going on.  

for example, there are 50 connections reading and 1 connection writing.  sometimes the writer would grab a table/row lock for too long, which would cause a pileup of locks on the readers and a few would eventually fail.  under pyramid, the finished callbacks unconditionally fire after a request -- including timeouts and errors where there was never a properly setup transaction or database to cleanup.  So it would try to abort, but it never got started in the first place. 

this may be related to what you're experiencing.

[Robin Harms Oredsson]

Hi, thanks for your reply.

The problem only occurs when serving things that doesn't have any writes. Like static files or JSON structures from Pyramid.

But essentially the problem would be the same. If pyramid_zodbconn expects a transaction manager to exist and it never got attached as a request callback then that problem would occur.

Does anyone know if any new mechanism as been applied around transactions to try to weed out situations where no transaction should be required?

All the best,

Robin

--

You received this message because you are subscribed to the Google Groups "pylons-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-devel+unsubscribe@googlegroups.com.
To post to this group, send email to pylons...@googlegroups.com.
Visit this group at https://groups.google.com/group/pylons-devel.
For more options, visit https://groups.google.com/d/optout.

--

Betahaus
phone: +46 70-333 00 10
web: http://www.betahaus.net