ZODB transaction callback problem

21 views
Skip to first unread message

Robin Harms Oredsson

unread,
Mar 11, 2017, 10:14:11 AM3/11/17
to pylons...@googlegroups.com
Hi everyone,

I'm encountering problems with transactions when using ZODB. On very rare occasions the primary connection seems to have dumped the transaction manager before running the request callbacks. As far as I understand the code, transaction_manager on the primary connection is set to None once it's finished.

Since I haven't seen this problem before and I've been running ZODB instances with roughly the same code for 5 years I'm guessing something else is new.

Does anyone have any advice on what could be going on?

Some relevant(?) versions:
pyramid_zodbconn 0.7
pyramid_tm 1.0.2
ZODB 4.4.4
transaction 1.7.0
Pyramid 1.7.3


The only pattern I've really found seems to be:
- It always happens on aborts
- It seems to be during some  server load
- It only seems to happen during requests that only read data

If anyone have some insight in this I'd really appreciate it.

All the best,
Robin

ps
Basically logged exceptions will look something like this:

<...>
  File "./eggs/pyramid-1.7.3-py2.7.egg/pyramid/request.py", line 147, in _process_finished_callbacks
    callback(self)
  File "./eggs/pyramid_zodbconn-0.7-py2.7.egg/pyramid_zodbconn/__init__.py", line 56, in finished
    primary_conn.transaction_manager.abort()
AttributeError: 'NoneType' object has no attribute 'abort'

Mikko Ohtamaa

unread,
Mar 11, 2017, 12:02:55 PM3/11/17
to pylons...@googlegroups.com

Some relevant(?) versions:
pyramid_zodbconn 0.7
pyramid_tm 1.0.2
ZODB 4.4.4
transaction 1.7.0
Pyramid 1.7.3

Are you using gevent?

-Mikko



The only pattern I've really found seems to be:
- It always happens on aborts
- It seems to be during some  server load
- It only seems to happen during requests that only read data

If anyone have some insight in this I'd really appreciate it.

All the best,
Robin

ps
Basically logged exceptions will look something like this:

<...>
  File "./eggs/pyramid-1.7.3-py2.7.egg/pyramid/request.py", line 147, in _process_finished_callbacks
    callback(self)
  File "./eggs/pyramid_zodbconn-0.7-py2.7.egg/pyramid_zodbconn/__init__.py", line 56, in finished
    primary_conn.transaction_manager.abort()
AttributeError: 'NoneType' object has no attribute 'abort'

--
You received this message because you are subscribed to the Google Groups "pylons-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-devel+unsubscribe@googlegroups.com.
To post to this group, send email to pylons...@googlegroups.com.
Visit this group at https://groups.google.com/group/pylons-devel.
For more options, visit https://groups.google.com/d/optout.



--

Robin Harms Oredsson

unread,
Mar 11, 2017, 12:46:37 PM3/11/17
to pylons...@googlegroups.com
Nope, and nothing async, longpoll, sockets etc...
Robin

Jonathan Vanasco

unread,
Mar 23, 2017, 2:25:51 PM3/23/17
to pylons-devel


On Saturday, March 11, 2017 at 10:14:11 AM UTC-5, Robin Harms Oredsson wrote:
The only pattern I've really found seems to be:
- It always happens on aborts
- It seems to be during some  server load
- It only seems to happen during requests that only read data

I don't use ZODB, but I may have an idea on where to look / possible ways to create a repeatable test-case.

I think I've seen similar situations caused by transaction deadlocks or timeouts on readers high-concurrency situations when there is an extended write going on.  

for example, there are 50 connections reading and 1 connection writing.  sometimes the writer would grab a table/row lock for too long, which would cause a pileup of locks on the readers and a few would eventually fail.  under pyramid, the finished callbacks unconditionally fire after a request -- including timeouts and errors where there was never a properly setup transaction or database to cleanup.  So it would try to abort, but it never got started in the first place. 

this may be related to what you're experiencing.

Robin Harms Oredsson

unread,
Mar 24, 2017, 11:25:55 AM3/24/17
to pylons...@googlegroups.com
Hi, thanks for your reply.

The problem only occurs when serving things that doesn't have any writes. Like static files or JSON structures from Pyramid.
But essentially the problem would be the same. If pyramid_zodbconn expects a transaction manager to exist and it never got attached as a request callback then that problem would occur.

Does anyone know if any new mechanism as been applied around transactions to try to weed out situations where no transaction should be required?

All the best,
Robin


--
You received this message because you are subscribed to the Google Groups "pylons-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-devel+unsubscribe@googlegroups.com.
To post to this group, send email to pylons...@googlegroups.com.
Visit this group at https://groups.google.com/group/pylons-devel.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages