pyramid_tm - is there an acceptable way to unjoin the transaction and use explicit commits?

[Jonathan Vanasco]

one of my apps uses pyramid_tm, but has reached a point where one (possibly two) views (out of dozens) need to make explicit commits with the SqlAlchemy session.  

does anyone know if it's currently possible to handle this?  i have a few apps that only integrate pyramid_tm on a small subset of views, but i haven't tried to bypass it on a single view in many years.  I'm trying to avoid a secondary database connection, because this app can support a sqlite database and I've had problems with too many connections and sqlite in the past.

[Mikko Ohtamaa]

one of my apps uses pyramid_tm, but has reached a point where one (possibly two) views (out of dozens) need to make explicit commits with the SqlAlchemy session.  

Because this does not fit well in the view/request model, I would usually try to offload these tasks to a worker processes (Celery) where it is much easier to have a explicit control over the transaction cycle. A request would only serve as a trigger for the background process. It does not work for all kind of use cases, though.

-Mikko

[Michael Merickel]

pyramid_tm does have a configurable activate_hook that you can use to turn off the TM for certain endpoints. You would need to coordinate that with something else that managed the transaction. If I were doing it myself I'd probably configure my activate hook to return true/false based on an custom attribute on the request like `request.activate_tm`. I'd then set this to False in a tween mounted over pyramid_tm. The reason I'd do it this way is because if you're turning off pyramid_tm then you probably still need to do something to manage the connection - and a tween would be a good spot to do this.

That being said, I'd ask why you would want to turn it off instead of just committing multiple times? You just need to use `request.tm.commit()`, `request.tm.abort()` and `request.tm.begin()` instead of `request.dbsession.commit()`. Make sure to always call begin after commit/abort and things should work fine - whether it's a good idea or not in general (see Mikko's comments) is a separate question.

- Michael

On Thu, Feb 6, 2020 at 7:03 PM Jonathan Vanasco <jona...@findmeon.com> wrote:

one of my apps uses pyramid_tm, but has reached a point where one (possibly two) views (out of dozens) need to make explicit commits with the SqlAlchemy session.  

does anyone know if it's currently possible to handle this?  i have a few apps that only integrate pyramid_tm on a small subset of views, but i haven't tried to bypass it on a single view in many years.  I'm trying to avoid a secondary database connection, because this app can support a sqlite database and I've had problems with too many connections and sqlite in the past.

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/c3d3f561-8bcc-4c20-b1e8-f2aa147483f0%40googlegroups.com.

--

Michael

[Jonathan Vanasco]

`request.tm.commit()`, `request.tm.abort()` and `request.tm.begin()` instead of `request.dbsession.commit()`. 

Thanks, Michael. This obvious answer is what I'll use.

I am very familiar with Celery and task runners, and have a few app using it with Pyramid.

In this particular application, the work/results are all instant - there are no potentially long-running subprocesses.  I just need to commit multiple times within a view to ensure the database records an action was performed.

[Mike Orr]

I sometimes use explicit commits to log usage stats, which I want to
succeed even if the request later aborts. I use a separate database
connection not attached to the transaction. I tried to use pyramid_tm
commit multiple times but it seemed it could only be used once. So
you're supposed to use transaction.commint/abort followed immediately
by transaction.begin? That wasn't clear in the docs.

I've also used a custom transaction adapter to get a Redis session
attached to the transaction. Since Redis is non-transactinal, t saves
its pending work in the object, and on commit it writes it to the
database.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwENeR4239BK2cNYTudNn28-uiDTkWQX%3D4GhkqgwrwNG_g%40mail.gmail.com.



--
Mike Orr <slugg...@gmail.com>

[Michael Merickel]

It’s not recommended. But it’s there for you to use if you want. When you do a commit you’re crossing transaction boundaries to continue the request and all of the checks you did early on are not guaranteed any longer to be valid.

Using a separate connection is the best and most generally sound way to go for recording data not specific to the resource-level goals of processing the request. 

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Duq890OGO-xVWqx0Dcu_kBeoNqBb%3DfbiwnWZ6xZAUZRz1g%40mail.gmail.com.

--

Michael