Can someone give a definitive tutorial on how to setup cherrypy with sqlalchemy using SQLite?

[Bao Niu]

Hi! I'm new to cherrypy. I've been reading about how to use cherrypy with sqlalchemy. I've searched on Google but most of the posts are dated way back, like 2010. And it always seems to involve wspbus, which is highly complicated.
I wonder if someone could point out a most updated version of this kind, hopefully simpler than the 2010 version. I'm using SQLite.

Many thanks.

[Sylvain Hellegouarch]

Hi Bao,

I would suggest you to simply copy the code at:

https://bitbucket.org/Lawouach/cherrypy-recipes/src/d140e6da973aa271e6b68a8bc187e53615674c5e/web/database/sql_alchemy/?at=default

This will show you how to use SQLAlchemy properly with CherryPy. Indeed, this post is still relevant today.

http://www.defuze.org/archives/222-integrating-sqlalchemy-into-a-cherrypy-application.html

If, you don't want to use SQLAlchemt you can also read this: http://cherrypydocrework.readthedocs.org/tutorials.html#tutorial-9-data-is-all-my-life

Hope that helps,

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-user...@googlegroups.com.
To post to this group, send email to cherryp...@googlegroups.com.
Visit this group at http://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.

--
- Sylvain
http://www.defuze.org
http://twitter.com/lawouach

[Bao Niu]

Hi Sylvain,

It is amazing! I have been reading that post (http://www.defuze.org/archives/222-integrating-sqlalchemy-into-a-cherrypy-application.html) for about two month trying to figure out how it works, and as soon as I posted my question here, the original author replies! It works like a ESP! That's the beauty of open source community!

Anyway, thank you very much for your reply. I seem to have trouble understanding some basic concept in Cherrypy, for example, you mentioned in your docstring that "the engine *is* a bus", this gave me a lot of puzzle. Could you possibly point out any easy-for-newbie resources that I can learn those "bus", "publising", "subscribe" stuff of Cherrypy? Those words now sound like alien language to me and the documentation itself presumes that I'm already a Cherrypy pro:(

Cheers,
Bao

On Wednesday, April 16, 2014 2:02:53 PM UTC-7, Bao Niu wrote:

Hi! I'm new to cherrypy. I've been reading about how to use cherrypy with sqlalchemy. I've searched on Google but most of the posts are dated way back, like 2010. And it always seems to involve wspbus, which is highly complicated

[Bao Niu]

Specifically, regarding the architecture of the code, why does it take two modules, cherrypy.Tools and cherrypy.process.plugins to fulfil a single task -- attaching a call back to the bus? What different purpose do Tools and plugins serve respectively?

Maybe my question is too junior, I really appreciate a little hint because currently I feel completely lost in this code. Thanks.

[Sylvain Hellegouarch]

On 17 April 2014 22:20, Bao Niu <niub...@gmail.com> wrote:

Specifically, regarding the architecture of the code, why does it take two modules, cherrypy.Tools and cherrypy.process.plugins to fulfil a single task -- attaching a call back to the bus? What different purpose do Tools and plugins serve respectively?

Maybe my question is too junior, I really appreciate a little hint because currently I feel completely lost in this code. Thanks.

They are not compulsory but Tools and Plugins will make your life much easier once you grasp them. Basically, a tool is mostly a function that runs during the request/response handling at a given stage of the request processing's life. It's similar, in psir to Django or WSGI middleware, though the design is different.

Plugins are standalone piece of software that are associated with the application server's life. They aren't even aware of the HTTP stack. They are just functionalities that are available across the whole stack for you to call, whenever you need.

Both complement each other. You will use a plugin to take care of anything that provides some service whereas tools are there to extend the page handler responding to a request.

In this example, the plugin will host the database connections (the sqlalchemy engine) and provide a mechanism to bind a session. The tool will automatically bind a session for each request and attach that session to the request. The session's life is handled by the plugin, the tool simply glue one to the request.

 

I hope this clarifies things a bit.

[Bao Niu]

This is so enlightening! All of a sudden those codes come to life! Thank you Sylvain! Now I have a much better top-down knowledge of how your code works.

Also in your code's docstring, you said "The plugin is registered to the CherryPy engine and therefore is part of the bus (the engine *is* a bus) registery." I don't quite understand why engine is a bus. Is the concept of *bus* uniquely for CherryPy, or it's some generic concept common for http? Is there some source of Cherrypy documentation tailored for beginners?

[Eric Larson]

Sylvain Hellegouarch <s...@defuze.org> writes:

> On 17 April 2014 22:20, Bao Niu <niub...@gmail.com> wrote:
>
>>
>> Specifically, regarding the architecture of the code, why does it take two
>> modules, cherrypy.Tools and cherrypy.process.plugins to fulfil a single
>> task -- attaching a call back to the bus? What different purpose do Tools
>> and plugins serve respectively?
>>
>> Maybe my question is too junior, I really appreciate a little hint because
>> currently I feel completely lost in this code. Thanks.
>>
>>
> They are not compulsory but Tools and Plugins will make your life much
> easier once you grasp them. Basically, a tool is mostly a function that
> runs during the request/response handling at a given stage of the request
> processing's life. It's similar, in psir to Django or WSGI middleware,
> though the design is different.
>
> Plugins are standalone piece of software that are associated with the
> application server's life. They aren't even aware of the HTTP stack. They
> are just functionalities that are available across the whole stack for you
> to call, whenever you need.
>

One thing that helped ease the confusion of the bus and plugins
vs. tools was at what scope do they work within a cherrypy app.

As Sylvain pointed out, if you want to hook into the request / response
cycle then use a Tool. Tools are applied for each request can't keep any
state beyond the request / response.

Plugins, on the other hand, are at the process level. These can manage
state across requests. If you wanted to start some service or create a
pool of workers that stop when the cherrypy server exits, then you want
to use a Plugin.

Going back to SQLAlchemy, assuming I remember the example correctly, the
plugin creates the session and a pool of connections that can be
used. The tool then asks the session for a connection for the life of
the request. This seperation of scope is nice because you have a clear
distinction how you write code. If you are doing something per-request,
use a Tool. If you want to something that lives past a request (like
running other processes), then use a plugin.

Just to give a final example of how we've used plugins, we have test
suite that starts up a cherrypy app that requires a suite of backing
services. Rather than use our test suite to manage this, we have some
plugins that start the supporting services and shut them down when the
tests have finished.

When we needed to use a specialized auth token for our cherrypy apps, we
wrote a tool. The tool could be applied to the URLs we required the
token for and could verify the token on each request.

These kind of examples are what made plugins and tools really make
sense for me. In general, you'll probably have fewer needs for plugins than
tools, but if your application does require a more complex backend,
plugins can be a helpful way to manage your process level requirements.

Best of luck!

Eric

--
Sent with my mu4e

[Bao Niu]

Hi Eric,
Thank you very much for your explanation.
Could you please further explain a bit about "The plugin is registered to the CherryPy engine and therefore is part of the bus (the engine *is* a bus) registery." I don't quite understand why engine is a bus. Is the concept of *bus* uniquely for CherryPy, or it's some generic concept common for http protocol?

[Sylvain Hellegouarch]

Hi Bao,

On 20 April 2014 05:09, Bao Niu <niub...@gmail.com> wrote:

Hi Eric,
Thank you very much for your explanation.
Could you please further explain a bit about "The plugin is registered to the CherryPy engine and therefore is part of the bus (the engine *is* a bus) registery." I don't quite understand why engine is a bus. Is the concept of *bus* uniquely for CherryPy, or it's some generic concept common for http protocol?

The engine didn't have to be a bus. We could have had some sort of Engine entity that internally uses a bus.

The bus is mostly a pubsub implementation, albeit a really simplistic one. All events that the engine or plugins generate or subscribe go through the bus. 

The choice was made that all the application server machinery would also be implemented as consumers/publishers. That's why the engine is just a bus. But because it's the backbone for the entire stack, it has a dedicated name: the engine.

Now, I'm not sure that was the most appropriate term to use overall but, the concepts behind are still straightforward once you understand that CherryPy is built around the pubsub messaging model.

[Bao Niu]

Hi Sylvain,

I am still studying your comment, and not sure if I fully understand the relationship between the Engine instance and the Bus.
In Python3 command line environment I got this:

>>> cherrypy.engine
<cherrypy.process.wspbus.Bus object at 0x7f51564cdb10>

So obviously Engine instances are Bus. But you mentioned that engine only internally uses a bus, which confuses me. Now I'm lost in their convoluted relationship.
Your earlier explanation about the relationship between bus/publisher/subscriber was very clear and easy to understand, could you use similar plain language and magic here again to enlighten me:) Thanks very much.

[Sylvain Hellegouarch]

Hi Bao,

On 28 April 2014 09:29, Bao Niu <niub...@gmail.com> wrote:

Hi Sylvain,

I am still studying your comment, and not sure if I fully understand the relationship between the Engine instance and the Bus.
In Python3 command line environment I got this:

>>> cherrypy.engine
<cherrypy.process.wspbus.Bus object at 0x7f51564cdb10>

So obviously Engine instances are Bus. But you mentioned that engine only internally uses a bus, which confuses me. Now I'm lost in their convoluted relationship.
Your earlier explanation about the relationship between bus/publisher/subscriber was very clear and easy to understand, could you use similar plain language and magic here again to enlighten me:) Thanks very much.

Could you read this article? http://cherrypydocrework.readthedocs.org/extend.html I wrote it yesterday and hopefully it may help.

In any case, just consider that we could have named the engine as follow "cherrypy.bus" and it would have provided the same functionality. In other words, the "engine" variable is a name for the global Bus instance that CherryPy uses. There is no relationship, it's just a name for our bus instance.

 

[Bao Niu]

Thank you Sylvain. It's very interesting resource to read!

--
You received this message because you are subscribed to a topic in the Google Groups "cherrypy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cherrypy-users/mpi58-AbBJU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cherrypy-user...@googlegroups.com.

[Bao Niu]

Hi Sylvain,

I've been studying your code and have some questions here.

#1. why do we import wspbus? It never gets used.

#2. Currently the tool is hooked onto 'on_start_resource', which definitely works. But isn't it more precise to hook it onto 'start_thread' hook? This will only trigger the event when the server starts a new thread?
I really don't see it's necessary to bind the engine to a session each time a request starts. Isn't our purpose here to make sure that sessions are dealt with in a thread-local manner?
(I'm really not quite confident here. Please let me know if I'm missing something important.)

#3. Could you please explain a bit on this line of the code:

        session = cherrypy.engine.publish('bind-session').pop()

In your earlier version it is:

        cherrypy.engine.publish('bind', self.session)

which is, in my opinion much easier to understand. In what consideration did you make that change?

Many thanks.

[Sylvain Hellegouarch]

Hi Bao,

On 3 May 2014 09:58, Bao Niu <niub...@gmail.com> wrote:

Hi Sylvain,

I've been studying your code and have some questions here.

#1. why do we import wspbus? It never gets used.

No reason. Just forgot to remove it I guess.

 

#2. Currently the tool is hooked onto 'on_start_resource', which definitely works. But isn't it more precise to hook it onto 'start_thread' hook? This will only trigger the event when the server starts a new thread?

 

Semantically, it makes more sense to use on_start_resource. The fact the underlying CP stack uses threads is merely an implementation detail. 

 

I really don't see it's necessary to bind the engine to a session each time a request starts. Isn't our purpose here to make sure that sessions are dealt with in a thread-local manner?
(I'm really not quite confident here. Please let me know if I'm missing something important.)

They are managed through thread-local objects (but it's hidden by SQLAlchemy mostly as well as CherryPy).

You don't have to start a session for each request, it's merely a convenience and a showcase. Don't read it too much into it.

 

#3. Could you please explain a bit on this line of the code:

        session = cherrypy.engine.publish('bind-session').pop()

In your earlier version it is:

        cherrypy.engine.publish('bind', self.session)

which is, in my opinion much easier to understand. In what consideration did you make that change?

God... I can't recall the rationale behind that change.  

Sorry,

[Bao Niu]

Ok, so in implementation the tool will only try to combine a new thread-local session IF Cherrypy has started a new thread, or otherwise it just locate the earlier generated session, is my understanding correct? I'm not sure if I got it right here?

On Monday, May 5, 2014 12:00:49 PM UTC-7, Sylvain Hellegouarch wrote:

Hi Bao,

 

#2. Currently the tool is hooked onto 'on_start_resource', which definitely works. But isn't it more precise to hook it onto 'start_thread' hook? This will only trigger the event when the server starts a new thread?

[Bao Niu]

Hi Sylvain,

After studying your plugin for quite a while I still have a very fundamental question.
http://www.defuze.org/archives/222-integrating-sqlalchemy-into-a-cherrypy-application.html

What is the benefit to attaching db_session object to cherrypy.response? Is there any magic to the attaching to cherrypy.response?
        cherrypy.engine.publish('bind-session', self.session)
        cherrypy.request.db_session = self.session
Is there any other solutions that does not store db_session within cherrypy module?

Thanks.