Any examples

[Juju]

Dear list,

Has anyone ever used SQLAlchemy query as data producer ?
I'm looking for examples, can anybody share about it ?

Regards

[Jonathan Schemoul]

Are you meaning in a type code producer, or with a plugin ?

I've done some specific plugins already for some sqlalchemy models, so that can be done easily.
To prototype you can play from a code producer. Just yield your items directly from it.

To get objects you can inteact with without any call to the database you can either run them through a packet encoder (specifying the fields you want to keep), or use the SAObjectScrubber (it takes sqlalchemy objects and put them in an anonymous objects without relation to the database, for reporting for example).

Anyway, if you plan to keep your items, I recommend to preload any relations to avoid making subsequent requests to the DB. (eager_load SA option).

Jon

[Juju]

C'est intéressant Jonathan. Tu pourrais poster des exemples de codes
correspondants ?
Mon cas précis est surtout de pouvoir traiter de gros jeu de données
sous forme d 'un flux, pour d'évidentes raisons d'économie mémoire.

On 22 nov, 12:34, Jonathan Schemoul <jonathan.schem...@gmail.com>
wrote:

[Mathieu Bridon]

On Tue, 2011-11-22 at 06:24 -0800, Juju wrote:
> C'est intéressant Jonathan. Tu pourrais poster des exemples de codes
> correspondants ?
> Mon cas précis est surtout de pouvoir traiter de gros jeu de données
> sous forme d 'un flux, pour d'évidentes raisons d'économie mémoire.

In a code producer:

----------
from myapp.model import session, MyModel

def produce():
query = session.query(MyModel)
for item in query:
yield item
----------

It really doesn't get any simpler than that, it's just a basic
SQLAlchemy request.

Items will be flowed to the next component in your tube.

--
Mathieu