[Announce] Version 0.6.0 - QLime redefined

[Shalabh Chaturvedi]

QLime is now just an object oriented data access library. The
configuration piece and integration with Quixote have been removed. The
result is a product with reduced scope and greater focus making it is
easier to understand, use and support. The website has a fresh new look
as well - http://www.qlime.org/.

It still works pretty well with Quixote - which, incidentally, is what
the new site is built on.

The data access piece is pretty much intact (with minor enhancements
and bug fixes). The Python API turns out to be *much* easier to use.
Check out the online examples: http://www.qlime.org/example.rst

Questions and comments welcome.

Cheers,
Shalabh

[chri...@gmail.com]

Hi Shalabh
I stumbled across your ORM yesterday while i was being fustrated by
SQLObject
Also have you noticed that its has problems with Postresql
authentication with passwords.
I don't like how SQLObject whats to create your tables and manage
everything for you
I work with time series and the raw data are stored in Postgres and i
have a complex of calendering and numpy based classes that i use to
materialize the raw data into a timeseries object.
I havn't done it yet but i reccon all i have to do is to rewrite two of
methods
get _resultset and execute?? to use Qlime and i am done. Of course i've
got to use the filters etc. instead of the straight SQL queries but
thats relatively straight forward
That said keep up the good work.
I am sure i will have lots of questions once i get going
Regards
Chris

Of course i need to do the usual updates,inserts and delele also.
I was looking for an ORM integrate with my stuff since i am doing lots
of webased front ends recently (currently with Spyce but i may switch
to Turobgears sans SQLlobject)

[Shalabh Chaturvedi]

On Nov 22, 2005, at 5:54 AM, chri...@gmail.com wrote:

>
> Hi Shalabh
> I stumbled across your ORM yesterday while i was being fustrated by
> SQLObject
> Also have you noticed that its has problems with Postresql
> authentication with passwords.
> I don't like how SQLObject whats to create your tables and manage
> everything for you
> I work with time series and the raw data are stored in Postgres and i
> have a complex of calendering and numpy based classes that i use to
> materialize the raw data into a timeseries object.
> I havn't done it yet but i reccon all i have to do is to rewrite two of
> methods
> get _resultset and execute?? to use Qlime and i am done.

Could you describe in more detail what you are trying to do? In general
you do not have to implement anything at the resultset and execute
level just to get data from the database.

> Of course i've
> got to use the filters etc. instead of the straight SQL queries but
> thats relatively straight forward
> That said keep up the good work.
> I am sure i will have lots of questions once i get going

Let me know how it goes.

[iGL]

Hi Shalabh,

Firstly, respects to you: qlime seems to be excellent, well, at least,
exactly what I would like :)

Just a question. If I didn't miss something, qlime works with the
tables, which are already in the DB? Is there a way to create table
from within the qlime's DataObject? The point is that seems to be
simple. The simplest way would be to use the docstring of this class,
e.g., like the following:

class aData( DataObject ):
'''CREATE TABLE PHONE (NAME VARCHAR(10), NUMBER VARCHAR(10));'''
pass

with some 'create' classmethod added to DataObject:
def create( cls, cxn ):
if cls.__doc__ != '':
cls._createStmt = cls.__doc__
c = cxn.conn.cursor()
try:
c.execute( cls._createStmt )
except:
cxn.conn.rollback()
cls.__created = False
else:
cxn.conn.commit()
del c

create = classmethod ( create )

This could be used in the way as below:
if __name__ == '__main__':
from qlime import pgsql
conn = pgsql.DSConn(dbname='test', username='tester',
password='guessit' )

class Phones( DataObject ):
'''CREATE TABLE PHONES (NAME VARCHAR(10), NUMBER VARCHAR(10));'''
pass

Phones.create( conn )

This creates Phones table in the DB 'test'. However, the version of
'create' relies on rollback if the table already exists. Probably,
QLime has a way to overcome it.

Greetings,
Giorgi

P.S. DSConn should probably be equiped with 'exec_from_sql_file'
method: just for the seek of convenience...

[Shalabh Chaturvedi]

On Dec 2, 2005, at 9:20 AM, iGL wrote:

>
> Hi Shalabh,
>
> Firstly, respects to you: qlime seems to be excellent, well, at least,
> exactly what I would like :)

Thanks for trying it out!

> Just a question. If I didn't miss something, qlime works with the
> tables, which are already in the DB?

Yes - you didn't miss anything.

> Is there a way to create table
> from within the qlime's DataObject?

No because, among other things, that would make the DataObject a
database specific class.

Hmm.. perhaps all you need is an execute() method on the conn object to
easily call whatever sql you want. Right now the DataObject is
completely independent of the backend it is loaded from (could be a CSV
file, or an object database in the future). In this context the SQL
does not always make sense. Also different databases (Postgres, SQLite)
have different syntax for some types.

Typically you would do this before you connect the class to the
obectclass. For example:

###
from phone import Phone
from qlime import pgsql

conn = pgsql.DSConn(dbname='test', username='tester', password='test')

if 'public.phones' not in conn.get_obclass_names():
# table doesn't exist, create it
conn.execute('''CREATE TABLE public.phones (NAME VARCHAR(10),
NUMBER VARCHAR(10));''')
conn.refresh_schema()

Phone.connect_to(conn['public.phones'])

###

Does this look better to you? Note that the database specific code (the
pgsql.DSConn class as well as the Postgres-valid SQL) is all in one
place now.

Cheers,
Shalabh

> P.S. DSConn should probably be equiped with 'exec_from_sql_file'
> method: just for the seek of convenience...

If I add the execute(), you could just do
conn.execute(open('test.sql').read()). The execute() method definitely
seems like a good idea.

[iGL]

Hi Shalabh,

Thank you for the reply.

>Hmm.. perhaps all you need is an execute() method on the conn object to
>easily call whatever sql you want. Right now the DataObject is
>completely independent of the backend it is loaded from (could be a CSV
>file, or an object database in the future). In this context the SQL
>does not always make sense. Also different databases (Postgres, SQLite)
>have different syntax for some types.

Yes, after the previeous posting, I kept playing with qlime and came to
the conclusion that any data-source-specific stuff should not be part
of core.- and rdb.DataObject. On another hand, there's always a
trade-off: if one sacrifies portability across different platforms,
he'd like to gain something as well. Therefore, pgsql (firebird) might
have pgDataObject (fbDataObject) class with some db-specific methods
and attributes.

>Does this look better to you? Note that the database specific code (the
>pgsql.DSConn class as well as the Postgres-valid SQL) is all in one
>place now.

Yes, that's certainly nice.

> If I add the execute(), you could just do conn.execute(open('test.sql').read()).

Would be nice, again.

Greetings,
Giorgi

P.S. I guess pgsql.DSCOnn should redefine rdb.DSConn's __getitem__ in a
way that one could look up for the ObjectClass without explicit
'public.'
###
def __getitem__(self, obclass):
obclass='public.'+obclass
return self.self.obclasses[obclass]
###

This would increase the code's portability.

[Shalabh Chaturvedi]

On Dec 3, 2005, at 5:18 AM, iGL wrote:
>
>> If I add the execute(), you could just do
>> conn.execute(open('test.sql').read()).
> Would be nice, again.
>

I just noticed that execute() already exists! In fact there are three
execute methods on the connection object:

execute(stmt, *l) # returns open cursor after execution
execute_and_close(stmt, *l) # executes, closes the cursor, returns
nothing
execute_select(stmt, *l) # returns an iterable over result set

You probably want to use execute_and_close() for creating the table.

> P.S. I guess pgsql.DSCOnn should redefine rdb.DSConn's __getitem__ in a
> way that one could look up for the ObjectClass without explicit
> 'public.'
> ###
> def __getitem__(self, obclass):
> obclass='public.'+obclass
> return self.self.obclasses[obclass]
> ###
>
> This would increase the code's portability.
>

Good suggestion - this also makes it easier to use. I've added this
feature in (but in such a way that even get_obclass_names() will return
both names for the table). You can get the latest code from the public
subversion repository if you run:

svn co svn://qlime.org/qlime/trunk qlime

The code in subversion also adds SQLite support, removes Firebird
support and adds psycopg support.

Cheers,
Shalabh