PickleType and MySQL: want mediumblob instead of blob

574 views
Skip to first unread message

soder...@gmail.com

unread,
Dec 13, 2006, 8:45:56 AM12/13/06
to sqlalchemy
Hi,

Using SA 0.3.2 with Python (TurboGears) and MySQL I see that my
PickleType column is stored as a blob in the MySQL table.

Can I change the following table definition so that my 'info' column is
stored as a mediumblob in MySQL instead of a blob?

data_set_table = Table('dataset', metadata,
Column('info', PickleType(), default={})
)

Thanks,
toffe

Michael Bayer

unread,
Dec 13, 2006, 2:27:13 PM12/13/06
to sqlalchemy
im assuming youre talking about the CREATE TABLE statements. make your
pickletype like this:

class MyPickle(PickleType):
impl = mysql.MSMediumBlob

t = Table("tab", meta, Column('data', MyPickle()))

soder...@gmail.com

unread,
Dec 14, 2006, 4:28:09 AM12/14/06
to sqlalchemy
Yes, that was exactly what I wanted!

Thanks a lot!

jose

unread,
Dec 14, 2006, 7:50:31 AM12/14/06
to sqlal...@googlegroups.com
Hi all,

I'm trying to insert UTF8 characters (like this: perchè)
in PostgreSQL but...


File "build/bdist.linux-i686/egg/sqlalchemy/orm/attributes.py", line 589, in __init__
File "build/bdist.linux-i686/egg/sqlalchemy/orm/attributes.py", line 51, in is_equal
File "build/bdist.linux-i686/egg/sqlalchemy/orm/strategies.py", line 30, in <lambda>
File "build/bdist.linux-i686/egg/sqlalchemy/types.py", line 179, in compare_values
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 12: ordinal not in range(128)

jo

Lee McFadden

unread,
Dec 14, 2006, 9:42:30 AM12/14/06
to sqlal...@googlegroups.com
Going from the line number in your exception that would be because
your columns are of type String. Change the columns to Unicode and it
should solve your problem.

foo_table = Table('foo', metadata,
Column('id', Integer, primary_key=True),
Column('bar', Unicode(255))
)


--
Lee McFadden

blog: http://www.splee.co.uk
work: http://fireflisystems.com
skype: fireflisystems

Marco Mariani

unread,
Dec 14, 2006, 10:04:50 AM12/14/06
to sqlal...@googlegroups.com
Lee McFadden wrote:
> Going from the line number in your exception that would be because
> your columns are of type String. Change the columns to Unicode and it
> should solve your problem.
>
> foo_table = Table('foo', metadata,
> Column('id', Integer, primary_key=True),
> Column('bar', Unicode(255))
> )
>

In case you're using reflection:


Overriding Reflected Columns

Individual columns can be overridden with explicit values when
reflecting tables; this is handy for specifying custom datatypes,
constraints such as primary keys that may not be configured within the
database, etc.

>>> mytable = Table('mytable', meta,
... Column('id', Integer, primary_key=True), # override reflected 'id' to have primary key
... Column('mydata', Unicode(50)), # override reflected 'mydata' to be Unicode
... autoload=True)

<http://www.sqlalchemy.org/docs/metadata.myt#metadata>
or use convert_unicode=True in create_engine, as I do, it should help.

In my case, I have UTF8 as default encoding for postgres, and SA
reflects columns as PGString.

jose

unread,
Dec 14, 2006, 11:43:24 AM12/14/06
to sqlal...@googlegroups.com
Marco Mariani wrote:

Ciao Marco,

I replaced line 7 with line 8 in my model but the error is still there...


1 from turbogears import database
2 from sqlalchemy import Table, relation
3 from sqlalchemy.engine import create_engine
4 from sqlalchemy.ext.assignmapper import assign_mapper
5 database.bind_meta_data()
6 session = database.session
7 # engine = database.metadata.engine
8 engine = create_engine("postgres://username:password@localhost/sfera",
convert_unicode=True, encoding='utf-8')
9 context = session.context
...

jo

Michael Bayer

unread,
Dec 14, 2006, 12:04:53 PM12/14/06
to sqlalchemy
let me show you whats happening:

rawdata = 'Alors vous imaginez ma surprise, au lever du jour, quand une
dr\xc3\xb4le de petit voix m\xe2\x80\x99a r\xc3\xa9veill\xc3\xa9. Elle
disait: \xc2\xab S\xe2\x80\x99il vous pla\xc3\xaet\xe2\x80\xa6
dessine-moi un mouton! \xc2\xbb\n'

unicodedata = rawdata.decode('utf-8')

print rawdata == unicodedata

will give you that error message. because you have
"convert_unicode=True", the rows returned from your database will be as
unicode, not str, objects. dealing with raw utf-8 strings on the other
side produces this mismatch. so either dont use
"convert_unicode"/sqlalchemy.types.Unicode or insure that you pass
only unicode() objects to your object properties.

Jose Soares

unread,
Dec 15, 2006, 4:06:25 PM12/15/06
to sqlal...@googlegroups.com
Thanks to Björn, I solved my problem by creating the file
/usr/lib/python2.4/site-packages/sitecustomize.py
like this:

import sys
sys.setdefaultencoding("utf-8")


jo

Michael Bayer ha scritto:

Reply all
Reply to author
Forward
0 new messages