EVOLUTION - Add Field Schema Evolution Support
Ilias Lazaridis
with the implementation of the schema evolution skeleton (which will
contain a functional "Add Field" support).
http://case.lazaridis.com/multi/wiki/DjangoSchemaEvolution
With a very small assistance from the team (mostly subjecting constructs
and of course review of resulting code), I estimate to have a working
solution within 3 days.
My questions to the team is:
Can I have a branch / or sandbox location for this (within the django
SVN), or do I have to setup an own svn?
If it is possible to create a branch, please assign me the username
"lazaridis_com".
Basicly I would prefere to place the code in an own SVN, but there are
several organizational problems (synchronization etc.), which I have not
solved yet. Additionally, the django core developers should have access
to the code, thus they can directly apply the neccessary corrections /
additions.
.
Ilias Lazaridis
> After reviewing the relevant source code base a little, I have started
> with the implementation of the schema evolution skeleton (which will
> contain a functional "Add Field" support).
>
> http://case.lazaridis.com/multi/wiki/DjangoSchemaEvolution
>
> With a very small assistance from the team (mostly subjecting constructs
> and of course review of resulting code), I estimate to have a working
> solution within 3 days.
The "Add Field" functionality is nearly ready.
A small problem:
I am wondering how to retrieve a collection of columns within a table,
whilst using the standard python dbapi2 functionality.
pyodbc has nice functions available:
http://pyodbc.sourceforge.net/docs.html#cursor_columns
http://pyodbc.sourceforge.net/docs.html#catalog
but it looks that they are not pydbapi2 specific, as looking at the
Python dbapi2 doc found on the net, does not show any catalog functions:
http://www.egenix.com/files/python/DatabaseAPI-2.0.html
-
The sqlite3 python bindings do not state anything about catalog
functions (or something similar), too:
http://initd.org/pub/software/pysqlite/doc/usage-guide.html#python-database-api-2-0-compliance
-
I've looked within the django code, but couldn't find a way.
Have I overseen anything, or is this functionality not available?
Can I use alternatively an SQL statement for this?
I am OO, and not familar with SQL, thus I would be very thankful for an
concrete example.
.
James Bennett
> The "Add Field" functionality is nearly ready.
You are aware that a full implementation of schema evolution for
Django was accepted as a Google Summer of Code project, right?
--
"May the forces of evil become confused on the way to your house."
-- George Carlin
Ilias Lazaridis
> On 5/29/06, Ilias Lazaridis <il...@lazaridis.com> wrote:
>> The "Add Field" functionality is nearly ready.
>
> You are aware that a full implementation of schema evolution for
> Django was accepted as a Google Summer of Code project, right?
Yes, I am.
http://case.lazaridis.com/multi/wiki/DjangoSchemaEvolution
.
DavidA
> whilst using the standard python dbapi2 functionality.
Do you mean cursor.description?
cursor = connection.cursor()
cursor.execute('select * from blog_post where 1 = 0')
for col in cursor.description:
print col
Each col is a 7-item sequence containing column name, type_code,
display_size, internal_size, precision, scale, and null_ok)
http://www.python.org/dev/peps/pep-0249/
-Dave
Ilias Lazaridis
Thanks!
Although this cannot be used as the final solution, it keeps me running
now, thus I can test the code.
sqlite3 has problems with this function, it returns only the column name:
http://initd.org/pub/software/pysqlite/doc/usage-guide.html#python-database-api-2-0-compliance
Additionally, it looks that "description" will be empty, if no rows are
returned.
But as said, I can continue with testing now!
.
Ilias Lazaridis
> After reviewing the relevant source code base a little, I have started
> with the implementation of the schema evolution skeleton (which will
> contain a functional "Add Field" support).
>
> http://case.lazaridis.com/multi/wiki/DjangoSchemaEvolution
>
> With a very small assistance from the team (mostly subjecting constructs
> and of course review of resulting code), I estimate to have a working
> solution within 3 days.
>
> My questions to the team is:
>
> Can I have a branch / or sandbox location for this (within the django
> SVN), or do I have to setup an own svn?
It would be nice if I could have an answer on this, thus I can avoid
redundant effort.
-
The simple "ADD FIELD" functionality is ready (draft version, but
functional).
To simplify things, I could provide several patches.
* 1 patch which decouples some existent functionality into a seperate
function
* 1 patch which adds some function
* 1 patch which refactors an existent function
* 1 patch which enlinks the "ADD FIELD" evolution into the syncdb command
But working with patches is terribly inconvenient. I prefere to make
incremental check-in's, thus the development is trackable for other
developers.
It would be nice, if a branch location could be opened, something like
"evolution-draft".
.
oggie rob
Perhaps there would be more confidence in your efforts if it:
a) Wasn't making someone else's work redundant
b) Came from some detailed design
c) Worked on all databases before being checked in
If you posted some designs on your wiki then we can at least critique
it. I don't think you've described what the "Field Add" functionality
is, or how it works, for example.
-rob
Ilias Lazaridis
I do not invest time to design and document something, where the code is
already available and selfdocumenting.
The question was:
" Can I have a branch / or sandbox location for this (within the django
SVN), or do I have to setup an own svn? "
From the SVN _branch_, you can review the code, I can ask the team for
some constructs etc. and complete the skeleton.
As, said, I can alternatively post the patches, but such a development
methodology is not very efficient.
I mean, for what are svn's there?
.
Ilias Lazaridis
> After reviewing the relevant source code base a little, I have started
> with the implementation of the schema evolution skeleton (which will
> contain a functional "Add Field" support).
>
> http://case.lazaridis.com/multi/wiki/DjangoSchemaEvolution
>
> With a very small assistance from the team (mostly subjecting constructs
> and of course review of resulting code), I estimate to have a working
> solution within 3 days.
you can find the full modifications (only within one file) here:
http://code.djangoproject.com/attachment/ticket/2043/simple_evolution_support.diff
Only the patch 2043 must be applied (as it contains all the other 3 patches)
http://code.djangoproject.com/ticket/2040
http://code.djangoproject.com/ticket/2041
http://code.djangoproject.com/ticket/2042
http://code.djangoproject.com/ticket/2043
In order to stabelize/generalize this and to create a basic skeleton, I
would need an SVN location (which I will most possibly provide myself)
and a few answers from the team.
The SoC Project can pickup the results as a starting point (we all know
how difficult it is to _start_ a work) or as a comparison point.
-
Some Remarks:
The most difficult part was to understand the existent source-code.
The source-code would need some refactoring prior to going into deeper work.
The refactoring affects source-code OO, source-code structure/modularity
and the terminology (which is not aligned to existent OO terminology).
The refactoring would allow e.g. the student of the "SoC Evolution
Project" to produce more and better results in the same timeframe, as he
would spend less time to understand the existent code-base.
There is a possibility that such a refactoring could happen in paralell
with the SoC Schema Evolution Project.
If the team is intrested in further elaborations, please let me know.
-