Change proposal: translation config on Meta
Joost Cassee
As I'm thinking about sweeping changes anyway I wanted to share another
idea I've been having.
While looking for a nice admin solution I thought about the ordering of
fields. At the moment any translatable fields are put at the bottom of
the change view. This is a consequence of using inline fieldsets. When
DM has its own real admin form we can think about interspersing
translatable fields with normal fields. That got me thinking about the
models themselves.
Because translatable fields are defined in their own inner class they
always come last in the field order. Of course you can override the
ordering in the ModelAdmin class, but why shouldn't DM work like this:
class Article(models.Model) [or TranslatableModel?]
title = ...
contents = ...
category = ...
creator = ...
created = ...
class Meta:
multilingual_fields = ('title', 'contents')
I feel this is a more intuitive interface. It preserves order and
integrates nicely with other model options. On the other hand, if we are
going to move in the direction of going less implicitly (or to put it
another way: around the back of the user) a Translation class would be
better.
How do the rest of you feel about this?
Regards,
Joost
--
Joost Cassee
http://joost.cassee.net
Panos Laganakos
had agreed it, that it looked nice and clean.
The syntax I had proposed was this:
class MyModel(models.Model):
name = models.CharField(max_length=25, multilingual=True)
desc = models.TextField(blank=True, multilingual=True)
creation_date = models.DateTimeField(...)
In my eyes, this would be ideal, no inner classes, no anything.
The ordering etc, should be handled at the admin model class in NFA.
--
Panos Laganakos
Joost Cassee
> A while back I discussed a different syntax with Michael Trier, and we
> had agreed it, that it looked nice and clean.
>
> The syntax I had proposed was this:
>
> class MyModel(models.Model):
> name = models.CharField(max_length=25, multilingual=True)
> desc = models.TextField(blank=True, multilingual=True)
> creation_date = models.DateTimeField(...)
>
>
> In my eyes, this would be ideal, no inner classes, no anything.
Agreed, this looks even nicer.
I was thinking about a solution that would require no monkey patching.
Fabio mentioned that reducing the feature set (lose 'name_en') would
remove the need to monkey patch the query mechanism. I think that if we
require models with translatable fields to extend some MultilingualModel
class we could pick pick out the fields with multilingual=True before
the Django Model class gets its hands on them. That way we could leave
the original Model.__new__ alone. If DM becomes part of Django proper
the code in MultilingualModel can be folded into Model.
If there are no conceptual objections with this approach I can whip up a
proof of concept.
> The ordering etc, should be handled at the admin model class in NFA.
Sure, but it is just so convenient to have fields in 'standard order' in
the model itself.
Fabio Corneti
>
> I was thinking about a solution that would require no monkey
> patching. Fabio mentioned that reducing the feature set (lose
> 'name_en') would remove the need to monkey patch the query mechanism.
Even if the feature set would be reduced we'd still need to copy and
modify the Query methods in our subclass to support ordering;
I'm trying to find out if a simpler patch for Django could allow to
avoid this, however once 1.0 is out the Query code should not change
that much.
I think that in order to avoid monkey patching it would be necessary
to create a base MultilingualModel class as you proposed earlier in
the list;
given the current rate of brainstorming, maybe a branch would be the
best place for proof of concepts.
Best,
Fabio Corneti
in...@corneti.com
Piotr Majewski
instead of creating Translation tables why not to add extra (dynamic)
colls
the table would look like:
id
name_en
name_de
name_pl
creation_date
this way ordering finding by name_pl will be enabled by default;
and the model would look like:
class MyModel(models.Model):
name = models.CharField(max_length=25, multilingual=True)
the multilingual flag would add to meta extra colls and would reffer
to current language collumn?
what do you think about this solution?
I think it requries less ingerention in django, therefor is better.
installing aditional languages would require ALTER command.
+ Imho this solution would be much more faster due to lack of JOINS
+ ingerention with admin would be easyier
Fabio Corneti
>
> Ok what about difrent solution?
> instead of creating Translation tables why not to add extra (dynamic)
> colls
>
> installing aditional languages would require ALTER command.
> + Imho this solution would be much more faster due to lack of JOINS
> + ingerention with admin would be easyier
There are some issues which need to be considered (and probably have
been already considered by Marcin):
1) If you have to change the type of a field, you need to issue alter
column commands for each language column
2) Some databases put a limit on the absolute number of columns in a
table or on the number of columns of a particular type
3) If you are dealing with Text fields and have many languages or many
fields, you could end up with really large records, which
is usually bad for performance
Best,
Fabio Corneti
in...@corneti.com
Piotr Majewski
into code of some bigger apps i think that "collumn method" is better.
In my opinion "collumn method" would be definitely easier to integrate
with django.
Moreover not many models have more than 10 multilingual fields, and
with 3 (standard) langs that gives us 30 cols. Putting indexes on
those colls would bring more preformance than getting the same
information from joins. Instead of having JOINED 30 rows we have 1
row. Isn't that performance?
> If you have to change the type of a field,
colls, and moreover why to change collumn type?
> Some databases put a limit on the absolute number of columns
postgresql: Maximum number of columns in a table? : 1600
sqlite: (don't know)
what else do we need? oracle isn't fully supported right now AFAIK
>If you are dealing with Text fields and have many languages or many
>fields, you could end up with really large records, which
> is usually bad for performance
the same ammount of data.
so when you compare fetching same ammount of data by doing it by JOINs
and simple collumns the difirence is obvious that "collumn storing" is
better
so IMHO the main problem is adding new languages and modifying the
model.
adding can be handled by extra method altering tables,
modifying, hmm. why to do that? let's first create a good model and
then enter data.
in other case we are left with custom sql, and that is not such a big
problem.
So maybe it's time for DM 2.0 ?
Panos Laganakos
seems to make sense. Can we get a design draft and maybe a proof of
concept?
--
Panos Laganakos
Piotr Majewski
programmer.
the thing i found that can be used to do this is:
>cls._meta.add_field(FIELD)
for adding extra fields
so now only thing we have to do is add multilingual flag to Field
class, and a method that would add extra columns to Model class.
this can be done by changing a bit __init__ of Django's Field class.
It will also require a method that would generate a list of extra
fields in Field class, so only changes needed:
def contribute_to_class:
has to have:
if(self.multilingual):
#gets a tuple/list of generated extra colls
#for each element : cls._meta.add_field(FIELD)
#enable using model.name instead of model.name_en by adding to
class field with db_column of current language
I think that's all
this is so simple that this can be easily integrated to Django core.
Next step is to integrate Admin, but i've no idea how to do that.
Some help?
yml
I coming a bit late to the fiesta but I would like to say that I like
this proposal very much. It sounds great! In order to make it more
explicit I would introduce a new field rather than modifying the
existing "CharField" let us say "MultilingualCharField".
A nice bonus of this is that it might be easier to provide a default
widget that could be use to represent it in both admin and "home made"
interface.
--yml
Marcin Kaszynski
thanks a lot for ideas. I was not able to respond earlier, but I hope
it's not too late to add my 2c.
These are actually two distinct topics:
1. model definition -- the way you specify which fields are
multilingual
2. storage -- whether you create a separate _translation table, or add
fields to the model table, or whatever.
They are somewhat connected, because certain combinations are easier
to implement than others, but for the most part they are separate
topics and require separate discussions.
So:
1. model definition
-------------------
I used the inner Translation class is because it freed me from having
to support fields -- I can have multilingual IntegerFields (you would
be surprised, but they are actually useful), BooleanFields,
YourCustomFieldNooneElseHaveEverSeen and whatever else you might put
in a model. It just works.
For other ideas:
Joost Cassee wrote:
> class Article(models.Model) [or TranslatableModel?]
> title = ...
> contents = ...
> category = ...
> creator = ...
> created = ...
> class Meta:
> multilingual_fields = ('title', 'contents')
Translation class was easier to implement (and still feels cleaner to
me, but that's just a matter of preference). I'm not sure this makes
anything simpler; the multilingual fields do require different
treatment anyway (since there are multiple versions of title for each
article).
Panos Laganakos wrote:
> class MyModel(models.Model):
> name = models.CharField(max_length=25, multilingual=True)
Field.__init__. ModelBase.__new__ happens too late -- it has no way
of intercepting the multilingual parameter before CharField.__init__
chokes on it.
Also, adding parameters to __init__ feels more implicit and "magical"
to me than marking multilingual fields from the outside, like using a
block or the Meta.multilingual_fields list.
yml wrote:
> I would introduce a new field rather than modifying the
> existing "CharField" let us say "MultilingualCharField".
to those which we provide and makes it difficult for people who want
to make their custom fields multilingual.
So, for me the only alternative that is functionally similar to what
we have now is the one suggested by Joost (Meta.multilingual_fields).
But I'm not sure if the difference is noticeable enough to reimplement
it -- what do you guys think?
2. storage
----------
Somewhere in the archive of django-users there is this first
discussion on DM where I wrote about the possibility of having two
backends: one that worked just like DM does now and another, which
would extend the main model table with fields like name_en, name_pl
and so on. Different people had different ideas on what was better
and cleaner, there were also some that did not like the idea of having
to ALTER the database after adding a new language.
The current "backend" makes some things easier; was easier to
implement (well, at least until QSRF; I believe now it would be
different) and to experiment with. It also makes other things
difficult, costs us some performance and all the joins are probably
more difficult to implement after QSRF.
So, it would be cool to check it -- perhaps this change would both
simplify things and give us significant performance boost. If anyone
wanted to experiment with remodeling DM to store data directly in the
model table -- please do, in a branch, so that we can play with it and
see if it works.
Two things to consider: we need constraints (unique and
unique_together) and a way to indicate that, say, English translation
is not present (so that we can fall back to some other version).
Cheers,
-mk
Panos Laganakos
while this is our app :), I think we should ask the Django devs that
are looking into i18n stuff, to get on this topic too.
We can do whatever feels right to DM devs/users, but if we're aiming
of being closer or integrate with Django at one point, it will be
easier to have them on board from this point.
--
Panos Laganakos
Piotr Majewski
multilingual flag aviable:
On 28 Lip, 17:57, Marcin Kaszynski <marcin.kaszyn...@gmail.com> wrote:
> Panos Laganakos wrote:
> > class MyModel(models.Model):
> > name = models.CharField(max_length=25, multilingual=True)
>
> This requires some monkey patching: you have to modify
> Field.__init__. ModelBase.__new__ happens too late -- it has no way
> of intercepting the multilingual parameter before CharField.__init__
> chokes on it.
>
> Also, adding parameters to __init__ feels more implicit and "magical"
> to me than marking multilingual fields from the outside, like using a
> block or the Meta.multilingual_fields list.
we can do the same thing with Field.__init__ here.
we can save old__init__ and in new init do:
def __init__(self, multilingual=False, *kwargs)
self.multilingual= multilingual
old__init__(**kwargs)
or sth.
this way we don't have to do monkey patching of whole Field class.
same thing with contribute_to_class.
this way we will be able to have multilingual all kinds of fields (I
also use bools and integer multilingual fields as you Marcin ;])
yml wrote:
> I would introduce a new field rather than modifying the
> existing "CharField" let us say "MultilingualCharField".
implemented in transdb django project.
> So, for me the only alternative that is functionally similar to what
> we have now is the one suggested by Joost (Meta.multilingual_fields).
> But I'm not sure if the difference is noticeable enough to reimplement
> it -- what do you guys think?
1, add multilingual flag as i said before
2, create "outside the model" a translation class, like:
ArticleTranslation(models.TranslationModel):
.... here the rest of the data.
this way we would achive nice edit_inline feature.
next thing is to add Model.translated_field functionality whitout
monkey patching.
i personlay think that this is more "magic" than column storing.
> 2. storage
> ----------
>
> Somewhere in the archive of django-users there is this first
> discussion on DM where I wrote about the possibility of having two
> backends: one that worked just like DM does now and another, which
> would extend the main model table with fields like name_en, name_pl
> and so on. Different people had different ideas on what was better
> and cleaner, there were also some that did not like the idea of having
> to ALTER the database after adding a new language.
is wrong?
you have to do custom sql when you change your model anyway , and that
is not wrong?
there can be a simple python script created for this operation, even :
manage.py addlanguage english en. what's so wrong with that? , faster,
cleaner, no extra joins and tables, no magic.
> The current "backend" makes some things easier; was easier to
> implement (well, at least until QSRF; I believe now it would be
> different) and to experiment with. It also makes other things
> difficult, costs us some performance and all the joins are probably
> more difficult to implement after QSRF.
>
> So, it would be cool to check it -- perhaps this change would both
> simplify things and give us significant performance boost. If anyone
> wanted to experiment with remodeling DM to store data directly in the
> model table -- please do, in a branch, so that we can play with it and
> see if it works.
functionality of "virtual fields"/"reference fields"
by that i mean that for example Article.body would refer actualy to
Article.translation['en'].body
If i just could write python i would create a proof of concept for
"column storing" and multilingual flag becouse in my idea this is not
very hard to implement , there is no magic in it and it is very simple
idea.
Anyway for those of you that want to join the discusion about
multilingual content :http://groups.google.com/group/django-developers/
browse_thread/thread/170a8a19cfb5bb68
Marcin Kaszynski
> > This requires some monkey patching: you have to modify
> > Field.__init__. ModelBase.__new__ happens too late -- it has no way
> > of intercepting the multilingual parameter before CharField.__init__
> > chokes on it.
>
> > Also, adding parameters to __init__ feels more implicit and "magical"
> > to me than marking multilingual fields from the outside, like using a
> > block or the Meta.multilingual_fields list.
>
> remember you used to rewrite inner lookup method or sth?
> we can do the same thing with Field.__init__ here.
and model definitions. Which does not mean it is a good thing, only
that we found no way to do these things better. In the future we want
_less_ monkey patching, not more.
If we have two quite similar solutions and only one of them requires
monkey patching, then the other loses unless it has some great
advantages. Otherwise it's not worth it.
> ArticleTranslation(models.TranslationModel):
> .... here the rest of the data.
>
> this way we would achive nice edit_inline feature.
edit_inline interface in both cases.
> > Different people had different ideas on what was better
> > and cleaner, there were also some that did not like the idea of having
> > to ALTER the database after adding a new language.
>
> this is stupid, I mean, why the idea of having to ALTER the database
> is wrong?
Also, different people have different ideas on whether the list of
languages is code or data -- it is in the settings file and influences
DM initialization, so I see it as code; but it maps directly to kind-
of-foreign-keys in the database and just begs to be dynamic,
changeable without restarting the server, so a lot of people see it as
data.
Still, I like the idea; this is something I planned to do (as an
option) since I started DM, but never had the time and a good reason
for it (as the main reason would be optimization -- denormalization is
not always a bad thing, sometimes the gains do outweigh its
disadvantages).
-mk
Piotr Majewski
On 28 Lip, 20:12, Marcin Kaszynski <marcin.kaszyn...@gmail.com> wrote:
> How is this better than the inner Translation class? You can have the
> edit_inline interface in both cases.
you always land with translation on the bottom,
if you had seperate Translation class you can use django newforms-
admin options to customize layout.
Piotr Majewski
I belive not, Translations will still be on the bottom,
are you able to change the order of Translated fields?
> 1KPobierz
Joost Cassee
>
> Joost Cassee wrote:
>> class Article(models.Model) [or TranslatableModel?]
>> class Meta:
>> multilingual_fields = ('title', 'contents')
>
> This is actually very similar, the main difference is that the inner
> Translation class was easier to implement (and still feels cleaner to
> me, but that's just a matter of preference). I'm not sure this makes
> anything simpler; the multilingual fields do require different
> treatment anyway (since there are multiple versions of title for each
> article).
I liked this syntax (and that's all it is: different syntax) because 1)
it allow default field ordering and 2) looks like other meta options.
> Panos Laganakos wrote:
>> class MyModel(models.Model):
>> name = models.CharField(max_length=25, multilingual=True)
>
> This requires some monkey patching: you have to modify
> Field.__init__. ModelBase.__new__ happens too late -- it has no way
> of intercepting the multilingual parameter before CharField.__init__
> chokes on it.
>
> Also, adding parameters to __init__ feels more implicit and "magical"
> to me than marking multilingual fields from the outside, like using a
> block or the Meta.multilingual_fields list.
This syntax feels even neater. If DM would be integrated with Django
proper this solution would neatly fold into the Field class.
In general I agree that a Translation inner class conveys that fact that
you're dealing with special cases better. To put it another way: DM must
work pretty transparently if we want to 'earn' less intrusive syntax.
And of course it should really be better to justify backward compatibility.
> yml wrote:
>> I would introduce a new field rather than modifying the
>> existing "CharField" let us say "MultilingualCharField".
>
> This, in turn, limits the list of fields you can mark as multilingual
> to those which we provide and makes it difficult for people who want
> to make their custom fields multilingual.
Seconded.
> 2. storage
> ----------
>
> [...] If anyone wanted to experiment with remodeling DM to store data
> directly in the model table -- please do, in a branch, so that we can
> play with it and see if it works.
That would be great, but I think the whole DM code depends on the
storage paradigm. Wouldn't that mean rewriting pretty much everything
else too?
My intuition says that the normalized storage model is:
- more relational
- more flexible with languages
- faster on sparse data
- faster on larger records with more languages
... while the column model is:
- faster on fewer languages
- faster on dense data
- potentially simpler in implementation
We must keep in mind that we have not had any real tests. Maybe we
should focus on finding a good data set for testing? Anyone have one
lying around? :-)
Joost Cassee
> Is this full suport of django admin?
> I belive not, Translations will still be on the bottom,
> are you able to change the order of Translated fields?
Certainly not, the admin still uses inlines. I'm looking into new ways
of doing the admin. My current thinking veers towards using a custom
MultilingualModelAdmin that overrides get_form() and save().
As for layout, I thought it would be cool to have a small javascript
dropdown selection box that selects the language being edited. (The
project django-i18ndynamic had something similar.) A non-javascript
back-up layout would look a bit like the current layout, but with field
ordering. That would probably look a bit ugly, though.
I am interested in any other admin layout ideas you may have.
Piotr Majewski
can do this :
class MyOtherModelInline(admin.StackedInline):
template = 'my_new_template_tabular.html'
so it would be cool to chose from few kinds of "translation"
templates, so we could have current template , the "javascript"
template and some others.
or can we do this currently inside translation class in admin ?
> 1KPobierz
Piotr Majewski
search_fields does not work ;]
Joost Cassee
> I think that having separate Translation class is better becouse we
> can do this :
>
> class MyOtherModelInline(admin.StackedInline):
> template = 'my_new_template_tabular.html'
Maybe I don't understand you correctly, but I thought the example in
articles/admin.py shows that this can be done. You just define a
Translation(TranslationModelAdmin) inner class. I just tried
formfield_for_dbfield() but any ModelAdmin feature should work.
> oh and about the [...] /articles/admin.py, search_fields does not
> work ;]
Right! :-) I don't think anyone will fix bugs in the current admin code,
though.
Joost Cassee
> I mean, can you define template for translation inner class ? (edit
> inline template)
Should work, try it? (If it doesn't work and the fix takes too much time
I will solve the problem in the new admin templates, though.)
Piotr Majewski
in fact.
Anyway i wrote this simple "lame" proof of concept that my solution
will work!
It needs ofcourse lot of work, but let me just show you how small
ammount of code needs to be written to implement my idea
this is my code of models.py (do syncdb and check that there are 2
columns - text_pl and text_en )
from django.db import models
from django.db.models.fields import CharField
CharField.old__init = CharField.__init__
CharField.multilingual=None
def new__init(self, multilingual=False, **kwagrs):
self.multilingual = multilingual
return self.old__init(**kwagrs)
CharField.__init__ = new__init
CharField.old__contribute_to_class = CharField.contribute_to_class
def new__contribute_to_class(self, cls, name):
if self.multilingual:
print "zajefanie"
self.set_attributes_from_name('text_en')
self.old__contribute_to_class(cls,'text_en')
import copy
self2= copy.copy(self)
self2.set_attributes_from_name('text_pl')
self2.old__contribute_to_class(cls,'text_pl')
CharField.contribute_to_class = new__contribute_to_class
class Test(models.Model):
text= models.CharField(multilingual=True, verbose_name=("The
description"),
max_length=20, null=False,)
I don't know any python so this code looks ugly, but someone can tide
it up and rewrite it again so it works as it should.
the best part is that you don't have to do anything to actual django
code ;]
you just overwrite 2 methods of Field class.
I'm awaiting for your comments and corrected version of that code
guys ;]
> 1KPobierz
Piotr Majewski
overwritten the method.
anyway i have chosen CharField becouse i have no idea how to overwrite
Field class ;], if this can be done , than I think 90% of wokr is
done.
Piotr Majewski
from django.db.models.fields import CharField
CharField.old__init = CharField.__init__
CharField.multilingual=None
def new__init(self, multilingual=False, **kwagrs):
self.multilingual = multilingual
return self.old__init(**kwagrs)
CharField.__init__ = new__init
CharField.old__contribute_to_class = CharField.contribute_to_class
def new__contribute_to_class(self, cls, name):
print "zajefanie"
self.set_attributes_from_name('text_en')
self.old__contribute_to_class(cls,'text_en')
import copy
self2= copy.copy(self)
self2.set_attributes_from_name('text_pl')
self2.old__contribute_to_class(cls,'text_pl')
#if self.choices:
# setattr(cls, 'get_%s_display' % self.name,
curry(cls._get_FIELD_display, field=self))
class Test(models.Model):
text= models.CharField(multilingual=True, verbose_name=("The
description"),
max_length=20, null=False,)
pass
from django.contrib import admin
admin.site.register(Test)
This shows that column storing is the ideal solution for django ;]
(works with django trunk)
Panos Laganakos
--
Panos Laganakos