Breaking out localflavor

[Adrian Holovaty]

I'd like to move all Django localflavor code into a separate package,
distributed separately from Django the framework.

WHY?

1. We shouldn't be in the business of updating Romanian phone number
rules (e.g., https://github.com/django/django/pull/275). That doesn't
belong in a Web framework.

2. The localflavor code makes Django more bloated than it should be,
in terms of number of files (a concern with stuff like App Engine) and
download size (not a huge concern, but still worth improving).

THE HISTORY

On Feb. 14, 2007, I created django.contrib.localflavor because I felt
guilty we had USA-specific form fields.
https://github.com/django/django/commit/27b28616b405578d81fb8e6a4efdc73459c81a2b

Nearly immediately, people saw this and started contributing form
fields and validation logic for their own countries / governments. It
was kind of amazing to watch all of the patches come in, and you could
feel the national pride from contributors all over the world.

In retrospect, setting this expectation and allowing all of this code
into Django proper was a bad idea. We should have created an
infrastructure for a *separate* package (or packages) of
country-specific stuff, not shipping it with Django proper.

PROPOSED SOLUTION

I think it makes most sense for there to be country-specific packages,
such as django-forms-us or django-forms-es, that are distributed
independently. These would be very easy to install (via pip), and
people outside of the Django core team could maintain them and take
full responsibility for them.

Considering backwards-compatibility, we could jumpstart these
country-specific packages by (1) creating them in the first place and
(2) replacing the django.contrib.localflavor code with code that
imports from the correct third-party library (e.g., djangoforms_us),
as a shim. For Django 1.5, we'd tell people to install the appropriate
django-forms-* packages for their sites, and their code referencing
django.contrib.localflavor would still work (with a
DeprecationWarning).

How does this sound? I'm happy to start doing the work myself on Sept.
1 (http://www.holovaty.com/writing/goodbye-everyblock/). :-)

Adrian

[Jacob Kaplan-Moss]

+1!

An added benefit that you didn't mention is easier maintenance -- each
individual localflavor package could have its own maintainer(s),
meaning they wouldn't have to get the attention of a core committer to
get fixes in.

One question: would you be thinking of doing some sort of namespace
package deal so that `django.contrib.localflavor.us.Whatever`
continues to work with an externally-installed `django-localflavor-us`
package, or is part of this requiring users to change imports? I have
no real opinion either way -- that is, I'm asking, not telling
</inside-joke>.

Jacob

> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
>

[Adrian Holovaty]

On Thu, Aug 16, 2012 at 4:33 PM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> An added benefit that you didn't mention is easier maintenance -- each
> individual localflavor package could have its own maintainer(s),
> meaning they wouldn't have to get the attention of a core committer to
> get fixes in.

Totally -- that's huge.

> One question: would you be thinking of doing some sort of namespace
> package deal so that `django.contrib.localflavor.us.Whatever`
> continues to work with an externally-installed `django-localflavor-us`
> package, or is part of this requiring users to change imports? I have
> no real opinion either way -- that is, I'm asking, not telling
> </inside-joke>.

You mean like a magic import thing, right? I think we should force the
issue and require people to change imports -- but they'd have two full
Django point versions to do that, per our deprecation policy. The one
controversial thing would be requiring people to install their needed
django-localflavor-* packages when they upgrade to 1.5.

Adrian

[Russell Keith-Magee]

On Fri, Aug 17, 2012 at 5:26 AM, Adrian Holovaty <adr...@holovaty.com> wrote:
> I'd like to move all Django localflavor code into a separate package,
> distributed separately from Django the framework.

+1. I've had the exact same thought myself over the past couple of
years. My hesitation historically has been the limitations of the
Python packaging environment, but we've pretty much reached the point
with pip et al where that isn't a serious impediment any more.

> PROPOSED SOLUTION
>
> I think it makes most sense for there to be country-specific packages,
> such as django-forms-us or django-forms-es, that are distributed
> independently. These would be very easy to install (via pip), and
> people outside of the Django core team could maintain them and take
> full responsibility for them.

I agree that this is certainly one way that we could address the
problem. However, localflavor isn't just forms. Some of the packages
(US in particular; and I think there's also a patch lurking for AU)
have database models as well. I'd be inclined to keep the
'localflavor' moniker.

Also, from a project management point of view, there's the issue of
translations. At present, there's a single locale directory that
contains all the translations for all the localflavors. If we break
out localflavor into separate installable directories, we probably
don't want to break them into 40 different locale directories. I won't
profess any particular expertise on translations, so there might be an
obvious and elegant solution that I'm missing, but I wanted to flag it
so that it's kept in mind.

Russ %-)

[Alex Gaynor]

I don't think that's correct, they have model *fields*, but no actual models.

 

Russ %-)

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

[Russell Keith-Magee]

On Fri, Aug 17, 2012 at 5:39 AM, Alex Gaynor <alex....@gmail.com> wrote:
>
>
> On Thu, Aug 16, 2012 at 9:38 PM, Russell Keith-Magee
> <rus...@keith-magee.com> wrote:
>>
>> On Fri, Aug 17, 2012 at 5:26 AM, Adrian Holovaty <adr...@holovaty.com>
>> wrote:
>> > PROPOSED SOLUTION
>> >
>> > I think it makes most sense for there to be country-specific packages,
>> > such as django-forms-us or django-forms-es, that are distributed
>> > independently. These would be very easy to install (via pip), and
>> > people outside of the Django core team could maintain them and take
>> > full responsibility for them.
>>
>> I agree that this is certainly one way that we could address the
>> problem. However, localflavor isn't just forms. Some of the packages
>> (US in particular; and I think there's also a patch lurking for AU)
>> have database models as well. I'd be inclined to keep the
>> 'localflavor' moniker.
>

> I don't think that's correct, they have model *fields*, but no actual
> models.

My apologies - I meant model fields, not models. Red-eye induced sleep
deprivation is kicking in :-)

Russ %-)

[Jacob Kaplan-Moss]

On Thu, Aug 16, 2012 at 3:38 PM, Adrian Holovaty <adr...@holovaty.com> wrote:
> You mean like a magic import thing, right?

Well, Python *does* have some namespace package support stuff. It's
something of a mess, with one implementation internally, another in
setuptools, and (IIRC) a new PEP that'll take hold in Python 3 and
clean things up. But yeah, it's a mess, and I'm not sure that
backwards-compatibility here is really *that* big a deal. Changing
imports certainly is pretty easy.

> I think we should force the
> issue and require people to change imports -- but they'd have two full
> Django point versions to do that, per our deprecation policy. The one
> controversial thing would be requiring people to install their needed
> django-localflavor-* packages when they upgrade to 1.5.

I think that's more sane, though I do think 1.5 is too early to pull
the rug out. Perhaps we can do something when we package 1.5 that
includes the broken-out packages, with warnings that
django-localflavor-xx isn't installed and that in 1.6 they'll need to.

Now I'm just thinking out loud though, not sure exactly what the right
balance between doing things right and not pissing off users is.

Jacob

[Alex Gaynor]

On Thu, Aug 16, 2012 at 9:42 PM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:

On Thu, Aug 16, 2012 at 3:38 PM, Adrian Holovaty <adr...@holovaty.com> wrote:
> You mean like a magic import thing, right?

Well, Python *does* have some namespace package support stuff. It's
something of a mess, with one implementation internally, another in
setuptools, and (IIRC) a new PEP that'll take hold in Python 3 and
clean things up. But yeah, it's a mess, and I'm not sure that
backwards-compatibility here is really *that* big a deal. Changing
imports certainly is pretty easy.

http://www.python.org/dev/peps/pep-0420/ is the relevant PEP.

 

>  I think we should force the
> issue and require people to change imports -- but they'd have two full
> Django point versions to do that, per our deprecation policy. The one
> controversial thing would be requiring people to install their needed
> django-localflavor-* packages when they upgrade to 1.5.

I think that's more sane, though I do think 1.5 is too early to pull
the rug out. Perhaps we can do something when we package 1.5 that
includes the broken-out packages, with warnings that
django-localflavor-xx isn't installed and that in 1.6 they'll need to.

Now I'm just thinking out loud though, not sure exactly what the right
balance between doing things right and not pissing off users is.

Jacob

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Łukasz Rekucki]

On 16 August 2012 23:42, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> On Thu, Aug 16, 2012 at 3:38 PM, Adrian Holovaty <adr...@holovaty.com> wrote:
>> You mean like a magic import thing, right?
>
> Well, Python *does* have some namespace package support stuff. It's
> something of a mess, with one implementation internally, another in
> setuptools, and (IIRC) a new PEP that'll take hold in Python 3 and
> clean things up.

It's not only new, but it works
(http://docs.python.org/dev/whatsnew/3.3.html#pep-420-namespace-packages)
;)

On Fri, Aug 17, 2012 at 5:39 AM, Alex Gaynor <alex....@gmail.com> wrote:
>
>
> On Thu, Aug 16, 2012 at 9:38 PM, Russell Keith-Magee
> <rus...@keith-magee.com> wrote:
>>
>>
>> I agree that this is certainly one way that we could address the
>> problem. However, localflavor isn't just forms. Some of the packages
>> (US in particular; and I think there's also a patch lurking for AU)
>> have database models as well. I'd be inclined to keep the
>> 'localflavor' moniker.
>
> I don't think that's correct, they have model *fields*, but no actual
> models.

I'm pretty sure everyone with South will still be pretty unhappy about
updating all their migration files.

--
Łukasz Rekucki

[Adrian Holovaty]

On Thu, Aug 16, 2012 at 4:38 PM, Russell Keith-Magee
<rus...@keith-magee.com> wrote:
> I agree that this is certainly one way that we could address the
> problem. However, localflavor isn't just forms. Some of the packages
> (US in particular; and I think there's also a patch lurking for AU)
> have database models as well. I'd be inclined to keep the
> 'localflavor' moniker.

Ah, yes, the packages should be django-localflavor-* -- good point.

> Also, from a project management point of view, there's the issue of
> translations. At present, there's a single locale directory that
> contains all the translations for all the localflavors. If we break
> out localflavor into separate installable directories, we probably
> don't want to break them into 40 different locale directories. I won't
> profess any particular expertise on translations, so there might be an
> obvious and elegant solution that I'm missing, but I wanted to flag it
> so that it's kept in mind.

I'm hoping Jannis or somebody else with translation expertise will
chime in on this -- I have no idea what the repercussions/solutions
could be.

Adrian

[Donald Stufft]

I could be wrong but offhand to make a namespace package you're

going to need to make a namespace package for everything above it.

So for a namespace at django.contrib.localflavor.* I *think* that django

and django.contrib would both need to be namespace packages as well.

If i'm right about that then https://github.com/django/django/blob/master/django/__init__.py

will need to be moved to somewhere else as well.

[bhuztez]

> So for a namespace at django.contrib.localflavor.* I *think* that django
> and django.contrib would both need to be namespace packages as well.

> If i'm right about that thenhttps://github.com/django/django/blob/master/django/__init__.py

> will need to be moved to somewhere else as well.

Yes, __init__.py need to be moved, but I'd like to see Django break
out all contrib packages as namespace packages.

On 8月17日, 上午6时21分, Donald Stufft <donald.stu...@gmail.com> wrote:
> I could be wrong but offhand to make a namespace package you're
> going to need to make a namespace package for everything above it.
>
> So for a namespace at django.contrib.localflavor.* I *think* that django
> and django.contrib would both need to be namespace packages as well.

> If i'm right about that thenhttps://github.com/django/django/blob/master/django/__init__.py

[Jannis Leidel]

On 16.08.2012, at 23:26, Adrian Holovaty <adr...@holovaty.com> wrote:

> I'd like to move all Django localflavor code into a separate package,
> distributed separately from Django the framework.

+1

> PROPOSED SOLUTION
>
> I think it makes most sense for there to be country-specific packages,
> such as django-forms-us or django-forms-es, that are distributed
> independently. These would be very easy to install (via pip), and
> people outside of the Django core team could maintain them and take
> full responsibility for them.
>
> Considering backwards-compatibility, we could jumpstart these
> country-specific packages by (1) creating them in the first place and
> (2) replacing the django.contrib.localflavor code with code that
> imports from the correct third-party library (e.g., djangoforms_us),
> as a shim. For Django 1.5, we'd tell people to install the appropriate
> django-forms-* packages for their sites, and their code referencing
> django.contrib.localflavor would still work (with a
> DeprecationWarning).

Given the number of packages in the localflavor app I'd rather strongly
recommend making a clear cut here, removing the apps from django/contrib/,
*not* providing a shim around the packages and advise users of the apps
to import it from the new packages instead. Breaking code backwards
compatibility is of course an issue, but the alternative to forever to have
to maintain every localflavor namespace doesn't sound better at all.

Python namespace packages are a mess, implementation wise (e.g. with setuptools
having its own implementation) and usage wise. We'd have to start down that
road for all other packages in contrib, which I'm certain the localflavor
isn't the right reason for. Python's own pkgutil.extend_path isn't useful
enough in every day

Instead let's provide a good working packaging template for
localflavor app maintainers (I volunteer to provide it if nobody else does)
that uses the names:

- a fake namespace django_localflavor_<countrycode> as the package (the directory)
- django-localflavor-<countrycode> as the distribution name (in setup.py)

Jannis

[Jannis Leidel]

This can be handled as we did it with the core translation, splitting it
up. It's a bit of work, but totally doable. The basic idea is to copy the
main localflavor po file to each sub app and run makemessages on it again.
gettext will drop the unneeded message IDs (from the other localflavors)
when specifying the --no-obsolete option.

Jannis

[Jannis Leidel]

Updating the South migration files is an unfortunate requirement, but also
something that can be done with a simple search/replace when migration to a
newer Django version.

Jannis