Re: Re : Validate that IntegerField is a valid Signed value

[Cal Leeming [Simplicity Media Ltd]]

Hey Mathieu,

Thanks for taking the time to reply. I'm starting to see now why the core devs are reluctant to modify IntegerField.

I'm wondering if maybe Django should have a SignedIntegerField and UnsignedIntegerField as part of the core (for those that wish to have enforced 32-bit integers), with the same INT_MIN and INT_MAX from limits.h (http://en.wikipedia.org/wiki/Limits.h). But there again, would this be considered un-pythonic or against the ethics of Django?

I guess really it should be up to MySQL to have strict mode by default. But, as this is unlikely to happen, could we perhaps consider having a commented out entry in the settings.py file that allows you to set strict mode for all SQL connections? Or, perhaps a documentation change, which explains easily to the user how to do it (Kinda like the storage_engine thing on http://docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables ) .

Let me know your thoughts :)

Cal

On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN <mathieu...@gmail.com> wrote:

Hello,

I'm afraid there isn't such a thing as "a valid signed value", if we're still talking about "size wise".

For django (python), the integer you gave in the ticket is perfectly valid. Here's a way for you to check that :
>>> s = '351760125423456632454565345363453423453465345453'
>>> int(s)
351760125423456632454565345363453423453465345453L

And indeed, an IntegerField validates that the content of the field can be converted to an int this way (check django/forms/fields.py line 230).

So definitely, as Alex pointed, this is an issue on MySQL's side, not Django's.
I believe this can't (shan't?) be fixed at Django's level, as there's no "size" limitation for the IntegerField, as you would have on a CharField with the max_length attribute.

And no, limiting the length of the string won't work, as "2147483647" isn't the same length as "-2147483647", but is the same length as "9999999999" (if we're taking the example of 2^32-1 as the max SIGNED INT value).

my two cents ;)

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Mathieu AGOPIAN]

For reference, this discussion is linked to
http://groups.google.com/group/django-developers/browse_thread/thread/f0b8ddbda03a2d8e
and to the ticket http://code.djangoproject.com/ticket/15923

Cal,

The only thing that i could imagine regarding "fixing" this issue
would indeed be a documentation addition in the MySQL (or MySQLdb)
part of the http://docs.djangoproject.com/en/dev/ref/databases/ page,
explaining the possible issue with integers that are too large to fit
in the appropriate mysql column (though i'm not sure how all this
works, as i'm far from a MySQL or ORM expert).
Specifically, explaining how to set the sql_mode to "traditional" as
explained by kmtracey (http://code.djangoproject.com/ticket/
15923#comment:10) I believe.

Let's see if there's anyone reading this mailing list showing interest
about this, and/or feel free to create a feature request on trac, and
see if it's accepted.

On a side note, as python doesn't seem to have any issue with large
integers, i guess you could subclass the IntegerField, and add to it's
validation a check to see if the resulting integer can be stored in a
32 bits using either a simple comparison, like the following:
-int('1' * 31, 2) < int(field_value) < int('1' * 31, 2)

Mathieu

On Apr 30, 8:31 pm, "Cal Leeming [Simplicity Media Ltd]"

<cal.leem...@simplicitymedialtd.co.uk> wrote:
> Hey Mathieu,
>
> Thanks for taking the time to reply. I'm starting to see now why the core
> devs are reluctant to modify IntegerField.
>
> I'm wondering if maybe Django should have a SignedIntegerField and
> UnsignedIntegerField as part of the core (for those that wish to have
> enforced 32-bit integers), with the same INT_MIN and INT_MAX from limits.h (http://en.wikipedia.org/wiki/Limits.h). But there again, would this be
> considered un-pythonic or against the ethics of Django?
>
> I guess really it should be up to MySQL to have strict mode by default. But,
> as this is unlikely to happen, could we perhaps consider having a commented
> out entry in the settings.py file that allows you to set strict mode for all
> SQL connections? Or, perhaps a documentation change, which explains easily

> to the user how to do it (Kinda like the storage_engine thing onhttp://docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables) .

>
> Let me know your thoughts :)
>
> Cal
>
> On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN

> <mathieu.agop...@gmail.com>wrote:

>
>
>
>
>
>
>
> > Hello,
>
> > I'm afraid there isn't such a thing as "a valid signed value", if we're
> > still talking about "size wise".
>
> > For django (python), the integer you gave in the ticket is perfectly valid.
> > Here's a way for you to check that :
> > >>> s = '351760125423456632454565345363453423453465345453'
> > >>> int(s)
> > 351760125423456632454565345363453423453465345453L
>
> > And indeed, an IntegerField validates that the content of the field can be
> > converted to an int this way (check django/forms/fields.py line 230).
>
> > So definitely, as Alex pointed, this is an issue on MySQL's side, not
> > Django's.
> > I believe this can't (shan't?) be fixed at Django's level, as there's no
> > "size" limitation for the IntegerField, as you would have on a CharField

> > with the *max_length* attribute.

[Cal Leeming [Simplicity Media Ltd]]

Yeah, subclassing sounds the way forward. I was thinking something like:

SignedIntegerField = IntegerField(min_value=–2147483648, max_value=2147483647)

UnsignedIntegerField = IntegerField(min_value=0, max_value=4294967295)

Do you think there would be much chance in having these two field types included in the core? If so, should I create a patch for fields.py and submit??

As for the MySQL strict mode, I'll submit a documentation patch, and see if it gets accepted :)

[Mathieu AGOPIAN]

I really can't tell if those two new fields would be integrated in
core, either wait for feedback here, and/or give it a try with a
ticket (and i'm pretty sure it'll land in "Design Decision Needed").

On May 1, 2:42 pm, "Cal Leeming [Simplicity Media Ltd]"

<cal.leem...@simplicitymedialtd.co.uk> wrote:
> Yeah, subclassing sounds the way forward. I was thinking something like:
>
> SignedIntegerField = IntegerField(min_value=–2147483648, max_value=
> 2147483647)
> UnsignedIntegerField = IntegerField(min_value=0, max_value=4294967295)
>
> Do you think there would be much chance in having these two field types
> included in the core? If so, should I create a patch for fields.py and
> submit??
>
> As for the MySQL strict mode, I'll submit a documentation patch, and see if
> it gets accepted :)
>
> On Sun, May 1, 2011 at 11:58 AM, Mathieu AGOPIAN

> <mathieu.agop...@gmail.com>wrote:

>
>
>
>
>
>
>
> > For reference, this discussion is linked to
>

> >http://groups.google.com/group/django-developers/browse_thread/thread...
> > and to the tickethttp://code.djangoproject.com/ticket/15923

[Cal Leeming [Simplicity Media Ltd]]

Done

[Django] #15939: SignedIntegerField? / UnsignedIntegerField? as part of the core fields.py

[Django] #15940: Patch database documentation to explain why using sql_mode=strict is important