why MySQL DOUBLE is asdecimal=True by default

[mdob]

Just out of curiosity, why it was decided that MySQL DOUBLE, which is approximation, will be presented in ORM as Decimal by default instead of float?

MySQL DOUBLE - https://dev.mysql.com/doc/refman/5.5/en/floating-point-types.html

SQLAlchemy DOUBLE - https://docs.sqlalchemy.org/en/13/dialects/mysql.html#sqlalchemy.dialects.mysql.DOUBLE

Thanks, 

Michal

[Mike Bayer]

no idea, lets do a git blame

it looks like the issue to first add Decimal support was:

https://github.com/sqlalchemy/sqlalchemy/issues/646

it appears to refer to a no longer tracked SVN changeset, the merge
can be seen at https://github.com/sqlalchemy/sqlalchemy/commit/ed4fc64bb0ac61c27bc4af32962fb129e74a36bf,
where quite arbitrarily the asdecimal flag is True for DOUBLE even
though the superclass is Float. That's all 12 years ago.

Let's try to blame for the ".. note" that was added, since someone
seems to have noticed this was wrong about six years ago, seems like
it was in https://github.com/sqlalchemy/sqlalchemy/commit/6b79d2ea7951abc2bb6083b541db0fbf71590dd3
where we made it a lot more explicit how the float coercion occurs.

All we have is a guess. It would have been that I either observed or
guessed that Python's float, at least the "ten digits" that was coming
out by default back on my cPython 2.3 interpreter on my PowerPC mac at
the time, was not accurate enough to represent the values coming back
from a double-precision floating point value.
Trying to see what the likely floating point representation is for
cPython is ...a ha we can do this:

>>> import sys
>>> sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024,
max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021,
min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16,
radix=2, rounds=1)

I have no idea what would have been happening at that time, it was
likely to preserve round trips for the additional digits, but this was
shortsighted because the driver at the time MySQLdb was likely
returning floats anyway, current branch of it does:
https://github.com/PyMySQL/mysqlclient-python/blob/master/MySQLdb/converters.py#L107
so seems like a careless oversight. Sorry!





>
> Thanks,
> Michal
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
> ---
> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.
> To post to this group, send email to sqlal...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.

[Michał Dobrzański]

Interesting. Thank you for the extensive explanation. 

[Jonathan Vanasco]

FWIW, that PR would have been during MySQL5.0 - not 5.5. 

5.0 was released in October 2005; 5.1 was not released until November 2008

assuming the 5.0 docs...

https://docs.oracle.com/cd/E19078-01/mysql/mysql-refman-5.0/data-types.html#numeric-types

looking deeper at their docs, this bit may have influenced someone:

----

10.7. Choosing the Right Type for a Column

For optimum storage, you should try to use the most precise type in all cases. For example, if an integer column is used for values in the range from 1 to 99999, MEDIUMINT UNSIGNED is the best type. Of the types that represent all the required values, this type uses the least amount of storage.

Tables created in MySQL 5.0.3 and above use a new storage format for DECIMAL columns. All basic calculations (+, -, *, and /) with DECIMAL columns are done with precision of 65 decimal (base 10) digits. See Section 10.1.1, “Overview of Numeric Types”.

Prior to MySQL 5.0.3, calculations on DECIMAL values are performed using double-precision operations. If accuracy is not too important or if speed is the highest priority, the DOUBLE type may be good enough. For high precision, you can always convert to a fixed-point type stored in a BIGINT. This enables you to do all calculations with 64-bit integers and then convert results back to floating-point values as necessary.

all that being said, updating mysql was a chore and often scary then, so most linux distributions and dbas held things back and did not upgrade as aggressively as people have the past 5 years. 3.23/4.1 was likely a core target for SqlAlachemy - not 5.x

http://ftp.nchu.edu.tw/MySQL/doc/refman/4.1/en/choosing-types.html

http://ftp.nchu.edu.tw/MySQL/doc/refman/4.1/en/numeric-types.html

[Mike Bayer]

still looks like DOUBLE should be dealing with Python floats.
DOUBLE_PRECISION in the Oracle dialect is also Float and doesn't
default to decimal coercion.

We can fix this problem, it's just one of those changes that you have
to warn users like crazy on and then just do the change on a major
release.