Calculation bug

[Weiguang Hou]

The following calculation result does not match the value calculated by a calculator!

a= 7871572981212338496

b=int(a/4)

print(b)

1967893245303084544 

The result calculated by a  calculator is 1,967,893,245,303,084,624

Something is wrong!

Any one has any idea?

Thanks,

Wei

[Kerwin Sun]

Hi Wei,

This is caused by the precision of float-point number.

You can fix it with `//`. e.g:

a = 7871572981212338496

b = a//4

Thanks,

Kerwin

Weiguang Hou <weigua...@gmail.com> 于2020年8月28日周五 下午1:02写道：

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/37ec1ff7-79d9-46c1-bb3a-549ae819783cn%40googlegroups.com.

[Jason Grout]

This is an issue with python (not jupyter) converting your number a/4 from floating point to integer, resulting in precision loss. See the table at https://docs.python.org/3/library/stdtypes.html#numeric-types-int-float-complex, and in particular footnote (2) and (3).

One way to get the answer is to use python 3 floor division:

a//4

Thanks,

Jason

--

[Weiguang Hou]

Thanks a lot!