floating problem of eval()

235 views
Skip to first unread message

oversky

unread,
Nov 13, 2012, 12:43:21 AM11/13/12
to vim_use
I use eval function to do some simple calculations.
Today I found that some calculations do not return correct answers.
For example,

:echo eval('1/2')
0

:echo eval('1/2.0')
0.5

:echo eval('1.0/2')
0.5

:echo eval('1.0/2.0')
0.5

I use vim from http://portableapps.com.
This version is compiled with +float.
Can anyone confirm this with your vim?

Danny Gratzer

unread,
Nov 13, 2012, 12:45:14 AM11/13/12
to vim...@googlegroups.com
These look correct to me, when you do eval("1/2") you are doing integer division, so it should return 0. In every other case you have a floating point so you will be doing floating point division, so you should get .5. Why is this strange?



--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php



--
Danny Gratzer

oversky

unread,
Nov 13, 2012, 1:00:46 AM11/13/12
to vim_use
Thanks for replying.
The eval() is used in a inline calculator function.
It's annoy that I have to remember not to input integer in an
equation,
especially when the equation is pasted from somewhere else.
Is there anyway to force floating calculation?

On 11月13日, 下午1時46分, Danny Gratzer <danny.grat...@gmail.com> wrote:
> These look correct to me, when you do eval("1/2") you are doing integer
> division, so it should return 0. In every other case you have a floating
> point so you will be doing floating point division, so you should get .5.
> Why is this strange?
>
>
>
>
>
>
>
>
>
> On Mon, Nov 12, 2012 at 11:43 PM, oversky <mail...@gmail.com> wrote:
> > I use eval function to do some simple calculations.
> > Today I found that some calculations do not return correct answers.
> > For example,
>
> > :echo eval('1/2')
> > 0
>
> > :echo eval('1/2.0')
> > 0.5
>
> > :echo eval('1.0/2')
> > 0.5
>
> > :echo eval('1.0/2.0')
> > 0.5
>
> > I use vim fromhttp://portableapps.com.

Christian Brabandt

unread,
Nov 13, 2012, 1:28:50 AM11/13/12
to vim...@googlegroups.com
On Tue, November 13, 2012 07:00, oversky wrote:
> Thanks for replying.
> The eval() is used in a inline calculator function.
> It's annoy that I have to remember not to input integer in an
> equation,
> especially when the equation is pasted from somewhere else.
> Is there anyway to force floating calculation?

Add 0.0 to each number.

regards,
Christian

John Beckett

unread,
Nov 13, 2012, 1:45:46 AM11/13/12
to vim...@googlegroups.com
oversky wrote:
> :echo eval('1/2')
> 0

As Danny mentioned, that is not strange as Vim follows the same
concepts as used in the C language (and others) where "1" is an
integer, but "1.0" (both without quotes) is a floating point
number.

You get the same in Python 2.7, where 1/2 also evaluates as 0.

You don't need "eval":
:echo 1.0/2
gives
0.5

If you have numbers as strings in variables, you could convert
them to floats like this:
:let n = "1"
:let d = "2"
:echo str2float(n)/str2float(d)
which gives 0.5.

John

Vlad Irnov

unread,
Nov 13, 2012, 7:55:07 AM11/13/12
to vim...@googlegroups.com
In Python 2.x this is easily solved like this:
>>> 1/2
0
>>> from __future__ import division
>>> 1/2
0.5

Unfortunately, my Vim is not able to import from future:
:py print 1/2
0
:py from __future__ import division
:py print 1/2
0

The workaround I use is to have the code that does calculations in a separate Python module which imports division from future, and then import that module from Vim script. I can post my script for inline calculations with Pythons if someone is interested.

There other advantages in using Python for calculations instead of VimScript.
Integer overflow is less likely to be a problem. This is what I get:
:echo 111111111*111111111
165372529
:py print 111111111*111111111
12345678987654321

The x**y notation is more convenient than pow(x,y).

Regards,
Vlad
Reply all
Reply to author
Forward
0 new messages