unexpected behavior for div

248 views
Skip to first unread message

Andy

unread,
Oct 5, 2020, 10:31:08 AM10/5/20
to Kdb+ Personal Developers
q)floor 15%2.5
6
q)15 div 2.5
5

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

q)1.1 xbar 5
5.5

Kimmo Linna

unread,
Oct 5, 2020, 12:58:26 PM10/5/20
to personal...@googlegroups.com

Hi Andy,

 

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

 

Best regards,

 

Kimmo

 

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 590 1074

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegroups.com.

 

Andy

unread,
Oct 5, 2020, 1:41:14 PM10/5/20
to Kdb+ Personal Developers
this is not true. integer division does not mean that the input need to be integer. 

in fact, on the reference page they have examples of using div on non integers such as
q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 
-3 -4 3 2 

Kimmo Linna

unread,
Oct 5, 2020, 1:56:37 PM10/5/20
to personal...@googlegroups.com

Hi Andy,

 

What do you mean? Which is not true?

 

div is for integer division. https://code.kx.com/q/ref/div/

 

q)-7 7 div/:\:-2.5 -2 2 2.5

2  3  -4 -3

-3 -4 3  2

q)-7 7 div/:\:-3 -2 2 3

2  3  -4 -3

-3 -4 3  2

 

If your input is something else it is cast for integer for you.

 

Xbar is working also according documentation. https://code.kx.com/q/ref/xbar/

Andy

unread,
Oct 5, 2020, 2:12:06 PM10/5/20
to Kdb+ Personal Developers
xbar is working? so you're saying that

the result of 1.1 xbar 5  is expected to be 5.5? that's ridiculous.

the kdb documentation is very clear, it says "returns y rounded down to the nearest multiple of x."

Kimmo Linna

unread,
Oct 5, 2020, 2:37:50 PM10/5/20
to personal...@googlegroups.com

Hi Andy,

 

Xbar is working according to documentation.

    1. xbar 5 should give you 5.5

And

q)1.1 xbar til 16

0 1.1 2.2 3.3 4.4 5.5 6.6 7.7 8.8 9.9 11 12.1 13.2 14.3 15.4 16.5

Compared to documentation

q)2.5 xbar til 16

0 0 0 2.5 2.5 2.5 5 5 5 7.5 7.5 7.5 10 10 10 12.5

Andy

unread,
Oct 5, 2020, 2:45:22 PM10/5/20
to Kdb+ Personal Developers
i'm aware that the example in the documentation matches the behavior. but you're missing the point. no reasonable kdb developer would expect this behavior.

the expected output of of 1.1 xbar 5 is 4. the purpose of my post is not to argue with some stranger on the internet, it's to alert kx of a possible bug.

Kimmo Linna

unread,
Oct 5, 2020, 3:50:28 PM10/5/20
to personal...@googlegroups.com

Hi Andy,

 

If I have understood right one of the main purpose of xbar is to use it with dates. To get proper intervals for dates and times. Thus one could say that no reasonable kdb developer would use decimals with xbar.

 

If you really want to look the logic of xbar for decimals you have to look a longer series. For me the following is quite consistent series but I’m just a student, not a developer.

 

    1. xbar 5            0
    2. xbar 5            0
    3. xbar 5            0
    4. xbar 5            0
    5. xbar 5            2.5
    6. xbar 5            3.0
    7. xbar 5            3.5
    8. xbar 5            4.0
    9. xbar 5            4.5
  1. xbar 5            5
    1. xbar 5            5.5
    2. xbar 5            6.0
    3. xbar 5            6.5  

1.4 xbar 5            7.0

1.5 xbar 5            3

1.6 xbar 5            3.2

1.7 xbar 5            3.4

1.8 xbar 5            3.6

1.9 xbar 5            3.8

2.0 xbar 5            4.0

2.1 xbar 5            4.2

2.2 xbar 5            4.4

2.3 xbar 5            4.6

2.4 xbar 5            4.8

2.5 xbar 5            5.0

2.6 xbar 5            2.6

2.7 xbar 5            2.7

2.8 xbar 5            2.8

2.9 xbar 5            2.9

3.0 xbar 5            3.0

3.1 xbar 5            3.1

3.2 xbar 5            3.2

3.3 xbar 5            3.3

3.4 xbar 5            3.4

3.5 xbar 5            3.5

3.6 xbar 5            3.6

3.7 xbar 5            3.7

…

4.8 xbar 5            4.8

4.9 xbar 5            4.9

5.0 xbar 5            5

 

Best regards,

 

Kimmo

 

--

Kimmo Linna
Nihtisalontie 3 as 1

02630  ESPOO
kimmo...@gmail.com
+358 40 xbar 590 1074

 

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 21.45
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

 

i'm aware that the example in the documentation matches the behavior. but you're missing the point. no reasonable kdb developer would expect this behavior.

 

the expected output of of 1.1 xbar xbar 5 is 4. the purpose of my post is not to argue with some stranger on the internet, it's to alert kx of a possible bug.

On Monday, October xbar 5, 2020 at 2:37:50 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

 

Xbar is working according to documentation.

    1. xbar xbar 5 should give you xbar 5.5

And

q)1.1 xbar til 16

0 1.1 2.2 3.3 4.4 xbar 5.5 6.6 7.7 8.8 9.9 11 12.1 13.2 14.3 15.4 16.5

Compared to documentation

q)2.5 xbar til 16

0 0 0 2.5 2.5 2.5 xbar 5 xbar 5 xbar 5 7.5 7.5 7.5 10 10 10 12.5

 

Best regards,

 

Kimmo

--

Kimmo Linna
Nihtisalontie 3 as 1

 

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 21.12
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

 

xbar is working? so you're saying that

 

the result of 1.1 xbar xbar 5  is expected to be xbar 5.5? that's ridiculous.

 

the kdb documentation is very clear, it says "returns y rounded down to the nearest multiple of x."

 

On Monday, October xbar 5, 2020 at 1:56:37 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

 

What do you mean? Which is not true?

 

div is for integer division. https://code.kx.com/q/ref/div/

 

q)-7 7 div/:\:-2.5 -2 2 2.5

2  3  -4 -3

-3 -4 3  2

q)-7 7 div/:\:-3 -2 2 3

2  3  -4 -3

-3 -4 3  2

 

If your input is something else it is cast for integer for you.

 

Xbar is working also according documentation. https://code.kx.com/q/ref/xbar/

 

Best regards,

 

Kimmo

 

--

Kimmo Linna
Nihtisalontie 3 as 1

 

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 20.41
To: Kdb+ Personal Developers
Subject: Re: [personal kdb+] unexpected behavior for div

 

this is not true. integer division does not mean that the input need to be integer. 

 

in fact, on the reference page they have examples of using div on non integers such as

q)-7 7 div/:\:-2.5 -2 2 2.5 
2 3 -4 -3 

-3 -4 3 2 

On Monday, October xbar 5, 2020 at 12:58:26 PM UTC-4 Kimmo Linna wrote:

Hi Andy,

 

According to documentation div is for integer division. I assume 15 div 2.5 equals 15 div 3. If you cast 2.5 to integer you will get 3. If you try 6h$2.5 q returns 3i.

 

Best regards,

 

Kimmo

 

--

Kimmo Linna
Nihtisalontie 3 as 1

 

From: Andy
Sent: maanantai xbar 5. lokakuuta 2020 17.31
To: Kdb+ Personal Developers
Subject: [personal kdb+] unexpected behavior for div

 

q)floor 15%2.5

6

q)15 div 2.5

5

 

seems like kdb is casting the rhs to the type of lhs before doing the divide? this also affects xbar

 

q)1.1 xbar xbar 5

5.5

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/41cb1970-101e-4249-9f24-b691f6492711n%40googlegroups.com.

 

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to personal-kdbpl...@googlegroups.com.

Devon McCormick

unread,
Oct 6, 2020, 3:44:59 AM10/6/20
to personal...@googlegroups.com
I was going to agree with Kimmo that xbar works as documented as I have my own rounding function in J that works in much the same fashion but then I noticed this:

q)0.05 xbar 0 1 2 3 +/:\: 0.04 0.05 0.06    # Round prices to nearest nickel
0 0.05 0.05
1 1.05 1.05
2 2    2.05
3 3    3.05

It's unclear why 1.05<->0.05 xbar 1.05 but 2<->0.05 xbar 2.05 but I suspect floating point imprecision.
However, the case of 5.5<->1.1 xbar 5.5 seems fine since (the integer 5)*1.1->5.5.
  


To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/13413708-9781-4093-9C68-6D69919648B4%40hxcore.ol.


--

Devon McCormick, CFA

Quantitative Consultant


Andy

unread,
Oct 6, 2020, 3:45:04 AM10/6/20
to Kdb+ Personal Developers
the purpose of xbar is to round down. from the kdb docs:

xbar

Round down

x xbar y xbar[x;y]

Where

  • x is a non-negative numeric atom
  • y is numeric or temporal

returns y rounded down to the nearest multiple of x.


although applying it on time columns is a common usage pattern, the point of xbar is to round down. 1.1 xbar 5 does not round down, it cannot be the intended behavior.

Kevin Smyth

unread,
Oct 6, 2020, 4:52:09 AM10/6/20
to personal...@googlegroups.com
Yes it is a floating point precision issue

Switch on max precision to examine the stored values

\P 0

q)1.05
1.05 (this is fine)

q)2.05
2.0499999999999998 (slightly smaller, rounds down to 2)

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/CAGdEmpH4jUacotCqbLm6yqTkC7DNh5xy3BZHTL5EcZmE3y2ASA%40mail.gmail.com.

Kimmo Linna

unread,
Oct 6, 2020, 4:53:32 AM10/6/20
to personal...@googlegroups.com

Hi Devon,

 

That’s true and it’s strange that after 7.05 xbar returns n.05.

 

Best regards,

 

Kimmo

 

--

Kimmo Linna
Nihtisalontie 3 as 1

Andy Sun

unread,
Oct 6, 2020, 9:24:41 AM10/6/20
to personal...@googlegroups.com
the question was not regarding 

1.1 xbar 5.5 being 5.5

it was 

1.1 xbar 5 being 5.5


On Oct 6, 2020, at 3:45 AM, Devon McCormick <devo...@gmail.com> wrote:


You received this message because you are subscribed to a topic in the Google Groups "Kdb+ Personal Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/personal-kdbplus/axQ_vhE8oPU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to personal-kdbpl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/CAGdEmpH4jUacotCqbLm6yqTkC7DNh5xy3BZHTL5EcZmE3y2ASA%40mail.gmail.com.

Ajay Rathore

unread,
Oct 6, 2020, 9:42:35 AM10/6/20
to personal...@googlegroups.com
I agree with Andy, the "issue" seems to be with div, not xbar

q)5 div 1.1
5
q)5 % 1.1
4.545455

The result of div contradicts Kx's documentation :-  "Integer division: returns the greatest whole number that does not exceed x%y"

I would suggest not mixing the types, either use floats or long/ints

q)1.1 xbar 5.0
4.4





To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/4EE4F5E8-C578-4D27-B815-25CD48716FFE%40gmail.com.

Andy

unread,
Oct 6, 2020, 11:57:24 AM10/6/20
to Kdb+ Personal Developers
xbar is implemented using div so it affects both. Ajay brings up a good point. when both the lhs and rhs are float, there are no issues. the issue only arises when types are mixed. I'm quite certain this is NOT the intended behavior.

q)5 div 1.1
5
q)5.0 div 1.1
4f
q)1.1 xbar 5
5.5
q)1.1 xbar 5.0
4.4

Alvi Kabir

unread,
Oct 6, 2020, 2:09:29 PM10/6/20
to personal...@googlegroups.com
You can override div:

q).q.div:{floor x%y}
q)5 div 1.1
4

Kimmo Linna

unread,
Oct 6, 2020, 2:09:45 PM10/6/20
to personal...@googlegroups.com
Hi,

Could it be so that implementation is according the smallest common factor? If one is integer then those  are used(including casting) for div and then multiplication is done by using given x (number). If both numbers are doubles then those numbers are used.

BR,

Kimmo Linna
Nihtisalontie 3 as 1
02630 ESPOO

Andy <iluv...@gmail.com> kirjoitti 6.10.2020 kello 18.57:

xbar is implemented using div so it affects both. Ajay brings up a good point. when both the lhs and rhs are float, there are no issues. the issue only arises when types are mixed. I'm quite certain this is NOT the intended behavior.

Andy Sun

unread,
Oct 6, 2020, 4:21:44 PM10/6/20
to personal...@googlegroups.com
no matter how you look at it, it’s inconsistent with the description from the reference, and the intent of the function. the most likely explanation is that it’s a bug

On Oct 6, 2020, at 2:09 PM, Kimmo Linna <kimmo...@gmail.com> wrote:

Hi,

Alvi Kabir

unread,
Oct 6, 2020, 4:21:48 PM10/6/20
to Kdb+ Personal Developers
Looks like xbar's implementation uses the original 'div' function in k namespace which cannot be changed, so here's the (cleanest imo) workaround if you're always wanting div/xbar to use floor division:


q).q.div:{floor x%y}
q).q.xbar:{x*y div x:$[16h=abs[type x];"j"$x;x]}
q)5 div 1.1
4
q)1.1 xbar 5
4.4


Charles Skelton

unread,
Oct 7, 2020, 9:13:55 AM10/7/20
to personal...@googlegroups.com
andy is right that it's an oversight(/bug) that div casts rhs to lhs type under some circumstances; good advice not to mix types. We'll look into fixing this to something more sensible.

The fp precision issues are compounded by the fact that floor does not use comparison tolerance asof v3.0. The best workaround for xbar with fp input, to my knowledge, is:

q).1 xbar .1 .2 .3 .4 .5 .6 / demonstrate the problem
0.1 0.2 0.2 0.4 0.5 0.5
q)k).q.xbar:{$[9h=t:abs[@x];x*r+y=x*1+r:y div x;x*y div x:$[16h=t;"j"$x;x]]}
q).1 xbar .1 .2 .3 .4 .5 .6
0.1 0.2 0.3 0.4 0.5 0.6 

To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/d7bf647c-2c81-47d3-870e-f7d5570b157fn%40googlegroups.com.


--
Charles G Skelton | Kx | cha...@kx.com

Stephen Taylor

unread,
Oct 7, 2020, 9:18:56 AM10/7/20
to personal...@googlegroups.com
Just about to write to you about this… want anything on code.kx.com/q/ref/div in the interim?


Stephen Taylor | Librarian | Kx | +44 7713 400852 | ste...@kx.com


To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/CANLWSqXZvowxEdeJNe%3DNZJf-_zPs3%3DNOeA0rnyvmBFecbR6iJQ%40mail.gmail.com.

Ajay Rathore

unread,
Oct 7, 2020, 10:33:17 AM10/7/20
to personal...@googlegroups.com
q)k).q.xbar:{$[9h=t:abs[@x];x*r+y=x*1+r:y div x;x*y div x:$[16h=t;"j"$x;x]]}
q)1.4 xbar 6
8.4
q)
q)k).q.xbar:{$[9h=t:abs[@x];x*$[t;y] div x;x*y div x:$[16h=t;"j"$x;x]]}  // with explicit cast
q)1.4 xbar 6
5.6
q)1.6 xbar 6
4.8
q)1.9 xbar 6
5.7
q)1.1 xbar 6
5.5

On Wed, 7 Oct 2020 at 14:13, Charles Skelton <cha...@kx.com> wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/personal-kdbplus/CANLWSqXZvowxEdeJNe%3DNZJf-_zPs3%3DNOeA0rnyvmBFecbR6iJQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages