[vim/vim] Add support for bitwise left/right shift functions (#8457)

166 views
Skip to first unread message

Yegappan Lakshmanan

unread,
Jun 25, 2021, 11:31:54 PM6/25/21
to vim/vim, Subscribed

The following bitwise functions are currently supported: and(), or(), xor() and invert().
But there is no function or operator for bitwise left/right shift.
Add the lshift() and rshift() functions for bitwise left shift and right shift.


You can view, comment on, or merge this pull request online at:

  https://github.com/vim/vim/pull/8457

Commit Summary

  • Add support bitwise left/right shift functions

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Yegappan Lakshmanan

unread,
Jun 26, 2021, 12:04:18 AM6/26/21
to vim/vim, Push

@yegappan pushed 1 commit.


You are receiving this because you are subscribed to this thread.

View it on GitHub or unsubscribe.

codecov[bot]

unread,
Jun 26, 2021, 12:06:44 AM6/26/21
to vim/vim, Subscribed

Codecov Report

Merging #8457 (815f53a) into master (6582e23) will decrease coverage by 87.21%.
The diff coverage is 0.00%.

Current head 815f53a differs from pull request most recent head da21dcf. Consider uploading reports for the commit da21dcf to get more accurate results
Impacted file tree graph

@@             Coverage Diff             @@

##           master    #8457       +/-   ##

===========================================

- Coverage   89.71%    2.50%   -87.22%     

===========================================

  Files         149      147        -2     

  Lines      167841   162656     -5185     

===========================================

- Hits       150584     4068   -146516     

- Misses      17257   158588   +141331     
Flag Coverage Δ
huge-clang-none ?
huge-gcc-none ?
huge-gcc-unittests 2.50% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/evalfunc.c 0.00% <0.00%> (-95.84%) ⬇️
src/float.c 0.00% <0.00%> (-98.91%) ⬇️
src/digraph.c 0.00% <0.00%> (-97.78%) ⬇️
src/gui_gtk_f.c 0.00% <0.00%> (-97.54%) ⬇️
src/match.c 0.00% <0.00%> (-97.13%) ⬇️
src/crypt_zip.c 0.00% <0.00%> (-97.06%) ⬇️
src/sha256.c 0.00% <0.00%> (-96.94%) ⬇️
src/evalbuffer.c 0.00% <0.00%> (-96.83%) ⬇️
src/cmdhist.c 0.00% <0.00%> (-96.63%) ⬇️
src/debugger.c 0.00% <0.00%> (-96.62%) ⬇️
... and 136 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6582e23...da21dcf. Read the comment docs.

Dominique Pellé

unread,
Jun 26, 2021, 5:58:41 AM6/26/21
to vim/vim, Subscribed

@dpelle requested changes on this pull request.


In runtime/doc/eval.txt:

> @@ -7175,6 +7177,15 @@ log10({expr})						*log10()*
 <
 		{only available when compiled with the |+float| feature}
 
+lshift({expr1}, {expr2})					*lshift()*
+		Bitwise left shift {expr1} by {expr2}. The arguments must be
+		Numbers.  Other types of arguments cause an error.

Usually bit shifts in C are done with unsigned (although it's also possible with signed).
Vim script does not have unsigned.

We should document how it behaves with the sign bit. In other words, what is the result of lshift(-2, 1)?
And we should test with negative numbers too.

In Java, left-shift has 2 different operators >> (sign extension) and >>> (shifts a zero into the leftmost position). See: https://docs.oracle.com/javase/tutorial/java/nutsandbolts/op3.html So maybe we need an extra (optional?) parameter to indicate whether to extend sign or not?


In src/evalfunc.c:

> @@ -6189,6 +6195,21 @@ f_line2byte(typval_T *argvars UNUSED, typval_T *rettv)
 #endif
 }
 
+/*
+ * "lshift()" function
+ */
+    static void
+f_lshift(typval_T *argvars, typval_T *rettv)
+{
+    if (argvars[0].v_type != VAR_NUMBER || argvars[1].v_type != VAR_NUMBER)
+    {
+	emsg(_(e_invarg));
+	return;
+    }
+    rettv->vval.v_number = tv_get_number(&argvars[0])
+				<< tv_get_number(&argvars[1]);

This will result in UB (undefined behavior) if we do e.g. lshift(1, 100) as in C, it is UB to bit-shifts by more bits than the number of bits in the number.

In Vim script, we don't want such undefined behavior. I'd say lshift(1, 100) should return 0, and we must thus return 0 if argvars[1] is too high. And we can add a test for bit shift with large number of bits.

We can verify that there is no runtime error using ubsan.

Also, what should happen when shifting by a negative number of bits? E.g. lshift(1, -1) In C, it is undefined behavior but again, we don't want UB in Vim script.

Bram Moolenaar

unread,
Jun 26, 2021, 6:04:51 AM6/26/21
to vim/vim, Subscribed


Yegappen wrote:

> The following bitwise functions are currently supported: and(), or(),
> xor() and invert().
> But there is no function or operator for bitwise left/right shift.
> Add the lshift() and rshift() functions for bitwise left shift and
> right shift.

I'm wondering if it is worth it. You can also use "* 2" for left shift
and "/ 2" for right shift. It's not exactly the same (considering
overflow), but does that matter in a Vim script?


--
My sister Cecilia opened a computer store in Hawaii.
She sells C shells by the seashore.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Yegappan Lakshmanan

unread,
Jun 26, 2021, 10:53:40 AM6/26/21
to vim_dev, reply+ACY5DGG7G63OVDV44P...@reply.github.com, vim/vim, Subscribed
Hi Bram,

On Sat, Jun 26, 2021 at 3:04 AM Bram Moolenaar <vim-dev...@256bit.org> wrote:


Yegappen wrote:

> The following bitwise functions are currently supported: and(), or(),
> xor() and invert().
> But there is no function or operator for bitwise left/right shift.
> Add the lshift() and rshift() functions for bitwise left shift and
> right shift.

I'm wondering if it is worth it. You can also use "* 2" for left shift
and "/ 2" for right shift. It's not exactly the same (considering
overflow), but does that matter in a Vim script?



Without these functions, to left/right shift a number, you need to 
do something like the following:

Left Shift:
let c = a * float2nr(pow(2, b))

Right Shift:
let c = float2nr(floor(a / pow(2, b)))

I thought it would be simpler to have a separate function for this.

Regards,
Yegappan

vim-dev ML

unread,
Jun 26, 2021, 10:53:57 AM6/26/21
to vim/vim, vim-dev ML, Your activity

Hi Bram,

On Sat, Jun 26, 2021 at 3:04 AM Bram Moolenaar ***@***.***>
wrote:


>
> Yegappen wrote:
>
> > The following bitwise functions are currently supported: and(), or(),
> > xor() and invert().
> > But there is no function or operator for bitwise left/right shift.
> > Add the lshift() and rshift() functions for bitwise left shift and
> > right shift.
>
> I'm wondering if it is worth it. You can also use "* 2" for left shift
> and "/ 2" for right shift. It's not exactly the same (considering
> overflow), but does that matter in a Vim script?
>
>
>
Without these functions, to left/right shift a number, you need to
do something like the following:

Left Shift:
let c = a * float2nr(pow(2, b))

Right Shift:
let c = float2nr(floor(a / pow(2, b)))

I thought it would be simpler to have a separate function for this.

Regards,
Yegappan

Yegappan Lakshmanan

unread,
Jun 26, 2021, 11:06:01 AM6/26/21
to vim/vim, vim-dev ML, Comment

@yegappan commented on this pull request.


In runtime/doc/eval.txt:

> @@ -7175,6 +7177,15 @@ log10({expr})						*log10()*
 <
 		{only available when compiled with the |+float| feature}
 
+lshift({expr1}, {expr2})					*lshift()*
+		Bitwise left shift {expr1} by {expr2}. The arguments must be
+		Numbers.  Other types of arguments cause an error.

Thanks for the review. Yes. We can add an option to rshift() to deal with sign extension.


In src/evalfunc.c:

> @@ -6189,6 +6195,21 @@ f_line2byte(typval_T *argvars UNUSED, typval_T *rettv)
 #endif
 }
 
+/*
+ * "lshift()" function
+ */
+    static void
+f_lshift(typval_T *argvars, typval_T *rettv)
+{
+    if (argvars[0].v_type != VAR_NUMBER || argvars[1].v_type != VAR_NUMBER)
+    {
+	emsg(_(e_invarg));
+	return;
+    }
+    rettv->vval.v_number = tv_get_number(&argvars[0])
+				<< tv_get_number(&argvars[1]);

I will add a check for argvars[1] and return 0 if it is too large. If the shift value is negative, we can return the original value.


You are receiving this because you commented.

Bram Moolenaar

unread,
Jun 26, 2021, 1:26:29 PM6/26/21
to vim...@googlegroups.com, vim-dev ML

Yegappen wrote:

> > > The following bitwise functions are currently supported: and(), or(),
> > > xor() and invert().
> > > But there is no function or operator for bitwise left/right shift.
> > > Add the lshift() and rshift() functions for bitwise left shift and
> > > right shift.
> >
> > I'm wondering if it is worth it. You can also use "* 2" for left shift
> > and "/ 2" for right shift. It's not exactly the same (considering
> > overflow), but does that matter in a Vim script?
> >
> >
> >
> Without these functions, to left/right shift a number, you need to
> do something like the following:
>
> Left Shift:
> let c = a * float2nr(pow(2, b))
>
> Right Shift:
> let c = float2nr(floor(a / pow(2, b)))
>
> I thought it would be simpler to have a separate function for this.

Hmm, what if we have an operator for integer pow()?
E.g. "**" could be used: "2 ** b"
That would be useful in general.

--
Everyone has a photographic memory. Some don't have film.

Bram Moolenaar

unread,
Nov 15, 2021, 2:08:15 PM11/15/21
to vim/vim, vim-dev ML, Comment

I appear to have missed this one. Bit shift does not apply to floats, your example does not seem representative.
lshift(val, 4) would be equivalent to "val * 16" and rshift(val, 4) equivalent to "val / 16".


You are receiving this because you commented.
Reply to this email directly, view it on GitHub.
Triage notifications on the go with GitHub Mobile for iOS or Android.

Yegappan Lakshmanan

unread,
Nov 16, 2021, 1:38:52 AM11/16/21
to vim_dev, reply+ACY5DGBH7ZZNOUOY4R...@reply.github.com, vim/vim, vim-dev ML, Comment
Hi Bram,

On Mon, Nov 15, 2021 at 11:08 AM Bram Moolenaar <vim-dev...@256bit.org> wrote:

I appear to have missed this one. Bit shift does not apply to floats, your example does not seem representative.


Left Shift:
let c = a * float2nr(pow(2, b))

 Right Shift:
let c = float2nr(floor(a / pow(2, b)))

The above examples are not doing bit shifts of a float. To compute 2 ^ n, you need to
use the pow() function which returns a float. You then need to use float2nr() to
remove the fraction.
 

lshift(val, 4) would be equivalent to "val * 16" and rshift(val, 4) equivalent to "val / 16".


If you know the number of bits to shift beforehand, then you can use a hard coded
number like 16. Otherwise, you need to use the pow() function to compute 2 ^ n.

Regards,
Yegappan

 

vim-dev ML

unread,
Nov 16, 2021, 1:39:09 AM11/16/21
to vim/vim, vim-dev ML, Your activity

Hi Bram,

On Mon, Nov 15, 2021 at 11:08 AM Bram Moolenaar ***@***.***>

wrote:

> I appear to have missed this one. Bit shift does not apply to floats, your
> example does not seem representative.
>

Left Shift:
let c = a * float2nr(pow(2, b))

Right Shift:
let c = float2nr(floor(a / pow(2, b)))

The above examples are not doing bit shifts of a float. To compute 2 ^ n,
you need to
use the pow() function which returns a float. You then need to use
float2nr() to
remove the fraction.


> lshift(val, 4) would be equivalent to "val * 16" and rshift(val, 4)
> equivalent to "val / 16".
>
>
> If you know the number of bits to shift beforehand, then you can use a
hard coded
number like 16. Otherwise, you need to use the pow() function to compute 2
^ n.

Regards,
Yegappan


You are receiving this because you are subscribed to this thread.

LemonBoy

unread,
Apr 26, 2022, 5:46:27 AM4/26/22
to vim/vim, vim-dev ML, Comment

Any update on this PR? The pow+float2nr approach works but is quite a mouthful, having the shift operators would greatly complement the other bit ops.


Reply to this email directly, view it on GitHub.
You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1109585815@github.com>

Bram Moolenaar

unread,
Apr 26, 2022, 6:37:21 AM4/26/22
to vim/vim, vim-dev ML, Comment

I have no see a response to my suggestion to use "N * 16" to shift 4 bits left and "N / 16" to shift 4 bits right.


Reply to this email directly, view it on GitHub.
You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1109634483@github.com>

Bram Moolenaar

unread,
Apr 26, 2022, 6:46:18 AM4/26/22
to vim/vim, vim-dev ML, Comment

I mean, why it would be needed to specify the number of bits. When do you not know that beforehand?


Reply to this email directly, view it on GitHub.
You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1109642593@github.com>

LemonBoy

unread,
Apr 26, 2022, 7:16:48 AM4/26/22
to vim/vim, vim-dev ML, Comment

Sometimes I use bitsets to keep track of a small (< bit size of number variable) number of boolean elements, that's much faster than eg. having a list with N bool elements.
Testing whether the i-th element is present can be achieved with set & (1 << index) and unless you know all the indices beforehand you're bound to use a runtime-known shift value.


Reply to this email directly, view it on GitHub.
You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1109669435@github.com>

Yegappan Lakshmanan

unread,
Apr 27, 2022, 2:49:19 PM4/27/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 7dd99a1 Add support bitwise left/right shift functions

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9737242553@github.com>

Yegappan Lakshmanan

unread,
Apr 27, 2022, 2:58:56 PM4/27/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9737317225@github.com>

Yegappan Lakshmanan

unread,
May 18, 2022, 11:32:18 PM5/18/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 6cb840c Add support bitwise left/right shift functions

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9923528053@github.com>

Yegappan Lakshmanan

unread,
May 19, 2022, 8:17:27 AM5/19/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • c7c2bc8 Add support bitwise left/right shift functions

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9927644790@github.com>

Bram Moolenaar

unread,
May 19, 2022, 10:17:04 AM5/19/22
to vim/vim, vim-dev ML, Comment

I was trying to find another language that uses a function for bitwise shift, but it appears all the popular languages use ">>" and "<<" operators. Would that be possible in Vim as well, or would it create a conflict?
It might seem a bit inconsistent with the other bitwise functions though.

Another thing to consider is to use one function, where a negative argument is used to shift in the other direction. Again, this would depend on how other languages do this, what people are used to.


Reply to this email directly, view it on GitHub.
You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1131757099@github.com>

Dominique Pellé

unread,
May 19, 2022, 11:49:19 AM5/19/22
to vim/vim, vim-dev ML, Comment

it appears all the popular languages use ">>" and "<<" operators

Java uses >>> and >> which has different behavior for the bit sign.
https://docs.oracle.com/javase/tutorial/java/nutsandbolts/op3.html

Something to consider if Vim script has bit shift operators.


Reply to this email directly, view it on GitHub.
You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1131894155@github.com>

Bram Moolenaar

unread,
May 19, 2022, 5:19:11 PM5/19/22
to vim...@googlegroups.com, Dominique Pellé

> > it appears all the popular languages use ">>" and "<<" operators
>
> Java uses `>>>` and `>>` which has different behavior for the bit sign.
> https://docs.oracle.com/javase/tutorial/java/nutsandbolts/op3.html
>
> Something to consider if Vim script has bit shift operators.

This stems from when it was "cheaper" to keep the top bit (the sign bit
for signed integers). In my opinion it is almost never what you want,
except when you want the last 1% of performance. In practice you need
to add an "unsigned" typecast in some languages, which is confusing.

Since bitwise operators should not care about a sign bit, I would very
much prefer ">>" to just make the topmost bit zero. No need for ">>>"
then.

--
hundred-and-one symptoms of being an internet addict:
236. You start saving URL's in your digital watch.

Yegappan Lakshmanan

unread,
May 21, 2022, 9:48:32 AM5/21/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • f7a1cb2 Add support for bitwise left shift (<<) and right shift (>>) operators

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9945917827@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 9:55:14 AM5/21/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 424094c Remove the lshift() and rshift() functions

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9945940902@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 10:06:21 AM5/21/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 2d37bfe Remove duplicate definition of variables

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9945981326@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 10:13:12 AM5/21/22
to vim_dev, Dominique Pellé
Hi,

On Thu, May 19, 2022 at 2:19 PM Bram Moolenaar <Br...@moolenaar.net> wrote:
>
> > > it appears all the popular languages use ">>" and "<<" operators
> >
> > Java uses `>>>` and `>>` which has different behavior for the bit sign.
> > https://docs.oracle.com/javase/tutorial/java/nutsandbolts/op3.html
> >
> > Something to consider if Vim script has bit shift operators.
>
> This stems from when it was "cheaper" to keep the top bit (the sign bit
> for signed integers). In my opinion it is almost never what you want,
> except when you want the last 1% of performance. In practice you need
> to add an "unsigned" typecast in some languages, which is confusing.
>
> Since bitwise operators should not care about a sign bit, I would very
> much prefer ">>" to just make the topmost bit zero. No need for ">>>"
> then.
>

I have updated the PR to implement the bitwise right (>>) and left shift (<<)
operators.

- Yegappan

LemonBoy

unread,
May 21, 2022, 10:27:17 AM5/21/22
to vim/vim, vim-dev ML, Comment

@LemonBoy commented on this pull request.


In src/vim9.h:

> @@ -151,6 +151,9 @@ typedef enum {
     ISN_COMPAREFUNC,
     ISN_COMPAREANY,
 
+    // bitwise left/right shift operation

This should be another case of ISN_OPNR instead of having its own opcode.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980832533@github.com>

LemonBoy

unread,
May 21, 2022, 10:29:31 AM5/21/22
to vim/vim, vim-dev ML, Comment

@LemonBoy commented on this pull request.


In src/vim9execute.c:

> +		    if (tv2->vval.v_number > MAX_LSHIFT_BITS)
+		    {
+			SOURCING_LNUM = iptr->isn_lnum;
+			emsg(_(e_bitshift_value_too_large));
+			goto on_error;
+		    }
+
+		    if (exprtype == EXPR_LSHIFT)
+			tv1->vval.v_number =
+				(uvarnumber_T)tv1->vval.v_number << tv2->vval.v_number;
+		    else
+			tv1->vval.v_number =
+				(uvarnumber_T)tv1->vval.v_number >> tv2->vval.v_number;
+
+		    // clear the topmost sign bit
+		    tv1->vval.v_number &=

I think this to be quite weird, I'd let the user deal with negative numbers rather than making the integers N-1bit wide.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980832687@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 11:16:19 AM5/21/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • c1d2392 Use ISN_OPNR opcode instead of using a new one for bit shift operations

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9946235104@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 11:42:27 AM5/21/22
to vim/vim, vim-dev ML, Comment

@yegappan commented on this pull request.


In src/vim9.h:

> @@ -151,6 +151,9 @@ typedef enum {
     ISN_COMPAREFUNC,
     ISN_COMPAREANY,
 
+    // bitwise left/right shift operation

I have updated the PR to use ISN_OPNR instead of using a new opcode.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980839522@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 11:45:08 AM5/21/22
to vim/vim, vim-dev ML, Comment

@yegappan commented on this pull request.


In src/vim9execute.c:

> +		    if (tv2->vval.v_number > MAX_LSHIFT_BITS)
+		    {
+			SOURCING_LNUM = iptr->isn_lnum;
+			emsg(_(e_bitshift_value_too_large));
+			goto on_error;
+		    }
+
+		    if (exprtype == EXPR_LSHIFT)
+			tv1->vval.v_number =
+				(uvarnumber_T)tv1->vval.v_number << tv2->vval.v_number;
+		    else
+			tv1->vval.v_number =
+				(uvarnumber_T)tv1->vval.v_number >> tv2->vval.v_number;
+
+		    // clear the topmost sign bit
+		    tv1->vval.v_number &=

This is based on the following comment from Bram:

Since bitwise operators should not care about a sign bit, I would very
much prefer ">>" to just make the topmost bit zero. No need for ">>>"
then.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980839723@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 12:27:28 PM5/21/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 7b6f85a Use 0 as arg index when checking for a valid types to the bitshift operator

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9946492020@github.com>

Bram Moolenaar

unread,
May 21, 2022, 1:06:46 PM5/21/22
to vim/vim, vim-dev ML, Comment

" The topmost bit (sign bit) is always cleared " that is for the ">>" operator, not for "<<".

For operator precedence it looks like we need a new level. It's in between "+" and "-"
and the comparative operators. Thus between eval4() and eval5().

This is what Ecmascript specifies:
https://tc39.es/ecma262/multipage/ecmascript-language-expressions.html#sec-left-shift-operator

Are there languages that use a different precedence? I know have have often been confused by this, but we should stick to what most other popular languages do.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133670431@github.com>

Yegappan Lakshmanan

unread,
May 21, 2022, 1:32:55 PM5/21/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 9b30291 Ignore the sign of the left operand for the bitshift operators

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9946721877@github.com>

errael

unread,
May 21, 2022, 3:23:36 PM5/21/22
to vim/vim, vim-dev ML, Comment

@errael commented on this pull request.


In src/vim9execute.c:

> +		    if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
+			// clear the topmost sign bit
+			res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);

This fixup only works if arg2 is 1. Something like following is needed.

result &= (1 << arg2) - 1 << NBITS - arg2

If right shifted by 3 and NBITS is 8 then this should be &= 1110_0000.

If you can count on the compiler to respect unsigned and make sure zeros are shifted in, then you don't need a fixup. If you can't count on the compiler, this might be more understandable and doesn't need a fixup.

res = arg1 >> 1;
res &= ~(1 << MAX_LSHIFT_BITS);
res = res >> arg2 - 1;

None of this is tested, simulated on paper... (I looked up the precedence against +/-)


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980871746@github.com>

errael

unread,
May 21, 2022, 3:27:46 PM5/21/22
to vim/vim, vim-dev ML, Comment

@errael commented on this pull request.


In src/vim9execute.c:

> +		    if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
+			// clear the topmost sign bit
+			res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);

Oops, updated to add the missing ~ and &= 0001_1111


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980872009@github.com>

LemonBoy

unread,
May 21, 2022, 3:31:56 PM5/21/22
to vim/vim, vim-dev ML, Comment

This is based on the following comment from Bram:

Ah I see, the comment didn't get posted to GH for some reason. That's fine as long as it's clearly stated in the docs that the operators perform logical shifts (opposed to arithmetic ones).

The sign-clearing is useless as you're already performing the shift on unsigned values.

The next logical step is to extend the and/or/not operators to work with numbers, right now shifts are the only exception among the bitwise ops.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133756774@github.com>

errael

unread,
May 21, 2022, 3:40:41 PM5/21/22
to vim/vim, vim-dev ML, Comment

In a performance sensitive function I recently used vim's and(arg1, arg2) and or(arg1, arg2) since I couldn't find &,|. Got me wondering what the difference is between a function and vim9 instruction execution; so I was interested in taking a look at how this instruction executes.

I guess if invoking and() involves setting up a stack frame then the difference is pretty big.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133757916@github.com>

errael

unread,
May 21, 2022, 3:44:09 PM5/21/22
to vim/vim, vim-dev ML, Comment

the and/or/not operators to work with numbers

xor gets no respect.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133758339@github.com>

Bram Moolenaar

unread,
May 21, 2022, 4:52:45 PM5/21/22
to vim...@googlegroups.com, errael

> In a performance sensitive function I recently used vim's `and(arg1,
> arg2)` and `or(arg1, arg2)` since I couldn't find `&`,`|`. Got me
> wondering what the difference is between a function and vim9
> instruction execution; so I was interested in taking a look at how
> this instruction executes.
>
> I guess if invoking `and()` involves setting up a stack frame then the
> difference is pretty big.

Unfortunately, yes, a function call has quite a bit of overhead.
Especially now that expressions are converted to byte code when
compiling. A function call still has to handle all kinds of possible
side effects. I wonder if we can make a difference between calling a
user defined function and a builtin function, since the latter should
have predictable behavior.

--
hundred-and-one symptoms of being an internet addict:
251. You've never seen your closest friends who usually live WAY too far away.

Ernie Rael

unread,
May 21, 2022, 11:18:01 PM5/21/22
to vim...@googlegroups.com
On 5/21/22 1:52 PM, Bram Moolenaar wrote:
>> In a performance sensitive function I recently used vim's `and(arg1,
>> arg2)` and `or(arg1, arg2)` since I couldn't find `&`,`|`. Got me
>> wondering what the difference is between a function and vim9
>> instruction execution; so I was interested in taking a look at how
>> this instruction executes.
>>
>> I guess if invoking `and()` involves setting up a stack frame then the
>> difference is pretty big.
> Unfortunately, yes, a function call has quite a bit of overhead.
> Especially now that expressions are converted to byte code when
> compiling. A function call still has to handle all kinds of possible
> side effects. I wonder if we can make a difference between calling a
> user defined function and a builtin function, since the latter should
> have predictable behavior.
>
My first thought was that you'd want a flag for each builtin that
indicated if it could be inlined-call in a compiled :def, when it had no
side effects. But I guess the side effects you speak of are to the
script context and it's variables. I don't know if there are any
builtins that have access to the vim9 script context, I'd guess they
shouldn't be inlined. Certainly and(), or(), xor() no side effects
anyway you look at it. I guess calling f_and() is same order of
magnitude as handling it through ISN_OPNR.

Hmm, I don't see a not(), bitwise complement, function. Called something
else? I guess you can do
val = -val - 1. Does that work on negative numbers?

-ernie

Yegappan Lakshmanan

unread,
May 22, 2022, 1:01:59 AM5/22/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 3095705 Introduce a new precedence level for bitshift operators

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9948647098@github.com>

Dominique Pelle

unread,
May 22, 2022, 5:13:40 AM5/22/22
to vim/vim, vim-dev ML, Comment

@DominiquePelle-TomTom commented on this pull request.


In src/testdir/test_expr.vim:

> @@ -946,4 +946,48 @@ func Test_string_interp()
   call v9.CheckDefAndScriptSuccess(lines)
 endfunc
 
+" Test for bitwise left and right shift (<< and >>)
+func Test_bitwise_shift()
+  let lines =<< trim END
+    call assert_equal(16, 1 << 4)
+    call assert_equal(2, 16 >> 3)
+    call assert_equal(0, 0 << 2)
+    call assert_equal(0, 0 >> 4)
+    call assert_equal(3, 3 << 0)
+    call assert_equal(3, 3 >> 0)
+    call assert_equal(0, 0 >> 4)

What happens if you do:

  • 42 >> -1 (in C, shifting by negative number is undefined behavior, in Vim it should be an error or maybe 0?)
  • 42 << 100 or 42 >> 100 (in C, shift by more bits than the underlying type is undefined behavior, in Vim it should be an error or maybe 0?)


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/980930329@github.com>

Yegappan Lakshmanan

unread,
May 22, 2022, 8:09:38 AM5/22/22
to vim_dev, nor...@github.com, vim/vim, vim-dev ML, Push
Hi,

Responding to Dominique Pelle's comments (which somehow are not added to the PR):

>
> What happens if you do:
>
> 42 >> -1 (in C, shifting by negative number is undefined behavior, in
> Vim it should be an error or maybe 0?)
>

In this case, Vim will display the "E1283: bitshift amount must be a positive number"
error message.

>
> 42 << 100 or 42 >> 100 (in C, shift by more bits than the underlying
> type is undefined behavior, in Vim it should be an error or maybe 0?)
>

In this case, Vim will display the "E1284: bitshift amount is too large" error message.

- Yegappan


Yegappan Lakshmanan

unread,
May 22, 2022, 8:11:58 AM5/22/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9950005503@github.com>

Yegappan Lakshmanan

unread,
May 22, 2022, 9:10:30 AM5/22/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9950243917@github.com>

Bram Moolenaar

unread,
May 22, 2022, 11:39:01 AM5/22/22
to vim/vim, vim-dev ML, Comment

I don't spot a test case for repeating the operator, e.g. 1 << 2 << 3
I believe this should work like (1 << 2) << 3


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133922036@github.com>

Yegappan Lakshmanan

unread,
May 22, 2022, 1:01:59 PM5/22/22
to vim/vim, vim-dev ML, Push

@yegappan pushed 1 commit.

  • 22313ea Add support for using multiple bitshift operators in an expression

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/push/9951152076@github.com>

Yegappan Lakshmanan

unread,
May 22, 2022, 1:02:56 PM5/22/22
to vim/vim, vim-dev ML, Comment

I have updated the PR to support using multiple bit shift operators in an expression and added additional tests.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133935317@github.com>

Bram Moolenaar

unread,
May 22, 2022, 1:25:47 PM5/22/22
to vim/vim, vim-dev ML, Comment

Looks like it is ready to include now, thanks.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1133939047@github.com>

errael

unread,
May 22, 2022, 1:41:17 PM5/22/22
to vim/vim, vim-dev ML, Comment

@errael commented on this pull request.


In src/vim9execute.c:

> +		    if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
+			// clear the topmost sign bit
+			res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);

Have you tried it without this post shift fixup? Since the type of the the left hand side is unsigned, no adjustment is needed.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/review/981000032@github.com>

Yegappan Lakshmanan

unread,
May 22, 2022, 2:00:45 PM5/22/22
to vim_dev, reply+ACY5DGBOC4YZI3GXOE...@reply.github.com, vim/vim, vim-dev ML, Comment
Hi,

On Sun, May 22, 2022 at 10:41 AM errael <vim-dev...@256bit.org> wrote:

@errael commented on this pull request.


In src/vim9execute.c:

> +		    if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
+			// clear the topmost sign bit
+			res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);

Have you tried it without this post shift fixup? Since the type of the the left hand side is unsigned, no adjustment is needed.



The left operand of a bitshift operator can be a negative number.  Earlier I had a check to display
an error if the left operand is a negative number.  But I removed that check based on one of the
comments.  I will add some tests for right shifting negative numbers and then remove this code.

- Yegappan

vim-dev ML

unread,
May 22, 2022, 2:01:01 PM5/22/22
to vim/vim, vim-dev ML, Your activity

Hi,

On Sun, May 22, 2022 at 10:41 AM errael ***@***.***> wrote:

> ***@***.**** commented on this pull request.
> ------------------------------
>
> In src/vim9execute.c
> <https://github.com/vim/vim/pull/8457#discussion_r878903278>:

>
> > + if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
> + // clear the topmost sign bit
> + res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);
>
> Have you tried it without this post shift fixup? Since the type of the the
> left hand side is unsigned, no adjustment is needed.
>
>
>
The left operand of a bitshift operator can be a negative number. Earlier
I had a check to display
an error if the left operand is a negative number. But I removed that
check based on one of the
comments. I will add some tests for right shifting negative numbers and
then remove this code.

- Yegappan


Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/c1133944860@github.com>

Ernie Rael

unread,
May 22, 2022, 2:09:45 PM5/22/22
to vim...@googlegroups.com, Yegappan Lakshmanan, reply+ACY5DGBOC4YZI3GXOE...@reply.github.com, vim/vim, vim-dev ML
On 5/22/22 11:00 AM, Yegappan Lakshmanan wrote:
>
>
> The left operand of a bitshift operator can be a negative number.

Consider this case: -1 >> 0

The result should be -1.

AFAICT. The code treats the number as unsigned and does a logical right
shift, so no fixup needed.

vim-dev ML

unread,
May 22, 2022, 2:10:06 PM5/22/22
to vim/vim, vim-dev ML, Your activity


Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/8457/c1133946447@github.com>

Bram Moolenaar

unread,
May 22, 2022, 2:14:14 PM5/22/22
to vim/vim, vim-dev ML, Comment

Closed #8457 via a061f34.


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/issue_event/6656478574@github.com>

Bram Moolenaar

unread,
May 22, 2022, 2:38:28 PM5/22/22
to vim...@googlegroups.com, errael

> @errael commented on this pull request.
>
>
> > + if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
> + // clear the topmost sign bit
> + res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);
>
> Have you tried it without this post shift fixup? Since the type of the
> the left hand side is unsigned, no adjustment is needed.

I accidentally remove the type cast (I didn't notice the "u" and thought
it was not needed).

I think that clearing the topmost bit was pointless anyway, since the
sign bit would also be replicated by as many positions it was shifted,
AFAIK if there were eight bits: "10000000 >> 2" would result in 11100000.
Only clearing the top bit leaves 01100000 instead of the desired 00100000

I was hoping to test with this, but it doesn't work:

if v:numbersize == 64
call assert_equal(0xfffffffffffffff8, 0x1fffffffffffffff << 3)
call assert_equal(0x1fffffffffffffff, 0xffffffffffffffff >> 3)
endif


--
XML is a nice language for computers. Not for humans.

Bram Moolenaar

unread,
May 22, 2022, 3:07:08 PM5/22/22
to vim...@googlegroups.com, Bram Moolenaar, errael

I wrote:

> > @errael commented on this pull request.
> >
> >
> > > + if (iptr->isn_arg.op.op_type == EXPR_RSHIFT)
> > + // clear the topmost sign bit
> > + res &= ~((uvarnumber_T)1 << MAX_LSHIFT_BITS);
> >
> > Have you tried it without this post shift fixup? Since the type of the
> > the left hand side is unsigned, no adjustment is needed.
>
> I accidentally remove the type cast (I didn't notice the "u" and thought
> it was not needed).
>
> I think that clearing the topmost bit was pointless anyway, since the
> sign bit would also be replicated by as many positions it was shifted,
> AFAIK if there were eight bits: "10000000 >> 2" would result in 11100000.
> Only clearing the top bit leaves 01100000 instead of the desired 00100000
>
> I was hoping to test with this, but it doesn't work:
>
> if v:numbersize == 64
> call assert_equal(0xfffffffffffffff8, 0x1fffffffffffffff << 3)
> call assert_equal(0x1fffffffffffffff, 0xffffffffffffffff >> 3)
> endif

Ah, that's because 0xffffffffffffffff actually becomes
0x7fffffffffffffff. Using -1 to get all bits set does work.

--
Error 42: No errors

Bram Moolenaar

unread,
Oct 11, 2022, 3:58:02 AM10/11/22
to vim/vim, vim-dev ML, Comment


> @DominiquePelle-TomTom commented on this pull request.

>
> > @@ -946,4 +946,48 @@ func Test_string_interp()
> call v9.CheckDefAndScriptSuccess(lines)
> endfunc
>
> +" Test for bitwise left and right shift (<< and >>)
> +func Test_bitwise_shift()
> + let lines =<< trim END
> + call assert_equal(16, 1 << 4)
> + call assert_equal(2, 16 >> 3)
> + call assert_equal(0, 0 << 2)
> + call assert_equal(0, 0 >> 4)
> + call assert_equal(3, 3 << 0)
> + call assert_equal(3, 3 >> 0)
> + call assert_equal(0, 0 >> 4)
>
> What happens if you do:
>
> * `42 >> -1` (in C, shifting by negative number is undefined

> behavior, in Vim it should be an error or maybe 0?)

Trying to shift by a negative amount is nearly always a mistake. In my
opinion it should be an error.

> * `42 << 100` or `42 >> 100` (in C, shift by more bits than the

> underlying type is undefined behavior, in Vim it should be an error or
> maybe 0?)

A huge number could be a mistake, but how does the user know what the
maximum is? Probably 64, but one can't be sure. And would then 64 be
an error (since the result is always zero) or above 64? Getting a zero
result is more useful than getting an error.


--
hundred-and-one symptoms of being an internet addict:
252. You vote for foreign officials.

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\

/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///


Reply to this email directly, view it on GitHub.

You are receiving this because you commented.Message ID: <vim/vim/pull/8457/c1274253679@github.com>

Reply all
Reply to author
Forward
0 new messages