mopp

unread,

Dec 8, 2018, 4:01:30 AM12/8/18

to vim/vim, Subscribed

This change add new small function flatten({list} [, {maxdepth}]) to flatten the given list to depth {maxdepth}.

This function assists list processing combining list manipulation functions.
Here is example

" Collect paths from some patterns
echo flatten(map(['./xxx/*', './yyy/*'], 'glob(v:val, 0, 1)'))
" Split and filter strings
echo filter(flatten(map(zzz, 'split(v:val, ",")')), '!empty(v:val)')

The performance is good because it uses loop instead of recursive function call.

Thanks

You can view, comment on, or merge this pull request online at:

https://github.com/vim/vim/pull/3676

Commit Summary

add flatten
refine flatten()
fix style
remove unnecessary argument current_depth
raise error if maxdepth is negetive (wip)
check interrupt
use loop instead of recursive call (by mattn)
add tests
make the default maxdepth 1
keep the given list if maxdepth is 0
fix error handling
return FAIL if out of memory
add document

File Changes

Patch Links:

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub

Codecov

unread,

Dec 8, 2018, 4:18:43 AM12/8/18

to vim/vim, Subscribed

Codecov Report

Merging #3676 into master will increase coverage by 0.01%.
The diff coverage is 97.36%.

@@            Coverage Diff             @@

##           master    #3676      +/-   ##

==========================================

+ Coverage   77.41%   77.42%   +0.01%     

==========================================

  Files          99       99              

  Lines      139440   139478      +38     

==========================================

+ Hits       107943   107989      +46     

+ Misses      31497    31489       -8

Impacted Files	Coverage Δ
src/evalfunc.c	`87.22% <100%> (+0.03%)`	⬆️
src/list.c	`89.41% <95.23%> (+0.3%)`	⬆️
src/gui.c	`57.63% <0%> (-0.57%)`	⬇️
src/ex_cmds2.c	`84.49% <0%> (-0.1%)`	⬇️
src/if_xcmdsrv.c	`84.17% <0%> (ø)`	⬆️
src/window.c	`82.91% <0%> (+0.03%)`	⬆️
src/gui_gtk_x11.c	`49.28% <0%> (+1.03%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5393281...4b17de2. Read the comment docs.

mopp

unread,

Dec 8, 2018, 6:42:08 AM12/8/18

to vim/vim, Push

@mopp pushed 1 commit.

934809f merge patch from h-east

—
You are receiving this because you are subscribed to this thread.

View it on GitHub

Bram Moolenaar

unread,

Dec 8, 2018, 7:58:29 AM12/8/18

to vim/vim, Subscribed

Mopp wrote:

> This change add new small function `flatten({list} [, {maxdepth}])` to
> flatten the given list to depth `{maxdepth}`.
>
> This function assists list processing combining list manipulation functions.
> Here is example

> ```vim

> " Collect paths from some patterns
> echo flatten(map(['./xxx/*', './yyy/*'], 'glob(v:val, 0, 1)'))
> " Split and filter strings
> echo filter(flatten(map(zzz, 'split(v:val, ",")')), '!empty(v:val)')
> ```
> The performance is good because it uses loop instead of recursive
> function call.

I'm a bit confused by the maxdepth argument. I would expect it to
specify the maximum depth of the resulting list, thus flatten(list, 1)
would always result in items that are not a list. But it appears it
flattens by one level, thus if one of the items is a list of lists, the
item becomes a list.

Isn't it more useful to specify the maximum resulting depth? With the
default being 1 this would result in a list of items:

flatten([[[1, 2], 3, [4]], [5, 6]])
gives
[1, 2, 3, 4, 5, 6]

And
flatten([[[1, 2], 3, [4]], [5, 6]], 2)
gives
[[1, 2, 3, 4], [5, 6]]
That would work for two-dimensional arrays.

I would expect the most often used form is to flatten a nested list into
a list of items.

--
BLACK KNIGHT: The Black Knight always triumphs. Have at you!
ARTHUR takes his last leg off. The BLACK KNIGHT's body lands upright.
BLACK KNIGHT: All right, we'll call it a draw.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

mopp

unread,

Dec 8, 2018, 8:48:46 AM12/8/18

to vim/vim, Subscribed

@brammool Thanks for your comment.

I would expect the most often used form is to flatten a nested list into a list of items.

Yes, I think so too.

The documentation about maxdepth is not enough or the name maxdepth is not suitable.
Some examples are here

It is called limit in case of vital.vim.

echo s:L.flatten([[['a']], [[['b']], 'c']], 2)
" => ['a', ['b'], 'c']

It is called level in case of Ruby's flatten.

a = [ 1, 2, [3, [4, 5] ] ]
a.flatten!(1) 
# => [1, 2, 3, [4, 5]]

The argument means "How deeply it makes flatten the list" usually.

I think it is better to follow them.
The name maxdepth may be required to change such as limit, level, expand_depth or etc.

mopp

unread,

Dec 9, 2018, 5:50:10 AM12/9/18

to vim/vim, Push

@mopp pushed 2 commits.

2e34152 fix test
c29fd36 update doc

—
You are receiving this because you are subscribed to this thread.

View it on GitHub

mopp

unread,

Dec 9, 2018, 6:01:46 AM12/9/18

to vim/vim, Push

@mopp pushed 1 commit.

a564396 use two spaces after a period

—
You are receiving this because you are subscribed to this thread.

View it on GitHub

Andy Massimino

unread,

Jan 6, 2019, 2:22:04 PM1/6/19

to vim/vim, Subscribed

@andymass commented on this pull request.

In src/list.c:

> + * Returns FAIL when out of memory.
+ */
+    int
+list_flatten(list_T *list, long maxdepth)
+{
+    listitem_T	*item;
+    int		n;
+
+    if (maxdepth == 0)
+	return OK;
+
+    n = 0;
+    item = list->lv_first;
+    while (item != NULL && !got_int)
+    {
+	line_breakcheck();

Why do a breakcheck? I think many list processing functions do not check for interrupts. But most importantly, it appears to put the list into an indeterminate state; if the function does return OK I would expect the list to be list flattened to maxdepth.

Andy Massimino

unread,

Jan 6, 2019, 2:37:28 PM1/6/19

to vim/vim, Subscribed

@andymass commented on this pull request.

In runtime/doc/eval.txt:

> @@ -4034,6 +4035,19 @@ findfile({name} [, {path} [, {count}]])				*findfile()*
 <		Searches from the directory of the current file upwards until
 		it finds the file "tags.vim".
 
+flatten({list} [, {maxdepth}])					*flatten()*
+		Flatten {list} to depth {maxdepth}.  {maxdepth} means how

It should be noted that flattening is done in-place, i.e., modifies the list (as other vim list functions)

In runtime/doc/eval.txt:

> @@ -4034,6 +4035,19 @@ findfile({name} [, {path} [, {count}]])				*findfile()*
 <		Searches from the directory of the current file upwards until
 		it finds the file "tags.vim".
 
+flatten({list} [, {maxdepth}])					*flatten()*
+		Flatten {list} to depth {maxdepth}.  {maxdepth} means how
+		deeply it makes flatten the list.  the depth is 1 when
+		{maxdepth} is omitted.  {list} is not modified when
+		{maxdepth} is 0.
+		Example: >
+			:echo flatten([1, [2], 3])
+<			[1, 2, 3] >
+			:echo flatten([[[1], [2], [3]]], 1)
+<			[[1], [2], [3]]
+							        *E964*
+		{maxdepth} must be positive number.

Since zero is explicitly allowed (why?), this should say "must be a non-negative Number."

In runtime/doc/eval.txt:

> @@ -4034,6 +4035,19 @@ findfile({name} [, {path} [, {count}]])				*findfile()*
 <		Searches from the directory of the current file upwards until
 		it finds the file "tags.vim".
 
+flatten({list} [, {maxdepth}])					*flatten()*
+		Flatten {list} to depth {maxdepth}.  {maxdepth} means how
+		deeply it makes flatten the list.  the depth is 1 when

This is not completely clear. Is it correct to say that the result is the depth of the list, i.e., a depth 1 list contains only Dict, Number, String, etc, whereas a depth 2 list may contain other lists (which do not contain lists themselves)?

mopp

unread,

Jan 8, 2019, 8:57:33 AM1/8/19

to vim/vim, Subscribed

@mopp commented on this pull request.

In src/list.c:

> + * Returns FAIL when out of memory.
+ */
+    int
+list_flatten(list_T *list, long maxdepth)
+{
+    listitem_T	*item;
+    int		n;
+
+    if (maxdepth == 0)
+	return OK;
+
+    n = 0;
+    item = list->lv_first;
+    while (item != NULL && !got_int)
+    {
+	line_breakcheck();

it appears to put the list into an indeterminate state; if the function does return OK

Yes, It should return FAIL if interrupted.
I will fix it.

mopp

unread,

Jan 8, 2019, 9:17:59 AM1/8/19

to vim/vim, Subscribed

@mopp commented on this pull request.

In runtime/doc/eval.txt:

> @@ -4034,6 +4035,19 @@ findfile({name} [, {path} [, {count}]])				*findfile()*
 <		Searches from the directory of the current file upwards until
 		it finds the file "tags.vim".
 
+flatten({list} [, {maxdepth}])					*flatten()*
+		Flatten {list} to depth {maxdepth}.  {maxdepth} means how
+		deeply it makes flatten the list.  the depth is 1 when
+		{maxdepth} is omitted.  {list} is not modified when
+		{maxdepth} is 0.
+		Example: >
+			:echo flatten([1, [2], 3])
+<			[1, 2, 3] >
+			:echo flatten([[[1], [2], [3]]], 1)
+<			[[1], [2], [3]]
+							        *E964*
+		{maxdepth} must be positive number.

This description is right because the given list is NOT modified when the maxdepth is 0.

It can deal with when a calculation result of maxdepth becomes 0.

mopp

unread,

Jan 8, 2019, 9:21:18 AM1/8/19

to vim/vim, Subscribed

@mopp commented on this pull request.

In runtime/doc/eval.txt:

> @@ -4034,6 +4035,19 @@ findfile({name} [, {path} [, {count}]])				*findfile()*

 <		Searches from the directory of the current file upwards until

 		it finds the file "tags.vim".

 

+flatten({list} [, {maxdepth}])					*flatten()*

+		Flatten {list} to depth {maxdepth}.  {maxdepth} means how

+		deeply it makes flatten the list.  the depth is 1 when

I'm sorry. My English is not good.

Is it correct to say that the result is a List having depth N

I already described at the #3676 (comment) .
Please check src/testdir/test_flatten.vim for more examples.

Thanks for your reviews 😄

mopp

unread,

Jan 8, 2019, 9:21:35 AM1/8/19

to vim/vim, Push

@mopp pushed 4 commits.

3e07951 Merge branch 'master' into add_flatten
e883857 use tv_get_number_chk
a26d299 return FAIL if interrupted
1176e23 Note that flatten modifies the given list

—
You are receiving this because you are subscribed to this thread.

View it on GitHub

mopp

unread,

Jan 14, 2019, 8:04:11 AM1/14/19

to vim/vim, Push

@mopp pushed 1 commit.

af475dd Merge branch 'master' into add_flatten

—
You are receiving this because you are subscribed to this thread.

View it on GitHub

Bram Moolenaar

unread,

Jun 8, 2020, 2:51:35 PM6/8/20

to vim/vim, Subscribed

Closed #3676 via 077a1e6.

—
You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

mopp

unread,

Jun 9, 2020, 12:19:45 AM6/9/20

to vim/vim, Subscribed

Thank you for merging! 🎉

—
You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

Bram Moolenaar

unread,

Jun 9, 2020, 8:51:31 AM6/9/20

to vim/vim, Subscribed

> Thank you for merging! :tada:

I did change the default depth from 1 to "max". It think it makes more
sense.

--
Luxury. We used to have to get out of the lake at three o'clock in the
morning, clean the lake, eat a handful of hot gravel, go to work at the
mill every day for tuppence a month, come home, and Dad would beat us
around the head and neck with a broken bottle, if we were LUCKY!

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

Yegappan Lakshmanan

unread,

Jun 9, 2020, 10:16:27 AM6/9/20

to vim_dev, reply+ACY5DGD3ACUZSZNM6X...@reply.github.com, vim/vim, Subscribed

Hi,

On Tue, Jun 9, 2020 at 5:51 AM Bram Moolenaar <vim-dev...@256bit.org> wrote:

> Thank you for merging! :tada:

I did change the default depth from 1 to "max". It think it makes more
sense.

The ASAN build in Travis/CI is failing in test_flatten.vim.

- Yegappan

vim-dev ML

unread,

Jun 9, 2020, 10:16:41 AM6/9/20

to vim/vim, vim-dev ML, Your activity

—
You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub, or unsubscribe.

Reply all

Reply to author

Forward

[vim/vim] add flatten() to flatten list (#3676)

mopp

You can view, comment on, or merge this pull request online at:

Commit Summary

File Changes

Patch Links:

Codecov

Codecov Report

mopp

Bram Moolenaar

mopp

mopp

mopp

Andy Massimino

Andy Massimino

mopp

mopp

mopp

mopp

mopp

Bram Moolenaar

mopp

Bram Moolenaar

Yegappan Lakshmanan

vim-dev ML