Pass arguments to append() with more flexibility

[Huan Du]

The built-in function append() is declared as following in Language Spec.

append(s S, x ...T) S

It denies the usage like this.

slice1 := []int{1,2}

slice2 := []int{4, 5}

slice3 := append(slice1, 3, slice2...) // compile error

I know I can work around this compile error as following. However, there would be significant waste of memory and CPU time if slices contain tons of data.

slice3 := append(slice1, 2)

slice3 = append(slice3, slice2...)

Therefore, my request finally becomes to suggest a new feature to extend ...T grammar in append(). It can make append() easy to use as well as better performance.

An idea append() implementation would be

• Accept flexible arguments. Code append(slice1, 3, slice2...) should compile. Even more, append(slice1, slice2..., 3, slice2...) should also compile, although it looks a bit wired.
• Be able to estimate new slice len, allocate memory if cap(slice) is smaller than len, copy all values to the slice space and finally return it.

Built-in functions have already used several "exceptional grammar", e.g. optional args in make() and generic types. It may be feasible to add one more. :)

[Thomas Bushnell, BSG]

If this were done, it would make sense to support it for all variadic functions.

In the general case however, it's not any more efficient than the two step way, though append could be special cased not to need the extra work.

Thomas

[David Symonds]

On Thu, May 3, 2012 at 3:59 PM, Huan Du <reald...@gmail.com> wrote:

> I know I can work around this compile error as following. However, there
> would be significant waste of memory and CPU time if slices contain tons of
> data.
>
> slice3 := append(slice1, 2)
> slice3 = append(slice3, slice2...)

How is that a waste of memory and CPU?

[Huan Du]

In the general case however, it's not any more efficient than the two step way, though append could be special cased not to need the extra work.

True. I wouldn't suggest to add this feature in general case. This feature is only valid in append().

How is that a waste of memory and CPU?

Consider that each call to append() will alloc a new array and copy old data to it, if src slice contains tons of data, it's a big waste.

From my point of view, append() is just a syntactic candy. In my case, it can save time on coding but waste time on running. I'd rather choose to waste my coding time.

I can write a "perfect" solution, but I cannot make it a func for reuse. I have to repeat, repeat, repeat myself in similar scenarios. It's wired. Why not make the candy tasty? :(

// a "perfect" solution

// this code is to append slice1 and slice2 to slice3

// all slices are []int type

len1, len2, len3 := len(slice1), len(slice2), len(slice3)

if cap(slice3) >= len1 + len2 + len3 {

    slice3 := slice3[:len1 + len2 + len3]

} else {

    s := make([]int, len1 + len2 + len3)

    copy(s, slice3)

    slice3 = s

}

copy(slice3[len3:len1 + len3], slice1)

copy(slice3[len1 + len3:], slice2)

[Rémy Oudompheng]

On 2012/5/3 Huan Du <reald...@gmail.com> wrote:
>> In the general case however, it's not any more efficient than the two step
>> way, though append could be special cased not to need the extra work.
>
> True. I wouldn't suggest to add this feature in general case. This feature
> is only valid in append().
>
>> How is that a waste of memory and CPU?
>
> Consider that each call to append() will alloc a new array and copy old data
> to it, if src slice contains tons of data, it's a big waste.

It's not true. However, you may do better by hand sometimes.
And,

s = append(append(s1, s2...), s3...)

should already be correct and if it is not fast enough, maybe the
compiler can be improved.
You should provide benchmarks and indicate which cases behave badly.

Rémy.

[Robert Carlsen]

My understanding was that append only allocates space if the slice spans the entire underlying array (which is not every time you use append).  And when it does allocate, it allocates a new array double the size of the current one - resulting in relatively few allocs - and not as much of a performance hit as you might suspect.