About the efficency of modifying map elements.

[tapi...@gmail.com]

Now, to modify a map element, especially the element type is not a basic type, two hashes are needed to compute. This is often unnecessary and inefficient. For example:

    package main

    type T struct{
        N int
        // ... more fields
    }

    func main() {
        var m = map[string]T{}
        m["foo"] = T{N: 0}
       
        // modify
        t := m["foo"] // first hashing
        t.N++
        m["foo"] = t  // second hashing
    }

Will it be good to add a new builtin function, modify(m Map[Key]Value, k Key, func(v *Value)), to modify map elements with only one hash? A use example:

    package main

    type T struct{
        N int
        // ... more fields
    }

    func main() {
        var m = map[string]T{}
        m["foo"] = T{N: 0}
       
        // modify
        modify(m. "foo", func(t *T) {
            t.N++
        })
    }

[Axel Wagner]

Hi,

have you verified this using a disassembler or benchmarks? Just asking because this is, as far as I'm concerned, a job for the compiler, to eliminate the overhead automatically - and I could well imagine that it's already doing it. There definitely shouldn't be a new language construct for this.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ba7b2c95-829b-4da4-916a-d53a06ec3428n%40googlegroups.com.

[Jan Mercl]

On Wed, Mar 17, 2021 at 9:18 PM tapi...@gmail.com <tapi...@gmail.com> wrote:

> Now, to modify a map element, especially the element type is not a basic type, two hashes are needed to compute.

https://github.com/golang/go/issues/5147

[tapi...@gmail.com]

I found this performance difference by benchmarking the two:

func IntAdd(words [][]byte) map[string]int {
    var m = make(map[string]int)
    for _, w := range words {
        m[string(w)] = m[string(w)] + 1
       
    }
    return m
}

func IntIncrement(words [][]byte) map[string]int {
    var m = make(map[string]int)
    for _, w := range words {
        m[string(w)]++
    }
    return m
}

IntAdd is obviously slower than IntIncrement.

[Axel Wagner]

Yes, Jan's link also pretty clearly shows that this optimization isn't currently happening :) Sorry for the noise.

I still believe it should be an optimization in the compiler, not a language-level feature.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d763c1fd-6e57-41f1-90e1-98a369ddf3bcn%40googlegroups.com.

[tapi...@gmail.com]

For simple scenarios, compiler optimizations might be possible.

But for some complicate ones, it is hard for compiler to do optimizations.

For example, the element is modified by an function between the map

element getter and setter and the behavior of the function is hard to

determined at compile time, it might modify or not modify the key.

[Axel Wagner]

On Wed, Mar 17, 2021 at 10:06 PM tapi...@gmail.com <tapi...@gmail.com> wrote:

For simple scenarios, compiler optimizations might be possible.

But for some complicate ones, it is hard for compiler to do optimizations.

For example, the element is modified by an function between the map

element getter and setter and the behavior of the function is hard to

determined at compile time, it might modify or not modify the key.

Yes. I would strongly prefer to have these uncommon cases be slower to complicating the language.

FWIW, if anything, the signature would have to take a `func(T) T`, not a `func(*T)` - values in a map are not addressable for a reason. If we would use a pointer, it would allow the programmer to retain that pointer outside of the function, which would not be safe.

But this also implies that any case covered by that function can be re-written as

_tmp := k // temporary variable, in case k is modified

v := m[_tmp]

// do the same as the function would

m[_tmp] = v

which should easily recognizable by the compiler. That means we would only need to optimize this simple case and anyone concerned about speed could rewrite their code into this form to get the same result.

So, I don't actually think doing it as an optimization is harder or less powerful than adding this function. It should cover the same cases, without complicating the language.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/52f00985-3e0b-4562-80d9-705ae8e5e507n%40googlegroups.com.

[Matthew Holiday]

Isn't this really just another form of issue 3117?

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEggD%2BnqVJFismVYK%3DqtviFozVft1jdBuT7-VM8cgxXZQ%40mail.gmail.com.

--

Matt Holiday

Senior Gopher, Marketing Technologies

620 Eighth Avenue

New York, NY 10018

matthew...@nytimes.com

[tapi...@gmail.com]

On Wednesday, March 17, 2021 at 5:19:55 PM UTC-4 axel.wa...@googlemail.com wrote:

On Wed, Mar 17, 2021 at 10:06 PM tapi...@gmail.com <tapi...@gmail.com> wrote:

For simple scenarios, compiler optimizations might be possible.

But for some complicate ones, it is hard for compiler to do optimizations.

For example, the element is modified by an function between the map

element getter and setter and the behavior of the function is hard to

determined at compile time, it might modify or not modify the key.

Yes. I would strongly prefer to have these uncommon cases be slower to complicating the language.

FWIW, if anything, the signature would have to take a `func(T) T`, not a `func(*T)` - values in a map are not addressable for a reason. If we would use a pointer, it would allow the programmer to retain that pointer outside of the function, which would not be safe.

Yes, "func(*T)" is problematic. But the "func(T) T" is inefficient for large types.

There are too many constraints in using the builtin map, though it really provides much convenience.

Hope custom maps will help in such situations.

[tapi...@gmail.com]

On Wednesday, March 17, 2021 at 5:23:57 PM UTC-4 matthew...@nytimes.com wrote:

Isn't this really just another form of issue 3117?

Some of, but really different.

[Axel Wagner]

To be clear: My point was that it should be neither. It should be a transparent compiler optimization :)

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8193fa0b-307f-421e-a2f8-a7e2f23777b5n%40googlegroups.com.

[Robert Engels]

Generics will solve this. 

On Mar 17, 2021, at 4:23 PM, Matthew Holiday <matthew...@nytimes.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGSa1CnPs3jnx07P23rzxKfCjDR0Bq580-k3Gxy0Zj1%3Dqo9dng%40mail.gmail.com.