How make fastest map[uint32]uint64 with fixed size(250m)?

[il]

I have task:

1. Initialize map(250 million items)

2. Fill all values

3. Start http server.

4. Process requests. Iterates all values.

I need fast "for" and read access with reasonable memory consumption. This is very important.

Can you help me best solution?

Thank you!

[Jan Mercl]

On Mon, Jul 24, 2017 at 5:32 PM il <ber...@gmail.com> wrote:

> Can you help me best solution?

Create a perfect hash function in 28 bits. Use a [1<<28]T array for the map values.

--

-j

[Klaus Post]

On Monday, 24 July 2017 17:32:48 UTC+2, il wrote:

I have task:

1. Initialize map(250 million items)

2. Fill all values

3. Start http server.

4. Process requests. Iterates all values.

I need fast "for" and read access with reasonable memory consumption. This is very important.

If you iterate all entries, why are you using a map and not a slice?

 

Can you help me best solution?

You need to be a bit more specific. This sound like something where you can easily share your existing code and describe the actual problem you are trying to solve as well as which problems you are having.

Thank you!

/Klaus 

[roger peppe]

If it's hard to make a perfect hash function, and assuming you don't need modification after initialisation, you could use struct {keys []uint32; vals []uint64}, sort the keys and their associated values, then binary search. By storing keys and values in different slices, you save 25% space because the keys can be packed together.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.