mongo output in golang map

[sundar.v...@gmail.com]

Hello,

We have documents where the id is a date 

{

 _id : "20150604"

name: "test"

..

},

{

 _id: "201500603"

name: "test2"

..

}

Currently, I read this into a structure 

type dataStruct struct {

id string 

name string

}

However, we I want to store it in a map  (map[string]dataStruct) when I get it back from mgo (without additional looping after receiving it from mgo). This is more a mongo question.

Any thoughts on how I can do that?

Thanks

Sundar.

[Gustavo Niemeyer]

Hi Sundar,

There's no way to do that without iterating over the documents. It's not clear why you'd want to avoid such a trivial loop, though?

    for iter.Next(&value) {

            m[value.Id] = value

    }

    if err := iter.Err(); err != nil {

            return err

    }

--
You received this message because you are subscribed to the Google Groups "mgo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mgo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

gustavo @ http://niemeyer.net

[Sundar Vadivel]

Hi Gustavo,

We are running our go server on multiple small systems. Each system has only 512 MB RAM. When we are testing our server using abtesting, and we bombard our requests: go server runs out of memory.

- So I’m checking how to reduce the number of variables we use. (In this URL, we get around 2000 documents in one request and when we bombard with 5 clients, we see the issue)

- The second one is how to limit the go routines safely from being spawned.. I’m still searching for the correct way to limit it. I’m using goji framework. Any help on that would be great!

I know that these variables should be trivial, hence I used them.. now with limited memory, I m trying to profile my code. Meanwhile, any profiling links would also be very helpful!

Thanks..

Sundar.

--
You received this message because you are subscribed to a topic in the Google Groups "mgo-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mgo-users/VG4eC0ISeWw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mgo-users+...@googlegroups.com.

[Gustavo Niemeyer]

On Thu, Jun 4, 2015 at 12:58 PM, Sundar Vadivel <sundar.v...@gmail.com> wrote:

Hi Gustavo,

We are running our go server on multiple small systems. Each system has only 512 MB RAM. When we are testing our server using abtesting, and we bombard our requests: go server runs out of memory.

- So I’m checking how to reduce the number of variables we use. (In this URL, we get around 2000 documents in one request and when we bombard with 5 clients, we see the issue)

That sounds very much like the request handler is putting too much data in memory. How long does it take for each of these requests to be handled, and are they loading it all in memory at once to work with? What's the size of each of these 2000 documents, and how long does it take to find and load them in memory?

 

- The second one is how to limit the go routines safely from being spawned.. I’m still searching for the correct way to limit it. I’m using goji framework. Any help on that would be great!

Before saying anything about this, please note the above point. If the server is not using information in an efficient way, you'll have problems again sooner or later.

With that out of the way, the proper way to restrict concurrency is at the door. In other words, don't accept more connections than the server can handle, otherwise eventually the server resources will be exhausted and provide a terrible experience even for those requests that manage to go in.

If you are relying on the standard net/http package, have a look at the code for the Server.Serve method. You'll see it manipulating a listener and accepting connections. An alternative implementation that does not accept connections out of a limit should be used instead. If you are not using net/http, the same idea holds: find who's accepting connections and preventing it from taking requests out of the limit.

Long ago I've provided this snippet as part of an old thread:

http://play.golang.org/p/hy9ouVmtKk

Not sure if it still works, but it should serve as a hint at least.

I know that these variables should be trivial, hence I used them.. now with limited memory, I m trying to profile my code. Meanwhile, any profiling links would also be very helpful!

This is the most well known one:

    http://blog.golang.org/profiling-go-programs

gustavo @ http://niemeyer.net