[ANN] mgo r2011.10.29, with simplified iteration

286 views
Skip to first unread message

Gustavo Niemeyer

unread,
Oct 29, 2011, 1:02:27 PM10/29/11
to golan...@googlegroups.com, mongod...@googlegroups.com, mgo-...@googlegroups.com
Greetings,
After a few weeks without new releases, here is a freshnew release
ofthe mgo MongoDB driver for Go with someuseful improvements.The
project page with details is available at:

    http://labix.org/mgo
The focus of release r2011.10.29 is a much needed simplificationon
theway result sets are iterated over. Please read throughsince this
isnot a backwards compatible change. I apologizefor the trouble
inadvance, but I'm sure you'll appreciate it.
The changes made are:

- Simplified the Iter interface significantly. Previously, an
iteration loop would look something like this:

iter, err := query.Iter()
if err != nil {
panic(err)
}
for {
err := iter.Next(&result)
if err == mgo.NotFound {
break
}
if err != nil {
panic(err)
}
...
}

  Now, the same loop is written as:

iter := query.Iter()
for iter.Next(&result) {
...
}
if iter.Err() != nil {
panic(iter.Err())
}

  See more details in the documentation:

      http://goneat.org/lp/mgo#Iter.Next      http://goneat.org/lp/mgo#Iter.Err
     http://goneat.org/lp/mgo#Query.Iter

- Tailable cursors were also changed accordingly, but were also
simplified further through the introduction of the Iter.Timeout
method that reports whether a false Next is a timeout or not.

  See the documentation for details and an example:
http://goneat.org/lp/mgo#Query.Tail
http://goneat.org/lp/mgo#Iter.Timeout

- New convenience Query.All and Iter.All methods, enabling one to
trivially retrieve the whole result set at once into a slice.

  For example:
      var result []struct{ Value int }
     err := collection.Find(nil).Limit(100).All(&result)
  Please note that the result set must obviously have a bound  size
for All to be used like this, or it will potentially  crash the system
out of memory. Even if you think you know  the set size, consider
limiting the size as done in the example  above just in case, or use
the simplified iteration method  described above.

  More details in the documentation:
      http://goneat.org/lp/mgo#Iter.All      http://goneat.org/lp/mgo#Query.All

- Given the above enhancements, the Iter.For and Query.For methods
are both considered obsolete now, and while they haven't been  touched
to give people some time to migrate away, they will  be removed in a
future release.

- Update and UpdateAll were fixed to handle a nil selector  parameter
in an equivalent way to an empty map.

Enjoy!
--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I never filed a patent.

Gustavo Niemeyer

unread,
Oct 29, 2011, 1:03:42 PM10/29/11
to golan...@googlegroups.com, mongod...@googlegroups.com, mgo-...@googlegroups.com
Ok,

sorry about that.. there's something seriously wrong with copy & paste
and gmail. What I see here isn't what's being sent.

André Paquet

unread,
Oct 29, 2011, 1:05:16 PM10/29/11
to mgo-...@googlegroups.com, golan...@googlegroups.com, mongod...@googlegroups.com
Repost of my original response which seems to have failed: Thank you very much for Query.All and for the great work in general.

André

Gustavo Niemeyer

unread,
Oct 29, 2011, 1:06:48 PM10/29/11
to golan...@googlegroups.com, mongod...@googlegroups.com, mgo-...@googlegroups.com
> sorry about that.. there's something seriously wrong with copy & paste
> and gmail. What I see here isn't what's being sent.

Properly formatted text: http://pastebin.ubuntu.com/722436/

Travis Reeder

unread,
Oct 29, 2011, 3:12:33 PM10/29/11
to mgo-...@googlegroups.com, golan...@googlegroups.com, mongod...@googlegroups.com
Great stuff Gustavo!

One thing though, when I try to update though, I'm getting this:

gobson.go:222: syntax error: unexpected <<
gobson.go:227: syntax error: unexpected ==

Thoughts?

Gustavo Niemeyer

unread,
Oct 29, 2011, 3:26:49 PM10/29/11
to mgo-...@googlegroups.com, golan...@googlegroups.com, mongod...@googlegroups.com
Hey Travis,

Wild guess: you have edited your gobson code locally and got a
conflict while updating.

Read the file and see what is the content of these lines.

Travis Reeder

unread,
Oct 29, 2011, 3:28:08 PM10/29/11
to mgo-...@googlegroups.com
Doh, just realized this google groups interface automatically included the cross posting groups. My post was meant for this group.

Gustavo Niemeyer

unread,
Oct 29, 2011, 3:49:47 PM10/29/11
to mgo-...@googlegroups.com
> Doh, just realized this google groups interface automatically included the
> cross posting groups. My post was meant for this group.

Have you seen the response?

Travis Reeder

unread,
Oct 29, 2011, 9:59:02 PM10/29/11
to mgo-...@googlegroups.com
Ok, looks like you're right, must have had something modified there. Wiped out my ~/go directory and then it worked. 

Pavel Korotkov

unread,
Oct 30, 2011, 8:21:34 AM10/30/11
to mgo-users
Hey Gustavo,
As you made up your mind to perform some refactoring in future mgo
releases I got a proposal:
To separate creation and getting existing databases and collections:
func (s *Session) DB(name string) Database ->
func (s *Session) CreateDataBase(name string) (Database, os.Error)
[Returns error if a db with a given name already exists]
func (s *Session) GetDataBase(name string) (Database, os.Error)
[Returns error if no db with a given name is found]
func (db Database) C(name string) Collection ->
Done in the same manner
At least for me, the consumer's code would become more clean if mgo
API had this methods.

- PK

Fabian Reinartz

unread,
Oct 30, 2011, 9:38:43 AM10/30/11
to mgo-...@googlegroups.com
This approach does not really feel mongodb-like.
If you check out the mongo console you'll see that it works in the
same way as mgo does. And I think it's good - it avoids useless
overhead and a database or a collection is just there if it contains
data. It's as simple as that.

I can't really see a usecase, either. The normal situation is that the
developer provides names for databases. As soon as this is not the
case you might consider writing your own validator for the names.

2011/10/30 Pavel Korotkov <pkor...@gmail.com>:

Pavel Korotkov

unread,
Oct 30, 2011, 11:18:06 AM10/30/11
to mgo-users
Well, I still believe the clarification can simplify business logic
code. The motivation is pretty straightforward, I just need
transparent and go-stylish bindings, but JS shell. We should be more
flexible in wrappers' syntax when it makes API easier to use.

On Oct 30, 5:38 pm, Fabian Reinartz <fab.reina...@googlemail.com>
wrote:
> This approach does not really feel mongodb-like.
> If you check out the mongo console you'll see that it works in the
> same way as mgo does. And I think it's good - it avoids useless
> overhead and a database or a collection is just there if it contains
> data. It's as simple as that.
>
> I can't really see a usecase, either. The normal situation is that the
> developer provides names for databases. As soon as this is not the
> case you might consider writing your own validator for the names.
>
> 2011/10/30 Pavel Korotkov <pkorot...@gmail.com>:

Evan Shaw

unread,
Oct 30, 2011, 9:00:21 PM10/30/11
to mgo-...@googlegroups.com
Are these releases intended to be used with Go release r60? If so, any
chance you could tag them in a way that goinstall will recognize?
Right now it still wants to use the old versions because they're
tagged with "go.r60".

- Evan

Gustavo Niemeyer

unread,
Oct 30, 2011, 10:48:47 PM10/30/11
to mgo-...@googlegroups.com

Oops.. my apologies. The tags have been updated, thanks for the note!

Gustavo Niemeyer

unread,
Oct 30, 2011, 10:53:10 PM10/30/11
to mgo-...@googlegroups.com
Hi Pavel,

> Well, I still believe the clarification can simplify business logic
> code. The motivation is pretty straightforward, I just need
> transparent and go-stylish bindings, but JS shell. We should be more
> flexible in wrappers' syntax when it makes API easier to use.

In MongoDB databases are created lazily when data is inserted in them.
There's no Create operation that mgo can implement in Go style or not
because that operation simply does not exist.

Pavel Korotkov

unread,
Oct 31, 2011, 5:37:36 PM10/31/11
to mgo-users
That's definitely up to one's code design fidelity. I would never have
posted my proposal if hadn't seen this: http://www.mongodb.org/display/DOCS/createCollection+Command
It looks like what I'm talking about =)
> Gustavo Niemeyerhttp://niemeyer.nethttp://niemeyer.net/plushttp://niemeyer.net/twitterhttp://niemeyer.net/blog

Gustavo Niemeyer

unread,
Nov 7, 2011, 8:10:10 AM11/7/11
to mgo-...@googlegroups.com
> (...)

> func (s *Session) CreateDataBase(name string) (Database, os.Error)
> (...)

>
>>In MongoDB databases are created lazily when data is inserted in them.
>>There's no Create operation that mgo can implement in Go style or not
>>because that operation simply does not exist.
>>
>>> That's definitely up to one's code design fidelity. I would never have
>>> posted my proposal if hadn't seen this: http://www.mongodb.org/display/DOCS/createCollection+Command
>>> It looks like what I'm talking about =)

Databases and collections are different beasts in MongoDB (note you
were talking about the former).

Even the creation of collections is more idiomatically done as part of
the insertion of data, besides the rare situations when you want to
create capped collections for instance. I can certainly add a wrapper
to that a command in a future release for those reasons, but it'll
continue to be optional.

Reply all
Reply to author
Forward
0 new messages