Where is the bottleneck? How to get more performance.

385 views
Skip to first unread message

jdill

unread,
Dec 30, 2010, 2:10:12 PM12/30/10
to mongodb-user
Can anyone see a way to improve performance here?

We have a C client on same server as mongod. The C client accepts a
tcp data stream, and for each record, we do a query on 1 collection,
then an upsert to another using getlasterror to both verify our write
and to help us ensure order of insertion on an incrementing field 'i'
for sorting. We generate the 'i' clientside. The collection we are
writing to looks like this:

> db.NY2010_tag.stats()
{
"ns" : "rt01.NY2010_tag",
"count" : 683344,
"size" : 96660920,
"avgObjSize" : 141.45279683439088,
"storageSize" : 111137536,
"numExtents" : 12,
"nindexes" : 5,
"lastExtentSize" : 25047552,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 225468416,
"indexSizes" : {
"_id_" : 38617088,
"tag_1_i_1" : 35078144,
"point_1_i_1" : 48881664,
"wavecode_1_i_1" : 51863552,
"sex_1_age_1_i_1" : 51027968
},
"ok" : 1
}

While running, Mongostat looks like this:

insert/s query/s update/s delete/s getmore/s command/s flushes/s
mapped vsize res faults/s locked % idx miss % q t|r|w
conn time
1196 3569 0 0 0 5945 0
3231 4404 672 0 30.4 0 0|0|0 86
08:40:15
1259 3646 0 0 0 6032 0
3231 4404 673 0 30.6 0 0|0|0 86
08:40:16
1289 3701 0 0 0 6114 0
3231 4404 673 0 30.8 0 0|0|0 86
08:40:17

You can't see it in mongostat, but on average, there are about
1500-2000 inserts that fail because of duplicates...so we are actually
doing about 3500 write attempts per second, aprox half of which are
successful.

iostat -x 2
avg-cpu: %user %nice %system %iowait %steal %idle
4.78 0.00 4.47 0.00 0.00 90.75

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
avgqu-sz await svctm %util
hda 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
hda1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
hda2 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
xvdb 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
xvdc 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
xvdl 0.00 0.00 0.00 1.00 0.00 12.00
12.00 0.00 1.50 1.50 0.15
xvdm 0.00 0.00 0.00 1.00 0.00 12.00
12.00 0.00 1.50 1.50 0.15
xvdk 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
xvdj 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
md0 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
md1 0.00 0.00 0.00 0.50 0.00 8.00
16.00 0.00 0.00 0.00 0.00
md2 0.00 0.00 0.00 0.50 0.00 8.00
16.00 0.00 0.00 0.00 0.00


In top, we see this:
4227 mongod 15 0 3838m 517m 514m S 99.8 2.3 8:26.16 mongod

But I dont think top is accurate...

The machine is an EC2 HVM compute cluster. cc1.4xlarge

mongod version 1.65

Any ideas on how we can get more speed out of this? If we don't do
the initial query before each insert, and if we don't do getLastErr on
write, we can process 20K-40K per second, half of which are dupes. It
is possible that we could eliminate the initial query, but if we use
either getLastErr or the initial query, we get knocked down to what
you see above.

We have been using getLastError both to ensure we do the write, and
also so that we can have some assurance that items actually get
committed in the order of our incrementing counter.

I don't think we can throw any more hardware at it than the
cc1.4xlarge.

Thanks in advance!

Jeremy

Eliot Horowitz

unread,
Dec 30, 2010, 2:19:03 PM12/30/10
to mongod...@googlegroups.com
Sounds like if you were doing the writes multi-threaded it would help.
Is that an option?

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

jdill

unread,
Dec 30, 2010, 8:44:55 PM12/30/10
to mongodb-user
It is actually multithreaded, but, sadly, we had to resort to locking
each thread to ensure only 1 thread is writing to the db at a time.
We did this so that we can get an incrementing index across all
threads and to have the insert happen in order. We can't have a
situation where, say, 'i' #231 gets committed and is readable by
other clients before 'i' #230. We used to use capped collections for
natural order but found them to be too inflexible, and we still didn't
have a way to sort other than $natural which was very slow in some
cases.

thx

Nat

unread,
Dec 30, 2010, 8:55:40 PM12/30/10
to mongodb-user
Can you describe more about the exact logic you are doing, how you
determine whether you insert or update as well as document structure?
There might be a better way to do this.

Dwight Merriman

unread,
Dec 30, 2010, 9:04:17 PM12/30/10
to mongod...@googlegroups.com
you can use the BSON Timestamp datatype, which has special semantics, to get incrementing values on inserts.

perhaps that would work?  i think bson_timestamp in the C driver. if you pass in a value of zero for the timestamp type, the server will set it to a combination of the server's current time plus an incrementing ordinal.

Eliot Horowitz

unread,
Dec 30, 2010, 9:50:15 PM12/30/10
to mongod...@googlegroups.com
one option is to have a thread pool doing the initial queries, then a
thread doing the inserts without get last error.

Can you describe exactly the semabtics you need?

On Thu, Dec 30, 2010 at 8:44 PM, jdill <jer...@dilltree.com> wrote:

jdill

unread,
Dec 31, 2010, 9:17:38 AM12/31/10
to mongodb-user
Thanks for helping here.

---
Nat/Eliot - This thread goes into detail of what we need to do:
http://groups.google.com/group/mongodb-user/browse_thread/thread/6fef26ed628660ea/d419f9f1ea32f4da?lnk=gst&q=jdill#d419f9f1ea32f4da

We provide live tracking for large running events (marathons).
Document looks like this:

{ "_id" : "10K~10004", "ts" : 1293735514, "point" : "10K", "tag" :
10004, "time" : 1289127697.78, "sex" : "M", "age" : 49, "wavecode" :
"39", "name" : "STUART SMALLY", "i" : 1443109 }

We just do an insert and let it fail on duplicates _id.

This service has to act almost as a conduit for a stream of
information. Records come in and we need to publish them instantly.
Any added latency is unfortunate, but may be tolerable if it increases
reliability. As soon as a runner gets scanned via RFID, we accept
that tag read in a tcp stream and we need it to be inserted and
available for query. We then have a web API which handles thousands
of requests per second. The API is setup to generally read only from
the secondaries.

---
Dwight. I have been looking for a way to do a server side timestamp.
I saw that you could do the BSON timestamp in java but I've been
looking for a way to do it. We will definitely try it in c. I've
commented before on this JIRA - http://jira.mongodb.org/browse/SERVER-1650


Thx..

Jeremy

Eliot Horowitz

unread,
Dec 31, 2010, 9:20:39 AM12/31/10
to mongod...@googlegroups.com
So basically you just need to poll inserts?
One option is just to poll the replication oplog.
You can only poll for inserts to a certain collection.

Nat

unread,
Dec 31, 2010, 9:36:08 AM12/31/10
to mongodb-user
How did you generate _id? Is that why you can't just simply do upsert
in atomic operation?

On Dec 31, 10:17 pm, jdill <jer...@dilltree.com> wrote:
> Thanks for helping here.
>
> ---
> Nat/Eliot -  This thread goes into detail of what we need to do:http://groups.google.com/group/mongodb-user/browse_thread/thread/6fef...

Brendan W. McAdams

unread,
Dec 31, 2010, 9:38:03 AM12/31/10
to mongod...@googlegroups.com

You said you are generating I clientside. Have you tried using $inc instead to increment ? This should help with the need to employ locking. 

jdill

unread,
Dec 31, 2010, 10:11:32 AM12/31/10
to mongodb-user
It's more complex then that because we are not just trying to get the
most recent inserts for a collection. We use indexes on basically
every request. Some API queries filter by tag, some by point, some by
wavecode, etc, or a combination of fields. All queries return filtered
lists, and API supports pagination such as skip first 10 records, max
20 records, etc.

jdill

unread,
Dec 31, 2010, 10:15:29 AM12/31/10
to mongodb-user
We can't use upsert because we don't want to update existing records.
We only want to save record the first time we see it. To accomplish
this in mongo, there are only 2 ways that I know of to accomplish
'insert only once'. You can query the collection to see if a record
exists and insert if not, or you can just let it fail on duplicates.
The former is a wasted round trip...too slow.

Eliot Horowitz

unread,
Dec 31, 2010, 10:15:40 AM12/31/10
to mongod...@googlegroups.com
Ok, so Timestamp is probably best.
Its not a widely used feature, so if you run into trouble let us know.

jdill

unread,
Dec 31, 2010, 10:18:17 AM12/31/10
to mongodb-user
We don't want to update if exists. We only want to insert if not
exist. I only know of 2 ways to do 'insert if not exist' with
mongo...either you have to query collection to check if a record
exists, and insert if not, or you can just let it fail on the dupe.
The former is a wasted round trip...too slow.

On Dec 31, 8:36 am, Nat <nat.lu...@gmail.com> wrote:

jdill

unread,
Dec 31, 2010, 10:20:52 AM12/31/10
to mongodb-user
Thanks Brendan, but I don't think $inc can work here. $inc is for
updating an existing document and not when creating a new one.

Sorry for the double post..



On Dec 31, 8:38 am, "Brendan W. McAdams" <bren...@10gen.com> wrote:
> You said you are generating I clientside. Have you tried using $inc instead
> to increment ? This should help with the need to employ locking.
> On Dec 31, 2010 9:17 AM, "jdill" <jer...@dilltree.com> wrote:> Thanks for helping here.
>
> > ---
> > Nat/Eliot - This thread goes into detail of what we need to do:
>
> http://groups.google.com/group/mongodb-user/browse_thread/thread/6fef...
>
>
>
>
>
>
>
>
>
> > We provide live tracking for large running events (marathons).
> > Document looks like this:
>
> > { "_id" : "10K~10004", "ts" : 1293735514, "point" : "10K", "tag" :
> > 10004, "time" : 1289127697.78, "sex" : "M", "age" : 49, "wavecode" :
> > "39", "name" : "STUART SMALLY", "i" : 1443109 }
>
> > We just do an insert and let it fail on duplicates _id.
>
> > This service has to act almost as a conduit for a stream of
> > information. Records come in and we need to publish them instantly.
> > Any added latency is unfortunate, but may be tolerable if it increases
> > reliability. As soon as a runner gets scanned via RFID, we accept
> > that tag read in a tcp stream and we need it to be inserted and
> > available for query. We then have a web API which handles thousands
> > of requests per second. The API is setup to generally read only from
> > the secondaries.
>
> > ---
> > Dwight. I have been looking for a way to do a server side timestamp.
> > I saw that you could do the BSON timestamp in java but I've been
> > looking for a way to do it. We will definitely try it in c. I've
> > commented before on this JIRA -http://jira.mongodb.org/browse/SERVER-1650
>
> > Thx..
>
> > Jeremy
>
> > --
> > You received this message because you are subscribed to the Google Groups
>
> "mongodb-user" group.> To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to
>
> mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> .> For more options, visit this group at
>
> http://groups.google.com/group/mongodb-user?hl=en.
>
>
>
>
>
>
>
>

jdill

unread,
Dec 31, 2010, 10:22:51 AM12/31/10
to mongodb-user
Eliot.. Will do. Thanks.

Nat

unread,
Dec 31, 2010, 10:23:56 AM12/31/10
to mongodb-user
As you mention that there is a synchronization on the client, too, I
assume you use only one client to update mongodb. If so, why don't you
queue the update and do it from the client? That way, you won't need
to wait for getLastError.

Dwight Merriman

unread,
Dec 31, 2010, 10:46:00 AM12/31/10
to mongod...@googlegroups.com

jdill

unread,
Dec 31, 2010, 10:51:38 AM12/31/10
to mongodb-user
Yeah, I guess this is where I don't know how mongo internally works.
If I use a single client to do inserts one after another without
getLastError, as far as the client is concerned...its fire and forget,
so I then have to have faith that mongo will respect the order.
Mongo accepts the insert commands in order, but is it still possible
for mongo to commit things out of order? Does mongo have a single
thread per client and give some insurance that queue commands from
that client will complete in order of receipt?

Thanks,

Eliot Horowitz

unread,
Dec 31, 2010, 11:10:27 AM12/31/10
to mongod...@googlegroups.com
All operations on a single socket are guaranteed to be processed in
the order they are received.
Across many sockets, there is no order guarantee.

jdill

unread,
Dec 31, 2010, 11:19:24 AM12/31/10
to mongodb-user
With timestamp, when doing queries, would there be a way to trick the
client into using a server side timestamp also?
Like..would there be a way to do some kind of magic trick such as

.find({ "ts" : { $lte: new Timestamp()-1 }}.sort({ts:1})

But have Timestamp-1 be generated on server side?

Probably not.

I guess this doesn't matter, if I can just be sure things are always
going in order when coming from one client. I can just sort by ts...

Curious, what does the BSON timestamp type 0 do when there are many
connections inserting? How would it create a "combination of the
server's current time plus an incrementing ordinal." Would it
sometimes end up using the same ordinal? How does it increment?

Guess we just need to try it out.

thx.

- Jeremy



On Dec 31, 9:46 am, Dwight Merriman <dwi...@10gen.com> wrote:
> http://www.mongodb.org/display/DOCS/Timestamp+Data+Type
>
>
>
>
>
>
>
> On Fri, Dec 31, 2010 at 10:22 AM, jdill <jer...@dilltree.com> wrote:
> > Eliot.. Will do.  Thanks.
>
> > On Dec 31, 9:15 am, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> > > Ok, so Timestamp is probably best.
> > > Its not a widely used feature, so if you run into trouble let us know.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > .

jdill

unread,
Dec 31, 2010, 11:27:27 AM12/31/10
to mongodb-user
OK.. thats good to know. Next question, does replset replication
always happen in this same order too? Will secondaries always have
same order of processing too?

Thx,

Jeremy

Eliot Horowitz

unread,
Dec 31, 2010, 11:35:18 AM12/31/10
to mongod...@googlegroups.com
> Curious, what does the BSON timestamp type 0 do when there are many
> connections inserting?  How would it create a "combination of the
> server's current time plus an incrementing ordinal."   Would it
> sometimes end up using the same ordinal?  How does it increment?

Its always increasing and guaranteed to be unique.
Implemented just like auto_incrememnt in RDBMs


>
> Guess we just need to try it out.
>
> thx.
>
> - Jeremy
>
>
>
> On Dec 31, 9:46 am, Dwight Merriman <dwi...@10gen.com> wrote:
>> http://www.mongodb.org/display/DOCS/Timestamp+Data+Type
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Dec 31, 2010 at 10:22 AM, jdill <jer...@dilltree.com> wrote:
>> > Eliot.. Will do.  Thanks.
>>
>> > On Dec 31, 9:15 am, Eliot Horowitz <eliothorow...@gmail.com> wrote:
>> > > Ok, so Timestamp is probably best.
>> > > Its not a widely used feature, so if you run into trouble let us know.
>>
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "mongodb-user" group.
>> > To post to this group, send email to mongod...@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
>> > .
>> > For more options, visit this group at
>> >http://groups.google.com/group/mongodb-user?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.

> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

Nat

unread,
Dec 31, 2010, 11:35:24 AM12/31/10
to mongodb-user
Yes and yes.

Eliot Horowitz

unread,
Dec 31, 2010, 11:35:43 AM12/31/10
to mongod...@googlegroups.com
Yes, same with replset.

For writes coming into a master on a repl set, a slave is guaranteed
to apply them in the same order.

jdill

unread,
Dec 31, 2010, 11:41:29 AM12/31/10
to mongodb-user
Hrm.. we are looking at the latest c driver now at
https://github.com/mongodb/mongo-c-driver but not seeing how to use
the bson timestamp.

mongo-c-driver$ grep -Ri ".*bson_timestamp.*" *
src/bson.c: case bson_timestamp:
src/bson.h: bson_timestamp = 17,

Any suggestions?

jdill

unread,
Dec 31, 2010, 11:43:10 AM12/31/10
to mongodb-user
"Its always increasing and guaranteed to be unique.
Implemented just like auto_incrememnt in RDBMs "

Wow, so if this exists, is there just some way to call it without
messing with a timestamp, etc? This is exactly what I need...a simple
serverside auto_increment.

Eliot Horowitz

unread,
Dec 31, 2010, 1:46:10 PM12/31/10
to mongod...@googlegroups.com
Looks like its not fully supported in the c driver.
Can you open a jira?

There isn't a way to get the functionality without it.

If you're using a single thread though, you shouldn't need to call getLastError.

You can call once every second or something to make sure things are
going in fine, but not after every call.

jdill

unread,
Dec 31, 2010, 3:00:53 PM12/31/10
to mongodb-user
Thanks everybody. Big help. Looking forward to bson_timestamp in c
driver.

Here is JIRA. http://jira.mongodb.org/browse/CDRIVER-32

On Dec 31, 12:46 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Looks like its not fully supported in the c driver.
> Can you open a jira?
>
> There isn't a way to get the functionality without it.
>
> If you're using a single thread though, you shouldn't need to call getLastError.
>
> You can call once every second or something to make sure things are
> going in fine, but not after every call.
>
>
>
>
>
>
>
> On Fri, Dec 31, 2010 at 11:41 AM, jdill <jer...@dilltree.com> wrote:
> > Hrm.. we are looking at the latest c driver now at
> >https://github.com/mongodb/mongo-c-driverbut not seeing how to use

jdill

unread,
Jan 6, 2011, 5:18:13 PM1/6/11
to mongodb-user
Got latest C and was able to create the server side ts. However, I am
not sure where to from here. If I have this:

"ts" : { "t" : 1294348773000, "i" : 339 }

...and we index ts, will sorting on ts work?



On Dec 31 2010, 2:00 pm, jdill <jer...@dilltree.com> wrote:
> Thanks everybody.  Big help.   Looking forward to bson_timestamp in c
> driver.
>
> Here is JIRA.  http://jira.mongodb.org/browse/CDRIVER-32
>
> On Dec 31, 12:46 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Looks like its not fully supported in the c driver.
> > Can you open a jira?
>
> > There isn't a way to get the functionality without it.
>
> > If you're using a single thread though, you shouldn't need to call getLastError.
>
> > You can call once every second or something to make sure things are
> > going in fine, but not after every call.
>
> > On Fri, Dec 31, 2010 at 11:41 AM, jdill <jer...@dilltree.com> wrote:
> > > Hrm.. we are looking at the latest c driver now at
> > >https://github.com/mongodb/mongo-c-driverbutnot seeing how to use

Dwight Merriman

unread,
Jan 6, 2011, 5:46:04 PM1/6/11
to mongod...@googlegroups.com
yes

jdill

unread,
Jan 8, 2011, 1:37:28 PM1/8/11
to mongodb-user
Ok. This is working great! One point I forgot about is that it has
to be one of the first 2 fields.

Now trying to get the php driver to do the same thing. It almost
works, but seems to be a bit funky.

If I do,

'i'=>new MongoTimestamp()

I get something weird like this:

{ "_id" : ObjectId("4d28ad978ead0e6f36000008"), "i" : { "t" : 3000,
"i" : 1294511511 } }

'i'=>new MongoTimestamp(0)

gives me

{ "_id" : ObjectId("4d28ae4a8ead0e6c3600000a"), "i" : { "t" : 3000,
"i" : 0 } }

But I found that I can do this:

'i'=>new MongoTimestamp(array())

..and it works.

{ "_id" : ObjectId("4d28ae738ead0e6e3600000b"), "i" : { "t" :
1294511731000, "i" : 1 } }


BUT, it throws a php warning of:

Warning: MongoTimestamp::__construct() expects parameter 1 to be long,
array given in...

Any way to make this work without the Warning?

Thanks,

Jeremy

On Jan 6, 4:46 pm, Dwight Merriman <dwi...@10gen.com> wrote:
> yes
>
>
>
>
>
>
>
> On Thu, Jan 6, 2011 at 5:18 PM, jdill <jer...@dilltree.com> wrote:
> > Got latest C and was able to create the server side ts.  However, I am
> > not sure where to from here.  If I have this:
>
> >  "ts" : { "t" : 1294348773000, "i" : 339 }
>
> > ...and we index ts, will sorting on ts work?
>
> > On Dec 31 2010, 2:00 pm, jdill <jer...@dilltree.com> wrote:
> > > Thanks everybody.  Big help.   Looking forward to bson_timestamp in c
> > > driver.
>
> > > Here is JIRA.  http://jira.mongodb.org/browse/CDRIVER-32
>
> > > On Dec 31, 12:46 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
>
> > > > Looks like its not fully supported in the c driver.
> > > > Can you open a jira?
>
> > > > There isn't a way to get the functionality without it.
>
> > > > If you're using a single thread though, you shouldn't need to call
> > getLastError.
>
> > > > You can call once every second or something to make sure things are
> > > > going in fine, but not after every call.
>
> > > > On Fri, Dec 31, 2010 at 11:41 AM, jdill <jer...@dilltree.com> wrote:
> > > > > Hrm.. we are looking at the latest c driver now at
> > > > >https://github.com/mongodb/mongo-c-driverbutnotseeing how to use
> > > > > the bson timestamp.
>
> > > > > mongo-c-driver$ grep -Ri ".*bson_timestamp.*" *
> > > > > src/bson.c:    case bson_timestamp:
> > > > > src/bson.h:    bson_timestamp = 17,
>
> > > > > Any suggestions?
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > Groups "mongodb-user" group.
> > > > > To post to this group, send email to mongod...@googlegroups.com.
> > > > > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > .
> > > > > For more options, visit this group athttp://
> > groups.google.com/group/mongodb-user?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > .

jdill

unread,
Jan 8, 2011, 1:39:52 PM1/8/11
to mongodb-user
..thought of simple workaround for now.

I can just supress the warning with @:

@$row=array('i'=>new MongoTimestamp(array()));
> > > > > >https://github.com/mongodb/mongo-c-driverbutnotseeinghow to use

jdill

unread,
Jan 8, 2011, 1:51:10 PM1/8/11
to mongodb-user
Also when retrieving, the values of sec and inc are reversed.

{ "_id" : "stime", "i" : { "t" : 1294512437000, "i" : 1 } }

$val=$tcoll->findOne(array('_id'=>'stime'));
print_r($val);

Array ( [_id] => stime [i] => MongoTimestamp Object ( [sec] => 1 [inc]
=> 1294512437 ) )

Should I make a bug report for this?

thx,
> > > > > > >https://github.com/mongodb/mongo-c-driverbutnotseeinghowto use

Eliot Horowitz

unread,
Jan 8, 2011, 2:00:12 PM1/8/11
to mongod...@googlegroups.com
That's just a display issue, its stored as a single value in the database.

> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

jdill

unread,
Jan 8, 2011, 2:16:33 PM1/8/11
to mongodb-user
So you mean it is an issue with the php driver, right? Currently, the
__toString method for MongoTimestamp is supposed to "Returns the "sec"
field of this timestamp." according to docs here:
http://us.php.net/manual/en/mongotimestamp.tostring.php

But when I convert to a string, I end up with the increment, not the
seconds. I need to be able to grab the seconds.

thx,

Eliot Horowitz

unread,
Jan 8, 2011, 3:38:36 PM1/8/11
to mongod...@googlegroups.com
Yes, I think its an issue with the php driver.
Can add a case for it @ http://jira.mongodb.org/browse/PHP

jdill

unread,
Jan 8, 2011, 4:18:55 PM1/8/11
to mongodb-user
http://jira.mongodb.org/browse/PHP-192

On Jan 8, 2:38 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Yes, I think its an issue with the php driver.
> Can add a case for it @http://jira.mongodb.org/browse/PHP

jdill

unread,
Jan 20, 2011, 4:52:00 PM1/20/11
to mongodb-user
Just want to report that this setup is working very well. Everything
is much faster and we are back to writing on multithreads. I really
think that this auto-increment should be an advertised feature. This
makes a huge difference for us and I imagine other people could
benefit from it as well.

Jeremy

jdill

unread,
Jan 20, 2011, 4:56:31 PM1/20/11
to mongodb-user
A little trick I am doing with this now is getting the "mongo DB
server time" with php like this:

$tcoll = $db->$coll;
@$row=array('_id'=>'stime','ts'=>new MongoTimestamp(array()));
$tcoll->save($row,array('safe'=>1));
$val=$tcoll->findOne(array('_id'=>'stime'));
return (int)(string)$val['ts'];

Works, but wonder if there a more built-in/secret way to do this?

Jeremy

Dwight Merriman

unread,
Jan 20, 2011, 5:44:22 PM1/20/11
to mongod...@googlegroups.com
i'm curious why you want the exact server time.  but db.eval() would be good to get it:

$mongo
MongoDB shell version: 1.7.5-pre-
connecting to: test
> db.eval( "new Date()" )
ISODate("2011-01-20T22:43:42.347Z")

jdill

unread,
Jan 27, 2011, 3:41:54 PM1/27/11
to mongodb-user
That works better. Thanks!

My reason for doing this is a bit involved, but generally, since I am
dealing with elapsed times and such, I need a master clock source for
all clients since there is no reliable time sync possible when using
virtual machines. Also, since I now have timestamps in my records, I
want the time to be coming from the system that generated the
timestamps.

Whenever I run this, I see that a command is executed. Does this
'eval' do any kind of locking or blocking that I should be aware of?
I'm trying to decide just how often I can go back and be doing this.

Thanks,

Jeremy

Dwight Merriman

unread,
Jan 27, 2011, 3:43:56 PM1/27/11
to mongod...@googlegroups.com
it does lock, but the eval above is fast.  a thousand of these evals times per second would be bad, one per second is no problem.



To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages