Capped collection

[Tomasz Prus]

I have created manually capped collection, i'm using MongoDB Java
Driver in my application and I have seen that after some time my
capped collection there is no more capped. I don't know how it is
possible and don't have any idea what could change this capped
collection into normal one. I was doing some update operations on
multiple objects but could it be the reason? Other thing is that that
problem appear only on live server when we have 64 bit version of
MongoDB installed (32 bit version on other machines).

Best regards,
Tomasz Prus

[Nat]

How do you know that? Did anything change in db.printCollectionStat()

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Scott Hernandez]

Are you using a replicaset, did you switch primary/secondaries? What
version are you using? Is it possible that you accidentally deleted
the capped collection and then started inserting without creating it
as capped first?

Can you describe the steps you are doing that show this problem? Can
you reproduce it?

[Tomasz Prus]

We are using natural order on capped collection and we suddenly that
we have lost good order of objects, so then I have executed such
command:

> db.events.stats();
{
"ns" : "sls.events",
"count" : 75640,
"size" : 36167352,
"avgObjSize" : 478.151136964569,
"storageSize" : 52067840,
"numExtents" : 9,
"nindexes" : 6,
"lastExtentSize" : 13890560,
"paddingFactor" : 1.0099999999996714,
"flags" : 1,
"totalIndexSize" : 24584192,
"indexSizes" : {
"_id_" : 3276800,
"resource_1" : 5660672,
"accountId_1" : 3923968,
"projectId_1" : 3776512,
"userFromId_1" : 3751936,
"userToId_1" : 4194304
},
"ok" : 1
}
>

and I don't see any info about capped but i'm sure we have created
capped collection beforehand

2011/2/15 Nat <nat....@gmail.com>:

[Nat]

I don't think you set it up as capped collection in the first place.
The stat doesn't seem to look like a capped collection. Do you still
have the log file on the day you created the collection?

Note that
db.createCollection("hello1", { capped:true}) works
db.createCollection("hello2", { capped:1}) doesn't works

> 2011/2/15 Nat <nat.lu...@gmail.com>:

>
>
>
>
>
>
>
> > How do you know that? Did anything change in db.printCollectionStat()
> > -----Original Message-----
> > From: Tomasz Prus <to.p...@gmail.com>
> > Sender: mongod...@googlegroups.com
> > Date: Mon, 14 Feb 2011 23:10:14
> > To: mongodb-user<mongod...@googlegroups.com>
> > Reply-To: mongod...@googlegroups.com
> > Subject: [mongodb-user] Capped collection
>
> > I have created manually capped collection, i'm using MongoDB Java
> > Driver in my application and I have seen that after some time my
> > capped collection there is no more capped. I don't know how it is
> > possible and don't have any idea what could change this capped
> > collection into normal one. I was doing some update operations on
> > multiple objects but could it be the reason? Other thing is that that
> > problem appear only on live server when we have 64 bit version of
> > MongoDB installed (32 bit version on other machines).
>
> > Best regards,
> > Tomasz Prus
>
> > --
> > You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.

[Scott Hernandez]

Yep, nothing indicates a capped collection. You can use the $natural
index and things will be ordered fine as long as you don't delete or
update (and increase the size of your docs); it will basically be
insertion order if you don't do those two things in a normal
collection. Did you recently do more of those two things before the
order started to change?

If you are using the default _id field as an ObjectId? If so you might
want to sort on that for a more stable order.

Also, if the collection already existed trying to create it as capped
won't work, and in some drivers may not cause an error either.

[Tomasz Prus]

I don't know yet what is replicaset, so probably I don't use it.

Is there a possibility we have started application for a while before
we created capped collection and after that we created capped
collection from command line tool and omit information that it doesn't
work?

Fragment of first logs:

Mon Jan 31 14:33:35 MongoDB starting : pid=15788 port=27017
dbpath=/var/data/backend/mongodb/ 64-bit
Mon Jan 31 14:33:35 db version v1.6.5, pdfile version 4.5
Mon Jan 31 14:33:35 git version: 0eb017e9b2828155a67c5612183337b89e12e291
Mon Jan 31 14:33:35 sys info: Linux domU-12-31-39-06-79-A1
2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64
BOOST_LIB_VERSION=1_41
Mon Jan 31 14:33:35 [initandlisten] waiting for connections on port 27017
Mon Jan 31 14:33:35 [websvr] web admin interface listening on port 28017
Mon Jan 31 14:33:37 [initandlisten] connection accepted from 127.0.0.1:34657 #1
Mon Jan 31 14:38:23 [initandlisten] connection accepted from 127.0.0.1:56462 #2
Mon Jan 31 14:53:34 [conn1] ClientCursor::find(): cursor not found in
map 9005583469156468252 (ok after a drop)
ClientCursor::find(): cursor not found in map 2843747816918766023 (ok
after a drop)
ClientCursor::find(): cursor not found in map 2473232030654207180 (ok
after a drop)
ClientCursor::find(): cursor not found in map 8379976668806945077 (ok
after a drop)
killcursors: found 33 of 102
Mon Jan 31 15:43:54 [conn2] ClientCursor::find(): cursor not found in
map 7675165539032524423 (ok after a drop)
ClientCursor::find(): cursor not found in map 8850855781967596134 (ok
after a drop)
ClientCursor::find(): cursor not found in map 5655076359109637518 (ok
after a drop)
ClientCursor::find(): cursor not found in map 6906784400742219905 (ok
after a drop)
ClientCursor::find(): cursor not found in map 8275119629303116791 (ok
after a drop)
killcursors: found 23 of 101

2011/2/15 Tomasz Prus <to....@gmail.com>:

[Scott Hernandez]

It is possible, but the shell is pretty good about warning you.

Here is an example of what you would have got:

> db.createCollection("test01", capped = true)
{ "errmsg" : "collection already exists", "ok" : 0 }

[Tomasz Prus]

Hmm it would be strange if we ignored such a warning... but if we
created capped collection manually we have given it size of 100 000
000 and if we let application create a normal collection size should
be different, can I check it in some way?

2011/2/15 Scott Hernandez <scotthe...@gmail.com>:

[Scott Hernandez]

It isn't capped. No way to know what you did, but I can tell it wasn't
created as a capped collection from the stats.

db.events.stats() -- it lists the size and other things, like if it is capped.

I would suggest you use the convertToCapped command to convert it but
it will lock everything while you do it.
http://www.mongodb.org/display/DOCS/List+of+Database+Commands

[Tomasz Prus]

Ahh, ok.

If I use convertToCapped will I have good order of objects?

Or maybe I should rename old collection, create new one (capped) and
import objects from old one to new one and if I use sorting by default
_id while doing import I can get back objects to proper order
(natural), am I wright?

If it is not a problem could You tell me how to do it all from command
line client (mongo)?

Thank You a lot

[Scott Hernandez]

If you want them sorted by _id then just sort them by _id when you use them.

On Tue, Feb 15, 2011 at 12:33 AM, Tomasz Prus <to....@gmail.com> wrote:
> Ahh, ok.
>
> If I use convertToCapped will I have good order of objects?

It will not order them by _id.

> Or maybe I should rename old collection, create new one (capped) and
> import objects from old one to new one and if I use sorting by default
> _id while doing import I can get back objects to proper order
> (natural), am I wright?

Yes, if you want them ordered by _id then you have to do it manually.

> If it is not a problem could You tell me how to do it all from command
> line client (mongo)?

Depending on the data types you should probably do this from java. The
shell is not type complete and it could incorrectly convert types on
you.

[Tomasz Prus]

I simply need inserting order so I think of use default _id to sort
data that I have gather while collection was not capped, is it good
way?

I would prefer use command line client, types are not a problem in my
case I think but if You could help me with commands? See that, is it
good way?

db.events.renameCollection("events_old", true)
db.createCollection("events", {capped=true, 100 000 000})
db.events_old.find( sorting here ).forEach(function (d) {db.events.insert(d)}

2011/2/15 Scott Hernandez <scotthe...@gmail.com>:

[Scott Hernandez]

On Tue, Feb 15, 2011 at 1:12 AM, Tomasz Prus <to....@gmail.com> wrote:
> I simply need inserting order so I think of use default _id to sort
> data that I have gather while collection was not capped, is it good
> way?

It will be close to insertion order, as close as you will get.

> I would prefer use command line client, types are not a problem in my
> case I think but if You could help me with commands? See that, is it
> good way?
>
> db.events.renameCollection("events_old", true)
> db.createCollection("events", {capped=true, 100 000 000})
> db.events_old.find( sorting here ).forEach(function (d) {db.events.insert(d)}

Yep, that is the basic code; here is a slightly modified version,
which you should test first.

db.events.renameCollection("events_old", true)
db.createCollection("events", capped=true, size=100*1024*1024)
db.events_old.find().sort({_id:1}).forEach(function (d) {db.events.insert(d)})