[mongodb-csharp] mongodb-sharp against MongoHQ

[Chris Baglieri]

Has anyone had experience hitting MongoHQ? Connecting is no problem,
however, when trying to add documents, I'm unable to do so on the
first attempt, while subsequent attempts work no problem. Am I doing
something wrong? I walked through the source and everything runs as
you'd expect, no exceptions thrown. Thoughts?

private List<MongoDBDriver.Document> AddTagDocuments(int
count) {
MongoDBDriver.Document document = null;
var docs = new List<MongoDBDriver.Document>();
var foos = this.mongoDatabase["Foo"];
for (int documentIndex = 1; documentIndex <= count; +
+documentIndex) {
document = new MongoDBDriver.Document();
document.Add("fooId", documentIndex);
foos.Insert(document);
docs.Add(document);
}
return docs;
}

[Chris Baglieri]

Update. It appears to work the first time if the collection is not
empty.

[Sam Corder]

Not empty or non-existent collections?  This has never been a problem before with a non-hosted server.  The entire test suite can be run against a freshly installed server even.  If the problem is with mongohq and collections that don't exist, you could always call Collection.Metadata.CreateIndex() for all your collections in an initialization routine.  Creating an index will cause the collection to be created as well.

-Sam

[John Zablocki]

I also see the same behavior (no errors, but no inserts), though it
doesn't work even when the collection is created and exists. I'm
using the code below to test out MongoHQ . I had their support look at
this and they see nothing wrong with the code or the server side.
I've had no problems with using a local server and a remote server
that's installed on my domain. Any suggestions on how to debug?

Thanks.

-- John

MongoConnectionStringBuilder sb = new MongoConnectionStringBuilder();
sb.AddServer("my mongohqserver:27055");
sb.Username = "my db user";
sb.Password = "my db password";

Mongo mongo = new Mongo(sb.ToString());
mongo.Connect();

Document doc = new Document();
doc["Testing"] = "A test value";

mongo.GetDatabase("jztest").GetCollection("Tests").Insert(doc);
mongo.Disconnect();

On Apr 30, 2:20 am, Sam Corder <sa...@codeargyle.com> wrote:
> Not empty or non-existent collections?  This has never been a problem before
> with a non-hosted server.  The entire test suite can be run against a
> freshly installed server even.  If the problem is with mongohq and
> collections that don't exist, you could always call
> Collection.Metadata.CreateIndex() for all your collections in an
> initialization routine.  Creating an index will cause the collection to be
> created as well.
>
> -Sam
>
> On Fri, Apr 30, 2010 at 12:03 AM, Chris Baglieri

> <chris.bagli...@gmail.com>wrote:

[Sam Corder]

Is it possible to replicate the behavior with the shell too?  We should be sending the exact same bytes across the wire as the shell or any of the other drivers.  Using a packet capturer would verify that.

[John Zablocki]

When I run the shell, the inserts work fine.

As for network traffic, I freely admit I'm not sure exactly what I'm
looking for here. But I setup Wireshark to watch for the inserts.
The first block is form the shell and the second mongo-csharp.


0000 00 d0 cf 05 96 91 00 22 19 20 b1 74 08 00 45
00 ......." . .t..E.
0010 00 77 5f 84 40 00 80 06 00 00 0a 0a 1e 87 b8
49 .w_.@... .......I
0020 e0 05 f8 27 69 af 20 35 cc 58 a6 4b 7b 72 50 18 ...'i.
5 .X.K{rP.
0030 3f b2 c1 49 00 00 4f 00 00 00 c0 e4 b8 78 ff
ff ?..I..O. .....x..
0040 ff ff d2 07 00 00 00 00 00 00 6a 7a 74 65 73
74 ........ ..jztest
0050 2e 54 65 73 74 73 00 2e 00 00 00 07 5f 69 64
00 .Tests.. ...._id.
0060 4b e1 8c 16 a0 0f 00 00 00 00 14 3d 02 54 65 73
K....... ...=.Tes
0070 74 69 6e 67 00 0b 00 00 00 54 65 73 74 69 6e 67
ting.... .Testing
0080 41 42 43 00 00
ABC..

0000 00 d0 cf 05 96 91 00 22 19 20 b1 74 08 00 45
00 ......." . .t..E.
0010 00 67 60 31 40 00 80 06 00 00 0a 0a 1e 87 b8
49 .g`1@... .......I
0020 e0 05 fc 3b 69 af 25 21 23 6a 95 bf 4a 91 50 18 ...;i.%!
#j..J.P.
0030 40 29 c1 39 00 00 00 00 00 00 6a 7a 74 65 73 74 @).
9.... ..jztest
0040 2e 54 65 73 74 73 00 2e 00 00 00 07 5f 69 64
00 .Tests.. ...._id.
0050 4b e1 8c 3b af 6b d9 16 88 00 00 01 02 54 65 73
K..;.k.. .....Tes
0060 74 69 6e 67 00 0b 00 00 00 54 65 73 74 69 6e 67
ting.... .Testing
0070 41 42 43 00 00

On May 4, 9:36 am, Sam Corder <sa...@codeargyle.com> wrote:
> Is it possible to replicate the behavior with the shell too?  We should be
> sending the exact same bytes across the wire as the shell or any of the
> other drivers.  Using a packet capturer would verify that.
>

[John Zablocki]

It also works when I used the Python driver:

0000 00 d0 cf 05 96 91 00 22 19 20 b1 74 08 00 45
00 ......." . .t..E.

0010 00 79 6f d0 40 00 80 06 00 00 0a 0a 1e 87 b8
49 .yo.@... .......I
0020 e0 05 08 25 69 af 9f a3 3b 9a 67 bb aa bb 50 18 ...
%i... ;.g...P.
0030 3f 07 c1 4b 00 00 51 00 00 00 bb 41 00 00 00
00 ?..K..Q. ...A....
0040 00 00 d2 07 00 00 00 00 00 00 6a 7a 74 65 73
74 ........ ..jztest
0050 2e 54 65 73 74 73 00 30 00 00 00 07 5f 69 64 00 .Tests.
0 ...._id.
0060 4b e1 a9 bd af 6b d9 15 d8 00 00 00 02 54 65 73
K....k.. .....Tes
0070 74 69 6e 67 00 0d 00 00 00 54 65 73 74 69 6e 67
ting.... .Testing
0080 31 32 33 34 35 00 00
12345..

[Sam Corder]

I took a look and can definitely see some odd behavior going on.  What version of the driver are you using?  MongoHQ uses authenticated databases right?  Are the other databases you are connecting to also using auth?

[Steve Wagner]

Can you try the latest development version of the driver
http://github.com/lanwin/mongodb-csharp/tree/typedcollections. Maybe
that is already fixed there.

[John Zablocki]

That did the trick (and gave me a great excuse to finally try Git).

Thanks for the help and for writing a great driver!

On May 5, 4:12 pm, Steve Wagner <li...@lanwin.de> wrote:
> Can you try the latest development version of the driverhttp://github.com/lanwin/mongodb-csharp/tree/typedcollections. Maybe

> that is already fixed there.
>
> On 05.05.2010 21:55, Sam Corder wrote:
>
> > I took a look and can definitely see some odd behavior going on.  What
> > version of the driver are you using?  MongoHQ uses authenticated databases
> > right?  Are the other databases you are connecting to also using auth?
>

[Steve Wagner]

Good to know it works now!

But be careful. The code is currently short before beta, so you should
expect bugs.

[John Zablocki]

FYI - I realized today that I had grabbed the wrong branch (I was
using the latest Master branch) and it appears fixed there, but not in
the typed collections branch. I've been stepping through the code,
trying to compare all of the insert code, but nothing seems obviously
different.

Just like before, it works perfectly, everywhere except for MongoHQ.

--John

On May 6, 3:03 pm, Steve Wagner <li...@lanwin.de> wrote:
> Good to know it works now!
>
> But be careful. The code is currently short before beta, so you should
> expect bugs.
>
> On 06.05.2010 16:04, John Zablocki wrote:
>
> > That did the trick (and gave me a great excuse to finally try Git).
>
> > Thanks for the help and for writing a great driver!
>
> > On May 5, 4:12 pm, Steve Wagner <li...@lanwin.de> wrote:
> >> Can you try the latest development version of the driverhttp://github.com/lanwin/mongodb-csharp/tree/typedcollections. Maybe
> >> that is already fixed there.
>
> >> On 05.05.2010 21:55, Sam Corder wrote:
>
> >>> I took a look and can definitely see some odd behavior going on.  What

> >>> version of the driver are you using?  MongoHQuses authenticated databases

> >>>>>>> using the code below to test outMongoHQ. I had their support look

> >>>>>>>>>> Has anyone had experience hittingMongoHQ?  Connecting is no

[dcaravana]

Any news on the issue? I grabbed the official code and the branch but
it's not working at all, even repeating insertion.

Thanks
Diego

On May 6, 4:04 pm, John Zablocki <john.zablo...@gmail.com> wrote:
> That did the trick (and gave me a great excuse to finally try Git).
>
> Thanks for the help and for writing a great driver!
>
> On May 5, 4:12 pm, Steve Wagner <li...@lanwin.de> wrote:
>
>
>
> > Can you try the latest development version of the driverhttp://github.com/lanwin/mongodb-csharp/tree/typedcollections. Maybe
> > that is already fixed there.
>
> > On 05.05.2010 21:55, Sam Corder wrote:
>
> > > I took a look and can definitely see some odd behavior going on.  What

> > > version of the driver are you using?  MongoHQuses authenticated databases

> > >>>>> using the code below to test outMongoHQ. I had their support look

> > >>>>>>>> Has anyone had experience hittingMongoHQ?  Connecting is no

[Steve Wagner]

Did you tryed the version from here? http://github.com/lanwin/mongodb-csharp/tree/typedcollections

Is it possible that you can borrow me you account there to test it? It really hard to track down if nobody of us has a MongoHQ account.

[dcaravana]

Yes I tried that version, too.

You can create an account for free on MongoHQ (you haven't even to
insert credit card data); anyway, if you need my account, ask me it,
no problem.

Thanks Steve
Diego

On Jun 1, 9:22 pm, Steve Wagner <lan...@lanwin.de> wrote:
> Did you tryed the version from here? http://
> github.com/lanwin/mongodb-csharp/tree/typedcollections
>
> Is it possible that you can borrow me you account there to test it? It
> really hard to track down if nobody of us has a MongoHQ account.
>

[Steve Wagner]

Ok ive tryed it. And i am not able to reproduce it. Here is the code
that ive tested with the latest development version, and it works very
well here.

private static void Main(string[] args)
{
using(var mongo = new
Mongo("Server=flame.mongohq.com:27027;Username=xxxx;Password=xxxx"))
{
mongo.Connect();
var tests = mongo.GetDatabase("Tests");

var documents =
AddTagDocuments(tests.GetCollection("tags"), 100);

foreach(var document in documents)
Console.WriteLine(document);
}
}

private static IEnumerable<Document>
AddTagDocuments(IMongoCollection collection, int count)
{
var docs = new List<Document>();
for(var documentIndex = 1; documentIndex <= count;
documentIndex++)
{
var document = new Document {{"fooId", documentIndex}};
collection.Insert(document, true);
docs.Add(document);
}
return docs;
}

Can you crate an minimal example application which fails for you?

[dcaravana]

It's working for me now, with typedcollections branch (not for main
branch), but only if I use:

...
collection.Insert(document, true);
...

and not working with (as per examples):

...
collection.Insert(document);
...


i.e. without safemode parameter of with safemode = false, the insert
does not work. Can you explain this behaviour?

BTW It's very important to me to know when you'll release a new
version of the driver containing this fix for MongoHQ, else I'll have
to switch to another (stabler) driver.

Thanks anyway for your response and you efforts.

Diego

[Steve Wagner]

Ok now i can reproduce it. What Insert(document,true) dose is only to
send an getpreerror command to that database and throw an exception if
it returns an error.

[craiggwilson]

Diego, sorry you had some problems.

Let me explain the behaviour a little more. Safemode forces the
server to write the data to disk immediately. While this sounds great
on the serface, you are telling mongodb that this data is more
important than the rest of it's responsibilities. You'll find we use
safemode in our unit tests because there is a very fast turnover rate
for the data, but I'd advise you not to use it in production.

Since MongoHQ administers the servers and you don't have control over
how they are handling replication and/or sharding, then sometimes
inserting data without safemode will result in data no appearing as
though it was inserted. In reality, the data is there, it just hasn't
hit the disk yet.

There is not a problem with the driver and other drivers will have the
same issue. This is truly something to ask about at MongoHQ and/or at
the MongoDB user group.

[Steve Wagner]

No in this case it is really ab bug.

I figured it out. The Problem is that currently no auth is done one
query's. Only on commands. So the insert with safe mode false tries to
insert to an unauthenticated database and ignores the auth error
message. Whereas the safe mode true sends a command after the first
inserts and authenticate and so the following inserts are working.

[dcaravana]

Yes Steve, that should be the reason, also because data not inserted
never appear, even after hours after the insert(); moreover, watching
internal data of collection with the debugger, you can see an auth
error after the failed insert() with safemode false.

[dcaravana]

I confirm that queries are not authorized: in fact, I'm trying to
retrieve the data I inserted in the database but it's not working (and
the $err field contains the string "unauthorized for db [XXX] lock
type: -1", just like failed inserts).

Steve, have you got a wordaround or, better, a patch that I can use to
proceed with my tests? Or a new commit on github? Have you an idea of
how much time to wait for it?

Thanks again.
Diego

[Steve Wagner]

No in this case it is really ab bug.

I figured it out. The Problem is that currently no auth is done one
query's. Only on commands. So the insert with safe mode false tries to
insert to an unauthenticated database and ignores the auth error
message. Whereas the safe mode true sends a command after the first
inserts and authenticate and so the following inserts are working.

On 03.06.2010 13:44, craiggwilson wrote:

[Steve Wagner]

Ok ive pushed the fix for that bug to my repo. It works now.

[dcaravana]

It works both for inserts and for queries.

Thanks Steve