Mongo java Sharding Example

[Rahat Masood]

Hello Everyone,

I am facing a problem in running Mongo Java Sharding Example. What i have done so far is:

1. Download a zip folder from github.
2. Create a new project in netbeans and add all the required files from zipped folder to the new project.
3. Run following servers: configd, mongos, shard1 and shard2 ...through following commands

mongod --configsvr --dbpath C:/data/configdb --port 20000

mongos --port 30000 --configdb localhost:20000

mongod --dbpath C:\data\dbs\shard1 --port 10000

mongod --dbpath C:\data\dbs\shard2 --port 10001

Hoever, when i debug or run the shardingExample file....:

it prompts following output:

{ "errmsg" : "no such cmd: addshard" , "bad cmd" : { "addshard" : "localhost:10005"} , "ok" : 0.0}
{ "errmsg" : "no such cmd: addshard" , "bad cmd" : { "addshard" : "localhost:10006"} , "ok" : 0.0}
{ "errmsg" : "no such cmd: enablesharding" , "bad cmd" : { "enablesharding" : "test"} , "ok" : 0.0}
{ "errmsg" : "no such cmd: shardcollection" , "bad cmd" : { "shardcollection" : "test.logs" , "key" : { "date" : 1 , "hash" : 1}} , "ok" : 0.0}
DBPort.findOne failed.....

Can anyone please help in finding the exact problem????

 Your efforts will be highly appreciated.

Regards,
Rahat Masood.

[thomasr]

Hi Rahat,

Could you provide more information what commands exactly you tried to run that caused these error messages?

Was it one of the official tutorials on the MongoDB website? Perhaps you could give a link or some code, that would help find the cause for the errors.

Also, which version of MongoDB and the Java driver are you using?

Regards,

Thomas

[thomasr]

Hi Rahat, 

Just to add to my previous reply: Are you by any chance connecting to a mongod instead of the mongos?

You need to make sure that you connect to mongos (in your example on port 30000) and then issue the

addshard commands. mongod doesn't know about shards. Connecting to a mongod instead would result 

in exactly the error you experienced.

So if you use the javascript shell, you would have to call mongo with the --port 30000 parameter.

Best regards,

Thomas

On Monday, September 3, 2012 3:15:12 PM UTC+10, Rahat Masood wrote:

[Rahat Masood]

Dear Thomas,

Thanks for the reply. Yes, you pointed out the exact problem. I was giving mongod parameters instead of mongos. Now the problem is resolved, however, now i am facing another problem in using the ports......

 private static final int [] _shardPorts = { 10049, 10057 };

When I run 2 shards with the ports 10049 and 10057 and then tun mongo java sharding example in netbeans, it gives an error:

{ "ok" : 0.0 , "errmsg" : "host already used"}
{ "ok" : 0.0 , "errmsg" : "host already used"}

I know this error is because these ports are already in use of sharding. To remove this error, what i am doing is that whenever i compile the code I give new port number.s

I want to know, is there any way to reconfigure the same ports because i am running this example for my testing and giving new ports each time is not feasible for me.

Regards,
Rahat Masood.

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb

[thomasr]

Hi Rahat,

It seems like you're trying to add the same shards to the mongos server that have already been added before. You only need to do this step once during the lifetime of the server, not every time you run the example. Try commenting out the lines that add the shards to the server and see if the code runs without errors.

Best regards,

Thomas

[Rahat Masood]

Yes, i found the problem and sorted it out. Can u please tell, how can i query the data stored on shards??

I think i need to connect to Mongos. I connected to it and then using simple view query I printed out the statement. However, it is not working because my dbcursor is empty.

Here is my code:

final Mongo mongo2 = new Mongo(new DBAddress("127.0.0.1", 30000,
"ExampleShardingData"));
       
 db = mongo2.getDB("ExampleShardingData");
 coll = db.getCollection("testShard");
 
  BasicDBObject view1 = new BasicDBObject();
       
        view1.put("x",20110114);
        DBCursor cur = coll.find(view1);
       
        Double id;
        Binary b;
         if (cur.hasNext()) {
            DBObject o = cur.next();
           
             id = (Double) (o.get("x"));
            
                 b = (Binary) (o.get("hash"));
            
             System.out.println(id);
             System.out.println(b);
         }


DBCursor variable is not returning anything....Can you please find the problem??

Do i need to query through CommnadResult because we are connecting now with Mongos....

Regards,
Rahat Masood.

[thomasr]

Hi Rahat,

I've looked through your code example, but unfortunately can't see any obvious reasons why you wouldn't be able to retrieve any data.

In order to diagnose your problem, could you try the following:

     1. Could you connect to the mongos through the shell directly (mongo --port 30000) and try to access the document there? Are you able to retrieve it?

     2. In your Java code, can you connect to the shard directly that contains the document, instead of the mongos? Are you able to retrieve it?

the answers to these questions will be very helpful in finding out why your code doesn't work.

Lastly, I'd like to refer you to the Java tutorials on the MongoDB website, in case you haven't seen them yet: http://www.mongodb.org/display/DOCS/Java+Tutorial

They provide you with some more examples on how to connect to MongoDB through the Java driver.

Best regards,

Thomas

[Rahat Masood]

Dear Thomas,

the answers to your queries are:

Could you connect to the mongos through the shell directly (mongo --port 30000) and try to access the document there? Are you able to retrieve it?

When I connect directly to mongos shell and query the data, i get nothing. Below is the shell commands:

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

D:\Users\tecnex>cd D:\mongodb\bin

D:\mongodb\bin>mongo localhost:30000/admin
MongoDB shell version: 2.0.6
connecting to: localhost:30000/admin
mongos> db.testShard.find();
mongos> show collections
mongos>

In your Java code, can you connect to the shard directly that contains the document, instead of the mongos? Are you able to retrieve it?

Sorry, i do not understand your question. Therefore i am putting my whole code here:

//--------------------------------------------------------------------------------------------
package main.com.deftlabs.examples.mongo;

import com.mongodb.*;
import java.security.MessageDigest;
import java.util.Random;

// JUnit
import org.junit.Test;
import org.junit.Before;
import static org.junit.Assert.*;

// Java
import java.util.List;
import java.util.Random;
import java.util.ArrayList;
import java.security.MessageDigest;
import org.bson.types.Binary;

/**
 *
 * @author tecnex
 */

public class ShardingTest {

   
    static DBCollection coll,coll2,coll3;
    static DB db;
   
    private static void insert1() throws Exception {

   
 final Mongo mongo2 = new Mongo(new DBAddress("127.0.0.1", 30000,
"ExampleShardingData"));
       
 db = mongo2.getDB("ExampleShardingD

ata");
 coll = db.getCollection("testShard");
 
  BasicDBObject view1 = new BasicDBObject();
       
        view1.put("x",20110114);
        DBCursor cur = coll.find(view1);
       
        Double id;
        Binary b;
         if (cur.hasNext()) {
            DBObject o = cur.next();
           
             id = (Double) (o.get("x"));
            
                 b = (Binary) (o.get("hash"));
            
             System.out.println(id);
             System.out.println(b);

    }
}
  
   

     public ShardingTest() {

      
    }


    private static final int [] _shardPorts = { 10049, 10057 };

    @Before
    public static void setupCluster() throws Exception {

        // Connect to mongos
        final Mongo mongo = new Mongo(new DBAddress("127.0.0.1", 30000, "admin"));

        // Add the shards
       /* for (final int shardPort : _shardPorts) {
            final CommandResult result
            = mongo.getDB("admin").command(new BasicDBObject("addshard", ("localhost:" + shardPort)));
            System.out.println(result);
        }

        // Sleep for a bit to wait for all the nodes to be intialized.
        Thread.sleep(5000);

        // Enable sharding on a collection.
        CommandResult result
        = mongo.getDB("admin").command(new BasicDBObject("enablesharding", "ExampleShardingData"));
        System.out.println(result);*/

       final BasicDBObject shardKey = new BasicDBObject("x", 1);
      shardKey.put("hash", 1);

        final BasicDBObject cmd = new BasicDBObject("shardcollection", "ExampleShardingData.testShard");

        cmd.put("key", shardKey);

    CommandResult result4 = mongo.getDB("admin").command(cmd);

       System.out.println(result4);
       
       final BasicDBObject re = new BasicDBObject("split", "ExampleShardingData.testShard");
      
       final BasicDBObject mid = new BasicDBObject("x", 100);
       re.put("middle", mid);
       CommandResult result3 = mongo.getDB("admin").command(re);
        System.out.println(result3);
       
        final BasicDBObject mov = new BasicDBObject("moveChunk", "ExampleShardingData.testShard");
      
       final BasicDBObject mo2 = new BasicDBObject("x", 10);
       mov.put("find", mo2);
      
   
       mov.put("to", "shard0001");
      
      
        CommandResult result2 = mongo.getDB("admin").command(mov);
        System.out.println(result2);
    }

    @Test
    public static void testShards() throws Exception {

        final Mongo mongo = new Mongo(new DBAddress("127.0.0.1", 27017, "ExampleShardingData"));

        final DBCollection shardCollection = mongo.getDB("ExampleShardingData").getCollection("testShard");

        final Random random = new Random(System.currentTimeMillis());

        // Write some data
        for (int idx=0; idx < 1000; idx++) {

            final BasicDBObject entry
            = new BasicDBObject("x", ("201101" + String.format("%02d", random.nextInt(30))));

            entry.put("hash", md5(("this is a value to hash-" + idx)));

            shardCollection.insert(entry);
        }
    }

    private static byte [] md5(final String pValue) throws Exception
    { return MessageDigest.getInstance("MD5").digest(pValue.getBytes("UTF-8")); }
   
     public static void main(String[] args) {
        // TODO code application logic here
        
      
       try {
            testShards();
        } catch (Exception ex) {
            System.out.println(ex.getMessage());
        }
       
          try {
            setupCluster();
        } catch (Exception ex) {
            System.out.println(ex.getMessage());
           
        }
       
  try {
            insert1();
        } catch (Exception ex) {
            System.out.println(ex.getMessage());
           
        }
       
    }
}

//-------------------------------------------------------------------------------------

yes i have Java Tutorials and also implemented them. They ware running accurately. What i want is to make shards through java driver and then query the data in those shards.

Regards,
Rahat Masood.

[Thomas Rueckstiess]

Hi Rahat,

were you able to solve your problem? I think what could have happened here is that you inserted documents directly to the mongod rather than through mongos. That would explain why you can't get any data back from mongos. I suggest you start your example code from scratch, and make sure that you always connect to mongos (port 30000), even for inserting the documents. Then you should be able to revtrieve them, both in the shell and through the Java driver.

Please let us know if you need any further assistance with this.

Regards,

Thomas