exception while running MongoDB client

[Yuri Gurevich]

On Wed, Oct 1, 2014 at 5:33 AM, Yuri Gurevich wrote:

Hi Sumita,

I changed the parameters that you suggested above and this time it didn't fail and no exception was thrown.. 

Here are the command lines that I used:

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -load -db  mongoDB.MongoDbClient -P ~/populateDB_new -p mongodb.url=127.0.0.1:27017 -p insertimage=true -p imagesize=2 -p threadcount=2 -p mongodb.database=local -p mongodb.writeConcern="strict" -p maxexecutiontime=60

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -t -db  mongoDB.MongoDbClient -P workloads/RealisticActions -p mongodb.url=127.0.0.1:27017 -p mongodb.database="local" -p maxexecutiontime=30 -p usercount=100 -p initapproach=deterministic -p resourcecountperuser=10 -p friendcountperuser=10 -p confper=1 -p numloadthreads=2 -p mongodb.writeConcern="strict"

Thank you so much for your assistance! 

Thanks,

-Yuri 

On Tue, Sep 30, 2014 at 12:38 PM, Sumita Barahmand wrote:

Hi Yuri,

I have not seen this exception before but with a search online I see that it happens because the MongoDB java driver loses its connection to the server. So I think you need to debug and check to see why this is happening. Can you please post the question on the Google group. Some of the other users may have encountered this exception and may be able to help you.

1. Have you checked the server to see if the schema is created and the correct number of users have been inserted once the loading is complete?

2. Can you also change the mongodb.writeConcern="none"  to mongodb.writeConcern="strict"  both for load and the benchmarking phase so we can see all the server side exceptions, just in case we are missing something.

3. Finally, I noticed that you had a typo/mistake in one of your parameters:  -p initapproach=deterministic -p resourcecountperuser=10 -p friendcountperuser=10 -p confper=0.5 -p numloadthreads=2

which should be confperc=1. The confperc should be the same as what is specified in the load phase which is 1 in you case. Also the confperc can either be 0 or 1.

Please post you questions to the Google group: https://groups.google.com/forum/?hl=en#!forum/bg-socialbenchmark

Thanks,
Sumita

On Mon, Sep 29, 2014 at 7:49 PM, Yuri Gurevich wrote:

Hi Sumita,

Thanks for your prompt response, I really appreciate that.

I have tried to execute the whole sequence again and here are the steps I perform:

1) schema creation

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -schema -db mongoDB.MongoDbClient -p mongodb.url=127.0.0.1:27017 -p mongodb.database="local" -p workload=edu.usc.bg.workloads.CoreWorkload

2) load data

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -load -db  mongoDB.MongoDbClient -P ~/populateDB_new -p mongodb.url=127.0.0.1:27017 -p insertimage=true -p imagesize=2 -p threadcount=2 -p mongodb.database=local -p mongodb.writeConcern="none" -p maxexecutiontime=60

My properties file is as following:

> cat ~/populateDB_new

usercount=100

useroffset=0

resourcecountperuser=100

friendcountperuser=10

confperc=1

userworkload=edu.usc.bg.workloads.UserWorkload

friendshipworkload=edu.usc.bg.workloads.FriendshipWorkload

resourceworkload=edu.usc.bg.workloads.ResourceWorkload

requestdistribution=dzipfian

requestmean=0.27

3) benchmark phase (this time I use 100 users)

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -t -db  mongoDB.MongoDbClient -P workloads/RealisticActions -p mongodb.url=127.0.0.1:27017 -p mongodb.database="local" -p maxexecutiontime=30 -p usercount=100 -p initapproach=deterministic

BGClient is in onetime mode.

*****max execution time specified : 30

BG Client: ThreadCount =1

Loading workload...

0

Create fragments in workload init phase

Creating the initial zipfian probabilities...

Sum of original probs = 0.9999999999999996

Initial Zipfian probabilities are created.

Done creating clusters.

ClusterId0, 100

ZipfianMean=0.27 Usercount=100 numMaxClusters=1; 20.0% of members have, 0.5379732061994167, initial probability.

ZipfianMean=0.27 Usercount=100 numMaxClusters=1; 20.0% of members have, 0.537973206199417 clustering(new)probability.

Time to create fragments : 8 msec

parameters not specified for initapproach=deterministic. Be sure to include resourcecountperuser, friendcountperuser, confperc and numloadthreads )

After init: Mon Sep 29 22:43:46 EDT 2014

EXECUTIONDONE

Connected

Starting benchmark.

Maximum execution time specified as: 30 secs

Sep 29, 2014 10:43:46 PM com.mongodb.DBPortPool gotError

WARNING: emptying DBPortPool to 127.0.0.1:27017/local b/c of error

java.lang.NullPointerException

at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:154)

at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:138)

at com.mongodb.DBApiLayer$MyCollection.update(DBApiLayer.java:348)

at com.mongodb.DBCollection.update(DBCollection.java:165)

at mongoDB.MongoDbClient.inviteFriend(Unknown Source)

at edu.usc.bg.base.DBWrapper.inviteFriend(DBWrapper.java:158)

at edu.usc.bg.workloads.CoreWorkload.doActioninviteFriend(CoreWorkload.java:2423)

at edu.usc.bg.workloads.CoreWorkload.doTransaction(CoreWorkload.java:1073)

at edu.usc.bg.base.ClientThread.run(ClientThread.java:201)

java.lang.NullPointerException

There is an exception in inviteFriend.

I also tried adding the parameters that you suggested, but I still got the same exception:

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -t -db  mongoDB.MongoDbClient -P workloads/RealisticActions -p mongodb.url=127.0.0.1:27017 -p mongodb.database="local" -p maxexecutiontime=30 -p usercount=100 -p initapproach=deterministic -p resourcecountperuser=10 -p friendcountperuser=10 -p confper=0.5 -p numloadthreads=2

BGClient is in onetime mode.

*****max execution time specified : 30

BG Client: ThreadCount =1

Loading workload...

0

Create fragments in workload init phase

Creating the initial zipfian probabilities...

Sum of original probs = 0.9999999999999996

Initial Zipfian probabilities are created.

Done creating clusters.

ClusterId0, 100

ZipfianMean=0.27 Usercount=100 numMaxClusters=1; 20.0% of members have, 0.5379732061994167, initial probability.

ZipfianMean=0.27 Usercount=100 numMaxClusters=1; 20.0% of members have, 0.537973206199417 clustering(new)probability.

Time to create fragments : 10 msec

parameters not specified for initapproach=deterministic. Be sure to include resourcecountperuser, friendcountperuser, confperc and numloadthreads )

After init: Mon Sep 29 22:45:41 EDT 2014

Connected

Starting benchmark.

EXECUTIONDONE

Maximum execution time specified as: 30 secs

Sep 29, 2014 10:45:41 PM com.mongodb.DBPortPool gotError

WARNING: emptying DBPortPool to 127.0.0.1:27017/local b/c of error

java.lang.NullPointerException

at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:154)

at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:138)

at com.mongodb.DBApiLayer$MyCollection.update(DBApiLayer.java:348)

at com.mongodb.DBCollection.update(DBCollection.java:165)

at mongoDB.MongoDbClient.inviteFriend(Unknown Source)

at edu.usc.bg.base.DBWrapper.inviteFriend(DBWrapper.java:158)

at edu.usc.bg.workloads.CoreWorkload.doActioninviteFriend(CoreWorkload.java:2423)

at edu.usc.bg.workloads.CoreWorkload.doTransaction(CoreWorkload.java:1073)

at edu.usc.bg.base.ClientThread.run(ClientThread.java:201)

java.lang.NullPointerException

There is an exception in inviteFriend.

Besides that I double checked: I'm not getting this exception when I run ReadOnly workload

Thanks,

-Yuri

On Mon, Sep 29, 2014 at 5:55 PM, Sumita Barahmand wrote:

Hi,

Could you please give me the details of your social graph? I understand that you have inserted 10 users, But how many friends have been inserted for each user and how many threads were used to create the social graph (what was the thread count used in the load phase). Also what is the threadcount you are using for the benchmarking (-t) phase?

Also do you see the same exception with a read only workload? 

Also when using the "-p initapproach=deterministic" flag you need to specify the followin parameters as well: resourcecountperuser, friendcountperuser, confper, numloadthreads and useroffset as mentioned in 
http://bgbenchmark.org/BG/runtimeparams.html

Thanks,
Sumita

On Mon, Sep 29, 2014 at 5:31 AM, Yuri Gurevich wrote:

Hi Sumita,

I'm getting an exception while running the test phase:

> java -cp build/bg.jar:db/MongoDB/lib/* edu.usc.bg.BGMainClass onetime -t -db  mongoDB.MongoDbClient -P workloads/RealisticActions -p mongodb.url=127.0.0.1:27017 -p mongodb.database="local" -p maxexecutiontime=30 -p usercount=10 -p initapproach=deterministic

Here is the output:

After init: Mon Sep 29 08:26:51 EDT 2014

Connected

Starting benchmark.

Maximum execution time specified as: 30 secs

Sep 29, 2014 8:26:51 AM com.mongodb.DBPortPool gotError

WARNING: emptying DBPortPool to 127.0.0.1:27017/local b/c of error

java.lang.NullPointerException

at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:154)

at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:138)

at com.mongodb.DBApiLayer$MyCollection.update(DBApiLayer.java:348)

at com.mongodb.DBCollection.update(DBCollection.java:165)

at mongoDB.MongoDbClient.inviteFriend(Unknown Source)

at edu.usc.bg.base.DBWrapper.inviteFriend(DBWrapper.java:158)

at edu.usc.bg.workloads.CoreWorkload.doActioninviteFriend(CoreWorkload.java:2423)

at edu.usc.bg.workloads.CoreWorkload.doTransaction(CoreWorkload.java:1073)

at edu.usc.bg.base.ClientThread.run(ClientThread.java:201)

java.lang.NullPointerException

There is an exception in inviteFriend.

Do I miss any parameter which is required? or what could the reason for that?

Thanks,

-Yuri Gurevich