BufBuilder attemted to grow() to 134217728 bytes, past the 64MB limit.

[arnied...@gmail.com]

I'm trying to perform join on two MongoDB collections. Both collections are quite big with around 900K data each. I'm new to MongoDB and we're trying to shift our database from SQL to MongoDB to handle huge amount of data fast. Both these collections contain test data. I'm using SlamData to perform joins on these collections. Joins runs perfectly fine on small collections but when I run it on these two collections SlamData is giving me the following error.

This is my join query

    SELECT f.ids, s.two FROM "/joins/sample/firstColl" f , "/joins/sample/secondColl" s WHERE f.ids = s.two

and I'm getting the following error

Error in query: Command failed with error 13548: 'exception: BufBuilder attempted to grow() to 134217728 bytes, past the 64MB limit.' on server localhost:27017. The full response is { "errmsg" : "exception: BufBuilder attempted to grow() to 134217728 bytes, past the 64MB limit.", "code" : 13548, "ok" : 0.0 }

Being a Noob to MongoDB any help would be appreciated. Thanks.

[Moss Prescott]

SlamData doesn't (yet) optimize "cross-joins" like this, so you have to explicitly write it as an inner join. Something like this should work better:

SELECT f.ids, s.two FROM "/joins/sample/firstColl" f JOIN "/joins/sample/secondColl" s ON f.ids = s.two

- moss

[Greg Pfeil]

I thought we already had an issue for this, but I couldn’t find one. So, I created a new one: https://slamdata.atlassian.net/browse/SD-1063

[arnied...@gmail.com]

Awesome guys. Thanks.
After your suggestion the query does seem to be executing but then after a few seconds I get a

Error in query: Command failed with error 16722: 'exception: JavaScript execution failed -- v8 is out of memory' on server localhost:27017. The full response is { "errmsg" : "exception: JavaScript execution failed -- v8 is out of memory", "code" : 16722, "ok" : 0.0 }

Sorry but I am a complete newbie to this. Can anyone explain what is happening here?!

[John A. De Goes]

MongoDB uses V8 to execute Javascript. It’s running out of memory, which could be a sign you need more memory on the machine (or allocate more to V8 if that’s configurable?). The other thing to check is that your fields actually exist. If they don’t exist, then in the current version of SlamData, i believe that projecting them will produce a lot of null values that all get mapped to the same join key and cause memory usage to go up.

If all else fails, you can copy paste the query plan (click on the Eye icon for a successful query).

-- 
John A. De Goes

Follow me @jdegoes

--
You received this message because you are subscribed to the Google Groups "SlamData-User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to slamdata-use...@googlegroups.com.
Visit this group at http://groups.google.com/group/slamdata-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/slamdata-user/863311bc-a099-43d8-8fba-cf05b6b76971%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[arnied...@gmail.com]

Hi. Thanks for the quick reply.

I ran the same query on a small dataset of 100 of exactly same collection values and it gave me correct output.

but when I ran the same query on the 800K data it gave me V8 engine out of memory

Is there anyway I can fix this issue?!

[John A. De Goes]

I am sure there is a way to fix it, but because the issue is occurring inside MongoDB (and not inside SlamData), you may need their help to diagnose and resolve the issue. If you do not subscribe to MongoDB Enterprise Advanced, then I recommend the mongodb-user forum (someone can probably help you in there).

-- 
John A. De Goes

Follow me @jdegoes

--
You received this message because you are subscribed to the Google Groups "SlamData-User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to slamdata-use...@googlegroups.com.
Visit this group at http://groups.google.com/group/slamdata-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/slamdata-user/ff901c46-a7a6-43bc-8f40-3c4bb34db548%40googlegroups.com.

[da...@lacailles.com]

It may also depend on the MongoDB version you are running. See this SERVER JIRA ticket for more info, this may be the cause (and fix)

https://jira.mongodb.org/browse/SERVER-17224

[Damon LaCaille]

Just following up.  Did this seem to fix the issue?  Since it has a perceived impact on SlamData, I'd like to make sure this issue has a known solution.  Thanks!

-Damon

[arnied...@gmail.com]

Yes. It's now working! Thanks for all the help. :D