Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Redis DB
Home
+ new post
About this group
Join this group
sharedobjects
Options
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
   4 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Aman Gupta  
View profile  
 More options Jul 8 2009, 4:22 am
From: Aman Gupta <themastermi...@gmail.com>
Date: Wed, 8 Jul 2009 01:22:33 -0700
Local: Wed, Jul 8 2009 4:22 am
Subject: sharedobjects
Print | Individual message | Show original | Report this message | Find messages by this author
Is there a way to tell how many objects are being shared? I'm working
a large dataset and not seeing very much memory savings with
sharedobjects, even though I know I have a lot of sets with the same
strings.

sharedobjects no:

08 Jul 08:17:42 . 0 clients connected (0 slaves), 1815084578 bytes in use
08 Jul 08:17:47 . DB 0: 60503 keys (0 volatile) in 65536 slots HT.
08 Jul 08:17:47 . DB 1: 2634133 keys (0 volatile) in 4194304 slots HT.
08 Jul 08:17:47 . DB 3: 97 keys (0 volatile) in 128 slots HT.
08 Jul 08:17:47 . DB 4: 6 keys (0 volatile) in 8 slots HT.
08 Jul 08:17:47 . DB 5: 97 keys (0 volatile) in 128 slots HT.
08 Jul 08:17:47 . DB 15: 21 keys (0 volatile) in 32 slots HT.

sharedobjects yes:

08 Jul 08:15:36 . 0 clients connected (0 slaves), 1804292361 bytes in use
08 Jul 08:15:41 . DB 0: 60503 keys (0 volatile) in 65536 slots HT.
08 Jul 08:15:41 . DB 1: 2634133 keys (0 volatile) in 4194304 slots HT.
08 Jul 08:15:41 . DB 3: 97 keys (0 volatile) in 128 slots HT.
08 Jul 08:15:41 . DB 4: 6 keys (0 volatile) in 8 slots HT.
08 Jul 08:15:41 . DB 5: 97 keys (0 volatile) in 128 slots HT.
08 Jul 08:15:41 . DB 15: 21 keys (0 volatile) in 32 slots HT.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Salvatore Sanfilippo  
View profile  
 More options Jul 8 2009, 6:15 am
From: Salvatore Sanfilippo <anti...@gmail.com>
Date: Wed, 8 Jul 2009 12:15:22 +0200
Local: Wed, Jul 8 2009 6:15 am
Subject: Re: sharedobjects
Print | Individual message | Show original | Report this message | Find messages by this author

On Wed, Jul 8, 2009 at 10:22 AM, Aman Gupta<themastermi...@gmail.com> wrote:

> Is there a way to tell how many objects are being shared? I'm working
> a large dataset and not seeing very much memory savings with
> sharedobjects, even though I know I have a lot of sets with the same
> strings.

Hello Aman,

indeed shareobjects is saving too little RAM in your use case, you can
make a much larger shared objects pool using redis.conf:

shareobjectspoolsize 1024

this is the default, try with a very large value

shareobjectspoolsize 100000

and se what happens... of course the pool itself will use more memory
but should be able to reach a better saving, so the best thing to do
probably is to experiment with different values.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Aman Gupta  
View profile  
 More options Jul 8 2009, 10:39 pm
From: Aman Gupta <themastermi...@gmail.com>
Date: Wed, 8 Jul 2009 19:39:32 -0700
Local: Wed, Jul 8 2009 10:39 pm
Subject: Re: sharedobjects
Print | Individual message | Show original | Report this message | Find messages by this author

awesome, thanks! I ran into a bug where it would not use the value
from the config, patch:

diff --git a/redis.c b/redis.c
index f4fa061..d44e900 100644
--- a/redis.c
+++ b/redis.c
@@ -751,7 +751,7 @@ static int serverCron(struct aeEventLoop
*eventLoop, long long id, void *clientD

     /* Show information about connected clients */
     if (!(loops % 5)) {
-        redisLog(REDIS_DEBUG,"%d clients connected (%d slaves), %zu
bytes in use",
+        redisLog(REDIS_DEBUG,"%d clients connected (%d slaves), %zu
bytes in use, %d shared objects",
             listLength(server.clients)-listLength(server.slaves),
             listLength(server.slaves),
             server.usedmemory,
@@ -900,6 +900,7 @@ static void initServerConfig() {
     server.dbfilename = "dump.rdb";
     server.requirepass = NULL;
     server.shareobjects = 0;
+    server.sharingpoolsize = 1024;
     server.maxclients = 0;
     server.maxmemory = 0;
     ResetServerSaveParams();
@@ -930,7 +931,6 @@ static void initServer() {
     server.el = aeCreateEventLoop();
     server.db = zmalloc(sizeof(redisDb)*server.dbnum);
     server.sharingpool = dictCreate(&setDictType,NULL);
-    server.sharingpoolsize = 1024;
     if (!server.db || !server.clients || !server.slaves ||
!server.monitors || !server.el || !server.objfreelist)
         oom("server initialization"); /* Fatal OOM */
     server.fd = anetTcpServer(server.neterr, server.port, server.bindaddr);

with this patch, I am seeing some savings:

09 Jul 02:38:46 . 0 clients connected (0 slaves), 1539137609 bytes in
use, 5992320 shared objects

  Aman


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Salvatore Sanfilippo  
View profile  
 More options Jul 17 2009, 5:57 am
From: Salvatore Sanfilippo <anti...@gmail.com>
Date: Fri, 17 Jul 2009 11:57:27 +0200
Local: Fri, Jul 17 2009 5:57 am
Subject: Re: sharedobjects
Print | Individual message | Show original | Report this message | Find messages by this author
Thanks Aman,
just merged.

On Thu, Jul 9, 2009 at 4:39 AM, Aman Gupta<themastermi...@gmail.com> wrote:
> with this patch, I am seeing some savings:

> 09 Jul 02:38:46 . 0 clients connected (0 slaves), 1539137609 bytes in
> use, 5992320 shared objects

--
Salvatore 'antirez' Sanfilippo
http://invece.org

"Once you have something that grows faster than education grows,
you’re always going to get a pop culture.", Alan Kay


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2013 Google