memcache architecture
151 views
Skip to first unread message
hawkett
Jun 30, 2009, 10:32:57 AM6/30/09
to Google App Engine
Hi,
I'm wondering what the memcache architecture is - is it
a) single instance with a massive pool of memory for everyone, or
b) many instances that are each shared by a few apps, or
c) an instance per app, or
d) something else?
I'm guessing (a) if there is an intelligent LRU policy that provides
separate LRU for each app, preventing a few apps evicting everyone
else's data, or (b) if the LRU policy is not intelligent. (c) would
lead to a very poor memory utilisation.
Apart from being interested, I am wondering about memcache
prioritisation - i.e. where the blunt LRU scheme is not suitable. I
might have a small amount of data that I really don't want evicted
from memcache (even if it is not used very often), and a large stream
of less important data for which the blunt LRU policy is fine. In the
current GAE implementation my important data can be evicted by my less
important data.
It would be great if there was some way to prioritise memcache data -
in a stand-alone system, this would be achieved using multiple
instances, one for each priority level.
Assuming we have option (a) with intelligent LRU on a per app basis,
then it shouldn't be too hard to provide multiple memcache instances
that can be used for prioritisation of cached data. It should be easy
enough to make this backward compatible by providing an optional
parameter to specify the instance, defaulting if it is not provided.
I can see that this would impact memory utilisation, but it would be
possible to make the second, non-default instance much smaller than
the default one - say 5% - forcing developers to think carefully about
using the smaller cache. Even if no one uses the secondary instance,
memory utilisation can still top 95%.
Interested to understand the issues, as this is forcing me not to
cache my 'big data' for fear of constantly evicting my 'little data'.
In my situation, my 'little data' is very important for general
response times, which are good to keep up even if the data is not used
particularly often. Essentially what I need is a cache with low
volatility, and a cache whose volatility isn't so important. Cheers,
Colin
I'm wondering what the memcache architecture is - is it
a) single instance with a massive pool of memory for everyone, or
b) many instances that are each shared by a few apps, or
c) an instance per app, or
d) something else?
I'm guessing (a) if there is an intelligent LRU policy that provides
separate LRU for each app, preventing a few apps evicting everyone
else's data, or (b) if the LRU policy is not intelligent. (c) would
lead to a very poor memory utilisation.
Apart from being interested, I am wondering about memcache
prioritisation - i.e. where the blunt LRU scheme is not suitable. I
might have a small amount of data that I really don't want evicted
from memcache (even if it is not used very often), and a large stream
of less important data for which the blunt LRU policy is fine. In the
current GAE implementation my important data can be evicted by my less
important data.
It would be great if there was some way to prioritise memcache data -
in a stand-alone system, this would be achieved using multiple
instances, one for each priority level.
Assuming we have option (a) with intelligent LRU on a per app basis,
then it shouldn't be too hard to provide multiple memcache instances
that can be used for prioritisation of cached data. It should be easy
enough to make this backward compatible by providing an optional
parameter to specify the instance, defaulting if it is not provided.
I can see that this would impact memory utilisation, but it would be
possible to make the second, non-default instance much smaller than
the default one - say 5% - forcing developers to think carefully about
using the smaller cache. Even if no one uses the secondary instance,
memory utilisation can still top 95%.
Interested to understand the issues, as this is forcing me not to
cache my 'big data' for fear of constantly evicting my 'little data'.
In my situation, my 'little data' is very important for general
response times, which are good to keep up even if the data is not used
particularly often. Essentially what I need is a cache with low
volatility, and a cache whose volatility isn't so important. Cheers,
Colin
bFlood
Jun 30, 2009, 11:00:50 AM6/30/09
to Google App Engine
i agree colin, there is a lot of big data that I just don't cache in
fear of evicting the smaller, more important chunks.
cheers
brian
fear of evicting the smaller, more important chunks.
cheers
brian
hawkett
Jun 30, 2009, 11:22:20 AM6/30/09
to Google App Engine
Thanks for the response Brian - it should be interesting to google
that I'm not the only one choosing not to cache data for this reason,
because it means that people are using more CPU and datastore
resources because of this issue. Given that the availability of CPU
(especially on a per request basis) seems to be the biggest bottleneck
in GAE at the moment, there may be a noticeable drop in the use of
this resource if developers are able to choose to use memcache more
effectively. Certainly I haven't heard any complaints that memcache
allocation is too small, so a 5% rejig as suggested would probably go
unnoticed. Sounds like a win-win situation. Are there any other
issues I'm not considering here? Thanks,
Colin
that I'm not the only one choosing not to cache data for this reason,
because it means that people are using more CPU and datastore
resources because of this issue. Given that the availability of CPU
(especially on a per request basis) seems to be the biggest bottleneck
in GAE at the moment, there may be a noticeable drop in the use of
this resource if developers are able to choose to use memcache more
effectively. Certainly I haven't heard any complaints that memcache
allocation is too small, so a 5% rejig as suggested would probably go
unnoticed. Sounds like a win-win situation. Are there any other
issues I'm not considering here? Thanks,
Colin
bFlood
Jun 30, 2009, 11:41:14 AM6/30/09
to Google App Engine
agreed, sounds like it would be better for GAE as a whole. something
like a simple priority value (or just low/high) when setting the cache
would be sufficient and then GAE could figure out where to place it
memcache
cheers
brian
like a simple priority value (or just low/high) when setting the cache
would be sufficient and then GAE could figure out where to place it
memcache
cheers
brian
Peter Recore
Jun 30, 2009, 12:31:45 PM6/30/09
to Google App Engine
This sounds like a useful feature. If you create an issue in the
issues list I'll star it :)
issues list I'll star it :)
hawkett
Jun 30, 2009, 12:55:12 PM6/30/09
to Google App Engine
I was hoping google could comment on the architecture, so I can
structure a feature request that is actually possible :)
structure a feature request that is actually possible :)
hawkett
Jun 30, 2009, 3:02:57 PM6/30/09
to Google App Engine
Mark Wolgemuth
Jul 2, 2009, 8:55:02 AM7/2/09
to Google App Engine
It would be interesting to offer different global memcache queues with
different policies.
In your case, it seems to me if you have things you want to cache
indefinitely, why don't you just load them in the current interpreter
into a global dict on first hit? Effectively run your own local cache.
Not sure how this plays out in GAE arch though, it would only help
subsequent requests that land on the same interpreter.
different policies.
In your case, it seems to me if you have things you want to cache
indefinitely, why don't you just load them in the current interpreter
into a global dict on first hit? Effectively run your own local cache.
Not sure how this plays out in GAE arch though, it would only help
subsequent requests that land on the same interpreter.
hawkett
Jul 2, 2009, 9:59:19 AM7/2/09
to Google App Engine
This might work for apps with very high load - but looking at other
posts on this group, it looks like a server instance only needs to be
idle for a couple of seconds to be brought down. In an application
with not so high traffic, essentially we would be swapping eviction
due the the pressure of unimportant data with eviction due to being
idle. In a system with low traffic volume, it may be even more
important to keep the important memcache data, since you will also be
contending with the cost of starting server instances. From my
perspective a key advantage of memcache is that its life-cycle
transcends the stop/start of server instances.
Even in a high load application, the global dict alternative is not
ideal, as every new server instance started has to populate the
cache. This essentially means the main reason for the high priority
memcache data is lost - i.e. reliably fast performance on all
requests. I'm not sure how GAE works re. stop/start of servers, but I
wouldn't be surprised if there was a fair bit of it on all apps,
regardless of traffic.
posts on this group, it looks like a server instance only needs to be
idle for a couple of seconds to be brought down. In an application
with not so high traffic, essentially we would be swapping eviction
due the the pressure of unimportant data with eviction due to being
idle. In a system with low traffic volume, it may be even more
important to keep the important memcache data, since you will also be
contending with the cost of starting server instances. From my
perspective a key advantage of memcache is that its life-cycle
transcends the stop/start of server instances.
Even in a high load application, the global dict alternative is not
ideal, as every new server instance started has to populate the
cache. This essentially means the main reason for the high priority
memcache data is lost - i.e. reliably fast performance on all
requests. I'm not sure how GAE works re. stop/start of servers, but I
wouldn't be surprised if there was a fair bit of it on all apps,
regardless of traffic.
jonathan
Jul 7, 2009, 7:40:41 PM7/7/09
to Google App Engine
I'd also be interested to know if there is some general predictions we
can make about how much data we can actually store in memcache before
our LRU data is evicted. I am using memcache as a processing queue for
non-critical writes and I run a cron-job every one minute because I am
not sure how often I need to run it before the writes would be
evicted.
jonathan
> I'm wondering what thememcachearchitecture is - is it
can make about how much data we can actually store in memcache before
our LRU data is evicted. I am using memcache as a processing queue for
non-critical writes and I run a cron-job every one minute because I am
not sure how often I need to run it before the writes would be
evicted.
jonathan
> I'm wondering what thememcachearchitecture is - is it
>
> a) single instance with a massive pool of memory for everyone, or
> b) many instances that are each shared by a few apps, or
> c) an instance per app, or
> d) something else?
>
> I'm guessing (a) if there is an intelligent LRU policy that provides
> separate LRU for each app, preventing a few apps evicting everyone
> else's data, or (b) if the LRU policy is not intelligent. (c) would
> lead to a very poor memory utilisation.
>
> Apart from being interested, I am wondering aboutmemcache
> prioritisation - i.e. where the blunt LRU scheme is not suitable. I
> might have a small amount of data that I really don't want evicted
> frommemcache(even if it is not used very often), and a large stream
> a) single instance with a massive pool of memory for everyone, or
> b) many instances that are each shared by a few apps, or
> c) an instance per app, or
> d) something else?
>
> I'm guessing (a) if there is an intelligent LRU policy that provides
> separate LRU for each app, preventing a few apps evicting everyone
> else's data, or (b) if the LRU policy is not intelligent. (c) would
> lead to a very poor memory utilisation.
>
> Apart from being interested, I am wondering aboutmemcache
> prioritisation - i.e. where the blunt LRU scheme is not suitable. I
> might have a small amount of data that I really don't want evicted
> of less important data for which the blunt LRU policy is fine. In the
> current GAE implementation my important data can be evicted by my less
> important data.
>
> It would be great if there was some way to prioritisememcachedata -
> current GAE implementation my important data can be evicted by my less
> important data.
>
Message has been deleted
Message has been deleted
Alkis Evlogimenos ('Αλκης Ευλογημένος)
Jul 8, 2009, 2:57:23 AM7/8/09
to google-a...@googlegroups.com
Maybe the fetch service is blocked in China?
--
Alkis
On Wed, Jul 8, 2009 at 4:14 AM, chenbaiping <chenb...@revenco.com> wrote:
I have a very simple html page http://cbc007.cn/test.html .
When I fetch it using http://shell.appspot.com/, an error occurs, sometimes.
When I fetch it in my app, it has the same error.
>>> r=urlfetch.fetch('http://cbc007.cn/test.html')
Traceback (most recent call last):
File "/base/data/home/apps/shell/1.334417654335420704/shell.py", line 278,
in get
exec compiled in statement_module.__dict__
File "<string>", line 1, in <module>
File "/base/python_lib/versions/1/google/appengine/api/urlfetch.py", line
241, in fetch
return rpc.get_result()
File
"/base/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py",
line 442, in get_result
return self.__get_result_hook(self)
File "/base/python_lib/versions/1/google/appengine/api/urlfetch.py", line
331, in _get_fetch_result
raise DownloadError(str(err))
DownloadError: ApplicationError: 5
Sometimes, it throws "DownloadError: ApplicationError: 2" error. Very
curious.
But it occurs on GAE production server only. On local sdk, it works fine.
Is it a bug?
--
Alkis
Message has been deleted
Alkis Evlogimenos ('Αλκης Ευλογημένος)
Jul 8, 2009, 7:54:34 AM7/8/09
to google-a...@googlegroups.com
The question is can Google access this page outside of China. The fact that you can is irrelevant :-)
--
Alkis
On Wed, Jul 8, 2009 at 9:12 AM, chenbaiping <chenb...@revenco.com> wrote:
No. I can fetch other servers in China, such as http://baidu.com.
And I can use wget to get this page outside China. So China does not block the side http://cbc007.cn/
发件人: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] 代表 Alkis Evlogimenos ('Αλκη? Ευλογημ?νο?)
发送时间: 2009年7月8日 14:57
收件人: google-a...@googlegroups.com
主题: [google-appengine] Re: a bug of urlfetch on GAE production server
Maybe the fetch service is blocked in China?
On Wed, Jul 8, 2009 at 4:14 AM, chenbaiping <chenb...@revenco.com> wrote:
When I fetch it using http://shell.appspot.com/, an error occurs.
When I fetch it in my app, it has the same error.
>>> r=urlfetch.fetch('http://cbc007.cn/test.html')
Traceback (most recent call last):
File "/base/data/home/apps/shell/1.334417654335420704/shell.py", line 278,
in get
exec compiled in statement_module.__dict__
File "<string>", line 1, in <module>
File "/base/python_lib/versions/1/google/appengine/api/urlfetch.py", line
241, in fetch
return rpc.get_result()
File
"/base/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py",
line 442, in get_result
return self.__get_result_hook(self)
File "/base/python_lib/versions/1/google/appengine/api/urlfetch.py", line
331, in _get_fetch_result
raise DownloadError(str(err))
DownloadError: ApplicationError: 5
Sometimes, it throws "DownloadError: ApplicationError: 2" error. Very
curious.
But it occurs on GAE production server only. On local sdk, it works fine.
Is it a bug?
--
Alkis
____ KILLӊȫؠґɨèKբⓊ ____
--
Alkis
Message has been deleted
Wooble
Jul 8, 2009, 1:10:35 PM7/8/09
to Google App Engine
It's possible your government is blocking the IP that the urlfetch
service runs on. Being able to access "GAE" from it is meaningless;
app engine itself runs on many servers and the ones serving your
requests aren't necessarily the same ones originating urlfetch calls.
On Jul 8, 11:22 am, "chenbaiping" <chenbaip...@revenco.com> wrote:
> It seems that GAE blocks the IP. GAE cann’t access the IP 121.11.86.71(cbc007.cn). But I can access GAE from the IP 121.11.86.71.
>
> Why GAE blocks that ip? It is just a virtual hosting server. It’s very strange.
>
> But my app needs to fetch some data from my virtual host. What could I do ?
>
> _____
>
> No. I can fetch other servers in China, such ashttp://baidu.com<http://baidu.com/> .
> ____ KILLӊȫؠґɨèKբⓊ ____
>
> ____ KILL ʼ ȫ Ѿ ɨ ʼ ____
service runs on. Being able to access "GAE" from it is meaningless;
app engine itself runs on many servers and the ones serving your
requests aren't necessarily the same ones originating urlfetch calls.
On Jul 8, 11:22 am, "chenbaiping" <chenbaip...@revenco.com> wrote:
> It seems that GAE blocks the IP. GAE cann’t access the IP 121.11.86.71(cbc007.cn). But I can access GAE from the IP 121.11.86.71.
>
> Why GAE blocks that ip? It is just a virtual hosting server. It’s very strange.
>
> But my app needs to fetch some data from my virtual host. What could I do ?
>
> _____
>
> 发件人: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] 代表 Alkis Evlogimenos ('Αλκη? Ευλογημ?νο?)
> 发送时间: 2009年7月8日 19:55
> 发件人: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] 代表 Alkis Evlogimenos ('Αλκη? Ευλογημ?νο?)
> 收件人: google-a...@googlegroups.com
> 主题: [google-appengine] Re: a bug of urlfetch on GAE production server
>
> 主题: [google-appengine] Re: a bug of urlfetch on GAE production server
>
> The question is can Google access this page outside of China. The fact that you can is irrelevant :-)
>
> On Wed, Jul 8, 2009 at 9:12 AM, chenbaiping <chenbaip...@revenco.com> wrote:
>
>
> No. I can fetch other servers in China, such ashttp://baidu.com<http://baidu.com/> .
>
> And I can use wget to get this page outside China. So China does not block the sidehttp://cbc007.cn/
>
> _____
>
> And I can use wget to get this page outside China. So China does not block the sidehttp://cbc007.cn/
>
> _____
>
> 发件人: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] 代表 Alkis Evlogimenos ('Αλκη? Ευλογημ?νο?)
> 发送时间: 2009年7月8日 14:57
> 收件人: google-a...@googlegroups.com
> 主题: [google-appengine] Re: a bug of urlfetch on GAE production server
>
> Maybe the fetch service is blocked in China?
>
> 发送时间: 2009年7月8日 14:57
> 收件人: google-a...@googlegroups.com
> 主题: [google-appengine] Re: a bug of urlfetch on GAE production server
>
> Maybe the fetch service is blocked in China?
>
> On Wed, Jul 8, 2009 at 4:14 AM, chenbaiping <chenbaip...@revenco.com> wrote:
>
> I have a very simple html pagehttp://cbc007.cn/test.html.
>
> When I fetch it usinghttp://shell.appspot.com/, an error occurs.
>
> I have a very simple html pagehttp://cbc007.cn/test.html.
>
>
> ____ KILL ʼ ȫ Ѿ ɨ ʼ ____
Message has been deleted
Nick Johnson (Google)
Jul 9, 2009, 5:13:53 AM7/9/09
to google-a...@googlegroups.com
Hi chenbaiping,
The IPs that App Engine does URL fetches from are not the same ones it
serves requests from. They may be on a different subnet, or even
served from a different datacenter - so it's entirely possible that
China is blocking the IPs App Engine does URL fetches from, even if
it's not blocking the serving IPs.
-Nick Johnson
On Thu, Jul 9, 2009 at 2:34 AM, chenbaiping<chenb...@revenco.com> wrote:
>
> No. China does not block the GAE IP. Because I can use GAE to fetch other web sites in China.
> For example, IP 121.11.66.71 is another IP and is in the same subnet with my virtual host, which ip is 121.11.86.71 . GEA can fetch 121.11.66.71 succefully every time, but cannot fetch 121.11.86.71.
>
> And I can use other servers hosted in USA to wget my IP 121.11.86.71 . So China does not block my IP too.
>
> So the conclusion is GAE block my IP.
>
> Has GAE some blocking patterns? And my ip unfortunately matches it?
>
> Can Google engineers check where the problem is ?
> Thanks.
>
>
>
> -----邮件原件-----
> 发件人: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] 代表 Wooble
> 发送时间: 2009年7月9日 1:11
> 收件人: Google App Engine
> ____ KILLʼȫ Ѿɨʼ ____
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number: 368047
The IPs that App Engine does URL fetches from are not the same ones it
serves requests from. They may be on a different subnet, or even
served from a different datacenter - so it's entirely possible that
China is blocking the IPs App Engine does URL fetches from, even if
it's not blocking the serving IPs.
-Nick Johnson
On Thu, Jul 9, 2009 at 2:34 AM, chenbaiping<chenb...@revenco.com> wrote:
>
> No. China does not block the GAE IP. Because I can use GAE to fetch other web sites in China.
> For example, IP 121.11.66.71 is another IP and is in the same subnet with my virtual host, which ip is 121.11.86.71 . GEA can fetch 121.11.66.71 succefully every time, but cannot fetch 121.11.86.71.
>
> And I can use other servers hosted in USA to wget my IP 121.11.86.71 . So China does not block my IP too.
>
> So the conclusion is GAE block my IP.
>
> Has GAE some blocking patterns? And my ip unfortunately matches it?
>
> Can Google engineers check where the problem is ?
> Thanks.
>
>
>
> -----邮件原件-----
> 发件人: google-a...@googlegroups.com [mailto:google-a...@googlegroups.com] 代表 Wooble
> 发送时间: 2009年7月9日 1:11
> 收件人: Google App Engine
>
>
>
>
> ____ KILL�ʼ���ȫ��� �Ѿ�ɨ��������ʼ� ____
>
>
>
> >
>
--
>
>
>
> ____ KILL�ʼ���ȫ��� �Ѿ�ɨ��������ʼ� ____
>
>
>
> >
>
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number: 368047
Message has been deleted
N. Rosencrantz
Jan 21, 2017, 9:01:49 AM1/21/17
to Google App Engine
I'm getting the hitrate 95 % but I don't think that is real. I'm going to look at it.
Reply all
Reply to author
Forward
0 new messages