Need some advice - Bucket and Caching

[Dimitri Adamou]

Hey there, we are looking at changing servers as we have very high storage and processing needs - so a bit of a background on how one process works

1. hits the index file
2. looks at the URL
3. if cache exists skip to step 6
4. scans directory reads XML's for any futher rules
5. caches the results based on the URL
6. deliver content

As these small cache files are roughly around 3-4kb each, I'm just wondering if Google can operate normally with me keeping this process in or skipping the cache part completely (Read the Bucket vs scanning the content locally - I'm certain local scan is faster)

I have various other cache's implemented but I believe those are safe to stay as they are generally larger processed files

My program has a very high File I/O as we are a Form service - Is Google App Engine for us?

[Barry Hunter]

Google doesnt give you magically powerful computers. 

What they do is give you access to *lots* and *lots* of relatively slow computers. Able to work in parallel. 

Scale Out, rather than Scale Up. 

Its possibly caching would be even more important on App Engine. But App Engine can give you access to *very* big caches. 

If by I/O you mean lots of small concurrent requests (rather than few big), then AppEngine may well work for you. 

Particularly if access patterns are very bursty

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[Vinny P]

On Sun, Aug 17, 2014 at 6:56 PM, Dimitri Adamou <dimitri...@mitechnologies.com.au> wrote:

As these small cache files are roughly around 3-4kb each, I'm just wondering if Google can operate normally with me keeping this process in or skipping the cache part completely (Read the Bucket vs scanning the content locally - I'm certain local scan is faster)

I have various other cache's implemented but I believe those are safe to stay as they are generally larger processed files

My program has a very high File I/O as we are a Form service - Is Google App Engine for us?

 

 

 It would really depend on how many of these cache files you have, the performance you are expecting, and the price you're willing to pay. Depending on the number of files, you could conceivably store the entire set in instance memory, which would make your application very fast. Even if that wasn't possible, you could also store the cache files as memcache objects - it would still be fast, but you'd need to pay for dedicated memcache service (you can try using shared memcache, but from your description you'll want the better cache hit ratio of dedicated).

 

In short: start sharing some performance numbers and your current setup!

 

 

-----------------

-Vinny P

Technology & Media Consultant

Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com