Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Caching a web query?
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  8 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Felix E. Klee  
View profile  
 More options Oct 1 2012, 4:13 pm
From: "Felix E. Klee" <felix.k...@inka.de>
Date: Mon, 1 Oct 2012 22:13:11 +0200
Local: Mon, Oct 1 2012 4:13 pm
Subject: Caching a web query?
In an app powered by Symfony 1.4, I would like to cache the response
from an ECB (European Central Bank) web service which provides currency
exchange rates [1]. While the web service generally responds very fast,
recently there have been outages lasting for hours.

The cache may work as follows:

  * A timeout is set to, say, ten minutes.

  * When loading the (Symfony powered) web page:

      - Before the timeout has been hit: Exchange rates are read from
        the cache.

      - After the timeout has been hit: The ECB web service is queried.
        If the ECB doesn't reply, then old data is used. Otherwise, the
        cache is refreshed with updated data. In any case, the timeout
        is extended.

  * When deploying, then the cache is emptied (part of the documented
    deployment process). This ensures that, if the ECB stops responding
    completely, then someone will become aware of it eventually. Of
    course, one could set up a system where an email is sent once the
    ECB hasn't responded for a day or so, but that seems overkill for
    the site in question.

Now, I could either store the cached data (currency exchange rates) in a
MySQL database, or I could use Cache_Lite [2], or:

Is there a specific Symfony way to do this? How would you do it?

[1]: http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml
[2]: http://pear.php.net/package/Cache_Lite


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Кирилл  
View profile  
 More options Oct 2 2012, 1:52 am
From: "Кирилл" <bestestm...@gmail.com>
Date: Tue, 2 Oct 2012 05:51:46 +0000
Local: Tues, Oct 2 2012 1:51 am
Subject: Re: [symfony1] Caching a web query?
Well I prefer this way: inside model which return bank query use
If($value = $memcache->get('key')){
    $value = $superModel->queryBank();
    $memcache->set('key', $value);
}

I don't remember how setup timeout, but memcache extension support timeout.
Отправлено с беспроводного устройства BlackBerry®


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Gareth McCumskey  
View profile  
 More options Oct 2 2012, 2:24 am
From: Gareth McCumskey <gmccums...@gmail.com>
Date: Tue, 2 Oct 2012 08:24:14 +0200
Local: Tues, Oct 2 2012 2:24 am
Subject: Re: [symfony1] Caching a web query?

Best thing to use is symfony's caching classes that tie into memory caches
such as APC, EAccelerator or Memcache. Personally I would recommend APC as
this has been more fully integrated into PHP. Also, memcaching is ideal for
your use case because it is transitory data. Storing temporary data in a
database should be avoided as much as possible usually.

--
Gareth McCumskey
http://garethmccumskey.blogspot.com
twitter: @garethmcc
identi.ca: @garethmcc

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Felix E. Klee  
View profile  
 More options Oct 2 2012, 4:17 am
From: "Felix E. Klee" <felix.k...@inka.de>
Date: Tue, 2 Oct 2012 10:17:03 +0200
Local: Tues, Oct 2 2012 4:17 am
Subject: Re: [symfony1] Caching a web query?
On Tue, Oct 2, 2012 at 8:24 AM, Gareth McCumskey <gmccums...@gmail.com>
wrote:

> Best thing to use is symfony's caching classes that tie into memory
> caches such as APC, EAccelerator or Memcache.

Thanks for the suggestion. However, caching to memory has one major
disadvantage:

Suppose the server was restarted, and I discover that - out of bad luck
- the ECB webservice is not available currently. Then there would be no
backup of the last response.

> Storing temporary data in a database should be avoided as much as
> possible usually.

While, for various reasons, I also don't really want to use the database
here, I don't think using it would have any performance impact. After
all, when a page is loaded, there are various database queries anyhow.
The database is fast. It is used for real-time autocompletion on the
web-site, for example.

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Felix E. Klee  
View profile  
 More options Oct 2 2012, 4:27 am
From: "Felix E. Klee" <felix.k...@inka.de>
Date: Tue, 2 Oct 2012 10:27:03 +0200
Local: Tues, Oct 2 2012 4:27 am
Subject: Re: [symfony1] Caching a web query?

On Tue, Oct 2, 2012 at 7:51 AM, Кирилл <bestestm...@gmail.com> wrote:
> If($value = $memcache->get('key')){
>     $value = $superModel->queryBank();
>     $memcache->set('key', $value);
> }

Thanks for the suggestion, but the logic would have to be different, as
the bank sometimes has long lasting outages:

   if ($cache->timeoutReached()) {
     if ($newRates = queryBank()) {
       $cache->set('rates', $newRates);
     } // else: outage => keep old data
   }
   $rates = $cache->get('rates');
   $cache->setTimeoutFromNow(600);


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Felix E. Klee  
View profile  
 More options Oct 2 2012, 6:10 am
From: "Felix E. Klee" <felix.k...@inka.de>
Date: Tue, 2 Oct 2012 12:09:33 +0200
Local: Tues, Oct 2 2012 6:09 am
Subject: Re: [symfony1] Caching a web query?
On Tue, Oct 2, 2012 at 10:27 AM, Felix E. Klee <felix.k...@inka.de>
wrote:

>    [...]
>    $rates = $cache->get('rates');
>    $cache->setTimeoutFromNow(600);

Sorry, this is BS - the timeout would never be hit. Correction:

    if ($cache->timeoutReached()) {
      if ($newRates = queryBank()) {
        $cache->set('rates', $newRates);
      } // else: outage =keep old data
      $cache->setTimeoutFromNow(600);
    }
    $rates = $cache->get('rates');

Any idea what caching system to use for the given purpose?

Quick thought: I could keep the timeout expiration time in memcache (or
APC, or whatever), and then just use standard PHP file funcitons to
read/write the file containing the rates. Memcache is necessary to share
the timeout expiration time among several processes.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Felix E. Klee  
View profile  
 More options Oct 2 2012, 10:26 am
From: "Felix E. Klee" <felix.k...@inka.de>
Date: Tue, 2 Oct 2012 16:26:21 +0200
Local: Tues, Oct 2 2012 10:26 am
Subject: Re: [symfony1] Caching a web query?
On Tue, Oct 2, 2012 at 12:09 PM, Felix E. Klee <felix.k...@inka.de>
wrote:

> I could keep the timeout expiration time in memcache [...]

Facepalm! I could just use the timestamp of the file with the rates
instead.

I am also investigating using the caching system that Symfony uses for
configuration data.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Felix E. Klee  
View profile  
 More options Oct 6 2012, 6:48 pm
From: "Felix E. Klee" <felix.k...@inka.de>
Date: Sun, 7 Oct 2012 00:48:07 +0200
Local: Sat, Oct 6 2012 6:48 pm
Subject: Re: Caching a web query?
Eventually, I used `sfFileCache`, but with custom expiration system
(standard lifetime set to `PHP_INT_MAX`): The custom system avoids
clearing of the expired cache when the ECB server doesn't respond. For
the given application, slightly outdated data is better than no data.

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »