File cache using lots of RAM

[Phill Brown]

Joomla 1.7

Does anyone on this list use the file cache with large sites?
Using the file cache is using massive amounts of memory.

When I have the file cache enabled on my site I run out of memory on
this line once the cache grows
PHP Fatal error: Allowed memory size of XXX bytes exhausted (tried to
allocate XXX bytes) in
/www/xxx/public_html/libraries/joomla/cache/storage/file.php on line
133

With the file cache enabled each apache process will grow in excess of 500 MB.
With the file cache disabled the processes reach 40 MB

Could this be an issue with the implementation of the file cache (A
Bug) or is this normal for a large cache using the file system?

1 _system 8 3.66 kb
2 com_banners 2 235 b
3 com_classifieds 105279 440.15 kb
4 com_content 6 8.34 kb
5 com_languages 2 460 b
6 com_modules 8 408.24 kb
7 com_plugins 4 22.14 kb
8 com_templates 2 1.59 kb
9 images 0 0
10 mod_banners 2 4.06 kb
11 mod_classifieds_categories 94671 347.8 kb
12 mod_classifieds_localities 94674 686.59 kb
13 mod_custom 3 7.28 kb
14 mod_menu 16 87.99 kb

Regards,

Phill Brown
M  04 2481 9754
Bathurst Software Solutions
-------------------------------------------------------------------------------------------------------------------

[Nicholas K. Dionysopoulos]

Hi Phill,

Well, I do, on AkeebaBackup.com, but I don't have this issue :)

I think your problem is the classifieds extension. It seems to be growing a huge list of cache files. Also note that Joomla! lies about the size of each cache directory. If you use an FTP application you'll probably discover that these huge cache directories have grown to a few hundred Mb or even a few Gb each. I had that problem a year ago, when I was wondering why the 1Gb I had allocated ran out, despite my whole site being only 100Mb (excluding the cache which I never thought would grow that big).

That said, I believe that you can solve the problem you're experiencing, without sacrificing caching or the classifieds extension, by setting the cache to "ON - Conservative caching" instead of the "ON - Progressive caching" option.

Cheers,

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/joomla-dev-general?hl=en-GB.

[Phill Brown]

Hi Nickolas,

Thanks for the quick reply.

The number of cache files was to be expected. The site has a large
number of pages based on locality and category. Each individual page
requires a fresh query and displays a different set of data. It is
this query that I am caching in both modules and the component.
I did wonder about the joomla cache folder size - I was getting an
actual of around 300MB vs 615KB reported.

With the caching set to Conservative I was still getting massive RAM
usage a high number of files in each com_classifieds,
mod_classifieds_localities and mod_classifieds_categories directories.

Any other ideas would be appreciated?

Regards,

Phill Brown
M  04 2481 9754
Bathurst Software Solutions
-------------------------------------------------------------------------------------------------------------------

[Nicholas K. Dionysopoulos]

Hi Phill,

Are you getting the memory exhaustion message on the same line of the same file?

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Phill Brown]

Yes - always the same line, same file.

Regards,

Phill Brown
M  04 2481 9754
Bathurst Software Solutions
-------------------------------------------------------------------------------------------------------------------

On Wed, Sep 7, 2011 at 11:03 PM, Nicholas K. Dionysopoulos

[Nicholas K. Dionysopoulos]

Hi Phil,

I am not sure why this may be happening. That line belongs to the store() method which saves the cache data to disk. All it does is open the file, write the data, then read back the file to verify that the data was correct. The total memory usage should be twice the size of the data which is being written to cache plus 20-something bytes for the die() statement on top. Is any of the cache files exceptionally big, like in the tenths of Megabytes? This could be the only reason I can think of of this happening - and also explain why your Apache instance size has grown to a gazillion Megabytes.

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Phill Brown]

The cache files were quite small - with 100K files the cache storage
used was under 300MB.
The average cache file is from 4kb to 40kb. The larger ones are 300kb.

This has me stumped.

Regards,

Phill Brown
M  04 2481 9754
Bathurst Software Solutions
-------------------------------------------------------------------------------------------------------------------

On Wed, Sep 7, 2011 at 11:13 PM, Nicholas K. Dionysopoulos

[Nicholas K. Dionysopoulos]

Hi Phill,

It sounds as if there is a memory leak somewhere in the cache handler. I am not sure where, though. The only place where excessive RAM could be used is in the getAll() function -which lists all of the cache files in each folder- but I can't see it being used anywhere except the back-end page which lists the number of files and the size of the cache. Maybe someone which is better versed in the inner working of the Joomla! cache system could provide us with some insight.

-- 
Nicholas K. Dionysopoulos
Lead Developer, AkeebaBackup.com

Web: http://www.AkeebaBackup.com

Blog: http://www.dionysopoulos.me/blog

[Ryan Kartzke]

Hi Phil,

I am also getting this caching error on one of my sites.  The website is not particularly big, about 500 pages.  I'm running Joomla 2.5.1 on PHP 5.3.5, and my fatal error comes up on line 142:

"Fatal error: Allowed memory size of 146800640 bytes exhausted (tried to allocate 15656475 bytes) in /libraries/joomla/cache/

storage/file.php on line 142"

Line 142 is for a "Data Integrity Check"..

// Data integrity check
        if ($written && ($data == file_get_contents($path)))
        {
            return true;
        }
        else
        {
            return false;
        }
    }


I examined my cache folder and found similar results to yours... typically small files, most are under 40 KB:

com_content - 80 files totaling 755 KB - Average size ~6-9 KB
com_fabrik - 3 files totaling 76.1 KB - Each file = 26 KB
com_k2 - 267 files totaling 2.99 MB - Average size ~10-20 KB
com_k2_extended - 345 files totaling 9.72 MB - Average size ~25-30 KB
com_languages - 1 file, 1 KB
com_modules - 1 file, 2.6 KB
com_plugins - 3 files, 14 KB each
com_templates - 2 files, 40 KB total
Gantry - 5 files, 65.3 KB total
mod_custom - 6 files, 8.29 KB
mod_rokgallery - 1 file, 1 KB
sh404sef_rconfig - 1 file, 1KB

I also had a number of compressed Javascript files in the cache directory: 40 files, 14.4 MB, average size is 400KB.  RokGZipper was doing the compression.

Did you ever find a resolution for this problem?  Was it a Joomla bug, or a configuration error of some sort?  Any help is greatly appreciated.

[Seth Warburton]

Is everyone with this issue using Gantry? It seems that might be the case from the results posted here. 

Is there anyone *not* using Gantry who is experiencing these same issues?

Best,

Seth

[Phil Brown]

I was not using Gantry

Phil Brown

M  04 2481 9754
Bathurst Software Solutions
-------------------------------------------------------------------------------------------------------------------

--

You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-gene...@googlegroups.com.

To post to this group, send an email to joomla-de...@googlegroups.com.

Visit this group at http://groups.google.com/group/joomla-dev-general.
For more options, visit https://groups.google.com/groups/opt_out.