Why the !@#$%^&* doesn't Opera respect the disk cache size?
0
"Disk cache" set to 50 MB.
If I run du -s on ~/.opera/cache, I get:
211782 .opera/cache/
I would expect something this basic to be fixed several
versions ago, but it still exists in 10.63.
Eirik Byrkjeflot Anonsen
There are some issues with this.
"du" and opera probably counts the "disk cache size" differently. "du"
tries to estimate the total amount of physical disk bits used by all
files in the directory. Opera probably only counts the size of the
actual data. There are two main discrepancies here. On the one hand du
will also count the size of the index, which I suspect opera does not
count (you can argue that this is a bug, and I won't necessarily
disagree with you...). And on the other hand, du will try to compensate
for internal fragmentation, which pretty much means rounding up all file
sizes to the nearest multiple of 4k. For small files, this can make a
pretty big difference.
The "disk cache" setting is unlikely to keep opera from growing the disk
cache if opera thinks it is necessary. If the page(s) you are viewing
needs more data than fits in the disk cache, opera will surely choose to
use a larger disk cache rather than refuse to display the page.
In fact, I believe the "disk cache" setting is more properly thought of
as a "minimum cache size" rather than a "maximum cache size". Or to be
more precise, "how much old data to keep around, just in case". I think
the way the cache works is that opera will try to delete things from the
disk cache until it is smaller than the disk cache setting. But opera
probably does not delete data that is very likely to be needed again
very soon, such as data that is currently being displayed. And maybe
this extends to data that have been unused for only a short time (in the
"couple of minutes" range).
eirik
Remco Lanting
<ei...@opera.com> wrote:
> The "disk cache" setting is unlikely to keep opera from growing the disk
> cache if opera thinks it is necessary. If the page(s) you are viewing
> needs more data than fits in the disk cache, opera will surely choose to
> use a larger disk cache rather than refuse to display the page.
Exactly. Since the OS can have Opera return RAM at any time, everything
needs to be kept on disk too. This effectively means that the disk cache
has a minimum size of the data used by all open pages (and perhaps also
pages in 'closed tabs' history). At least this is what I believe is
happening.
> In fact, I believe the "disk cache" setting is more properly thought of
> as a "minimum cache size" rather than a "maximum cache size". Or to be
> more precise, "how much old data to keep around, just in case". I think
> the way the cache works is that opera will try to delete things from the
> disk cache until it is smaller than the disk cache setting. But opera
> probably does not delete data that is very likely to be needed again
> very soon, such as data that is currently being displayed. And maybe
> this extends to data that have been unused for only a short time (in the
> "couple of minutes" range).
Keeping the top paragraph in mind, it makes sense when you see that Opera
trims the disk cache to the set size when you close it. No pages open
means the size can apply whatever happens.
--
Remco Lanting
[Unofficial Opera bug tracker links]
http://opera.remcol.ath.cx/bugs |
http://my.opera.com/community/forums/topic.dml?id=217364 |
remco.lanting...@gmail.com
Jorgen Grahn
> On Mon, 13 Dec 2010 08:35:46 +0100, Eirik Byrkjeflot Anonsen
> <ei...@opera.com> wrote:
>
>> The "disk cache" setting is unlikely to keep opera from growing the disk
>> cache if opera thinks it is necessary. If the page(s) you are viewing
>> needs more data than fits in the disk cache, opera will surely choose to
>> use a larger disk cache rather than refuse to display the page.
>
> Exactly. Since the OS can have Opera return RAM at any time, everything
> needs to be kept on disk too.
Uh, no it can't, and no, it doesn't. You're probably thinking of
virtual memory, but the Opera disk cache is a pure application-level
feature.
/Jorgen
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
0
> On the one hand du will also count the size of the
> index, which I suspect opera does not count (you
> can argue that this is a bug, and I won't necessarily
> disagree with you...).
You (or somebody there) should know (not just suspect)
whether opera counts the index or not. :)
> And on the other hand, du will try to compensate
> for internal fragmentation, which pretty much
> means rounding up all file sizes to the nearest
> multiple of 4k.
That's not "internal fragmentation" - it is called
"slack space." And it is not always 4k on linux
(indeed, for a filesystem with a lot of small files,
which is what one would get in a browser cache, a 1k
block size would be much more efficient).
> The "disk cache" setting is unlikely to keep opera
> from growing the disk cache if opera thinks it is
> necessary. If the page(s) you are viewing needs
> more data than fits in the disk cache, opera will
> . surely choose to use a larger disk cache rather
> than refuse to display the page.
This seems like the wrong behavior to me. Opera
should just ask for more ram to display the page (it
is certainly not shy about that, as I have mentioned
previously, I have seen Opera's ram usage grow to
more than 3 GB.
> In fact, I believe the "disk cache" setting is more
> properly thought of as a "minimum cache size" rather
> than a "maximum cache size".
Why would a user want to specify a minimum cache size
(as opposed to setting some kind of limit so that a
program doesn't suck up all of the available resources)?
The history settings APPEAR to allow the user to make
a choice between caching in memory or caching on the
disk. In reality, those setting seem to have very
little (no) effect (the "empty now" button does not
even work).
> I think the way the cache works is that opera will
> try to delete things from the disk cache until it
> is smaller than the disk cache setting.
I don't see that behavior. The disk cache just keeps
growing and growing endlessly (until I remove it via
a shell command - "rm -fr /home/user/.opera/cache").
By the way, doing that doesn't affect the display of
any open pages.
> And maybe this extends to data that have been unused
> for only a short time (in the "couple of minutes"
> range).
Why? Do what the user wants. This still seems very
broken to me.
Jorgen Grahn
> On Mon, 13 Dec 2010 07:35:46 -0000, Eirik Byrkjeflot Anonsen <ei...@opera.com> wrote:
>
>> On the one hand du will also count the size of the
>> index, which I suspect opera does not count (you
>> can argue that this is a bug, and I won't necessarily
>> disagree with you...).
>
> You (or somebody there) should know (not just suspect)
> whether opera counts the index or not. :)
Eirik rarely claims to know all about Opera, and his presence here
seems more like a useful hobby than an official part of Opera support.
So remember to cut him some slack, okay?
0
> Eirik rarely claims to know all about Opera, and his
> presence here seems more like a useful hobby than an
> official part of Opera support.
> So remember to cut him some slack, okay?
Nobody knows "all" about something as complicated as
Opera, but sometimes it seems better to say "I know
nothing about that" rather than make a guess with
inadequate knowledge.
Besides, he probably has decent access to the people
who DO know how it works. :)
Eirik Byrkjeflot Anonsen
> On Mon, 13 Dec 2010 07:35:46 -0000, Eirik Byrkjeflot Anonsen <ei...@opera.com> wrote:
>
>> On the one hand du will also count the size of the
>> index, which I suspect opera does not count (you
>> can argue that this is a bug, and I won't necessarily
>> disagree with you...).
>
> You (or somebody there) should know (not just suspect)
> whether opera counts the index or not. :)
I'm sure someone here does :)
>> And on the other hand, du will try to compensate
>> for internal fragmentation, which pretty much
>> means rounding up all file sizes to the nearest
>> multiple of 4k.
>
> That's not "internal fragmentation" - it is called
> "slack space." And it is not always 4k on linux
> (indeed, for a filesystem with a lot of small files,
> which is what one would get in a browser cache, a 1k
> block size would be much more efficient).
Sure. And if you use one of the small-file-efficient file systems
(e.g. reiserfs), this has hardly any effect at all. (But I still
haven't investigated what du will do in that case...)
>> The "disk cache" setting is unlikely to keep opera
>> from growing the disk cache if opera thinks it is
>> necessary. If the page(s) you are viewing needs
>> more data than fits in the disk cache, opera will
>> . surely choose to use a larger disk cache rather
>> than refuse to display the page.
>
> This seems like the wrong behavior to me. Opera
> should just ask for more ram to display the page (it
> is certainly not shy about that, as I have mentioned
> previously, I have seen Opera's ram usage grow to
> more than 3 GB.
Considering that ram is (usually) a more precious resource than disk, I
think that sounds like a poor choice... (And yes, I think 3 GB sounds
excessive. While it is easy to construct web pages where the browser
would be required to use such amounts of memory, I would hope that there
aren't many web pages around that do that...)
>
>> In fact, I believe the "disk cache" setting is more
>> properly thought of as a "minimum cache size" rather
>> than a "maximum cache size".
>
> Why would a user want to specify a minimum cache size
> (as opposed to setting some kind of limit so that a
> program doesn't suck up all of the available resources)?
The cache is the amount of data that is kept around even though it isn't
actually needed right now. Clearly, there must be a setting for how
much such data to store. That would be your minimum size. Opera's
"cache" also includes data that is "in use" right now, which could
require the use of more than the minimum size. Otherwise, the maximum
size would be about the same as the minimum size.
Except that it isn't implemented as a hard limit (or at least, it wasn't
the last time I looked at it). So the cache could (in principle)
temporarily grow arbitrarily large until a cache cleanup comes around to
shrink it again.
Yes, I can imagine users wanting to set a hard maximum size as well
(I've certainly wanted to do so on some of our device projects...).
I do fully agree that most users probably expect "cache size" to mean
"maximum cache size". And it is unfortunate that opera does not follow
users' expectations. But that's how the code works today.
> The history settings APPEAR to allow the user to make
> a choice between caching in memory or caching on the
> disk. In reality, those setting seem to have very
> little (no) effect (the "empty now" button does not
> even work).
If the "empty now" button doesn't do anything, I'd say that is a sign
that opera actually thinks all the data in the cache is "in use". (Or
the "empty now" button is broken for some reason, of course...)
>> I think the way the cache works is that opera will
>> try to delete things from the disk cache until it
>> is smaller than the disk cache setting.
>
> I don't see that behavior. The disk cache just keeps
> growing and growing endlessly (until I remove it via
> a shell command - "rm -fr /home/user/.opera/cache").
Unless the pages viewed actually requires such increasing amounts of
data, that sounds like a bug to me. Though I haven't noticed such
behaviour myself. My cache at home (which has been running for a few
months now...) is 423 MB (according to du).
> By the way, doing that doesn't affect the display of
> any open pages.
The details of this tend to be a bit magic. (But yes, currently
displayed data is probably stored in memory. However, the original data
is probably not. So if we should need that again, we would usually have
to download it again.)
>> And maybe this extends to data that have been unused
>> for only a short time (in the "couple of minutes"
>> range).
>
> Why? Do what the user wants. This still seems very
> broken to me.
We always try to do what the user wants. But it isn't trivial to figure
out what that is. I would suggest that this particular policy may be
intended to avoid worst-case scenarios where the user re-downloads the
same data every 10 seconds. (Admittedly a bigger problem in the time of
56k pay-per-byte dial-up connections rather than 100 Mbit unlimited data
always-on connections. Then again, fairly large parts of the world are
still in that time...)
0 <ro...@mouse-potato.com> writes:
> On Fri, 17 Dec 2010 19:44:02 -0000, Jorgen Grahn <grahn...@snipabacken.se> wrote:
>
>> Eirik rarely claims to know all about Opera, and his
>> presence here seems more like a useful hobby than an
>> official part of Opera support.
>> So remember to cut him some slack, okay?
>
> Nobody knows "all" about something as complicated as
> Opera, but sometimes it seems better to say "I know
> nothing about that" rather than make a guess with
> inadequate knowledge.
But that would give you no useful information. A bit of guesswork could
give you something to work with.
> Besides, he probably has decent access to the people
> who DO know how it works. :)
Yes, but finding and asking them would cost me more effort :)
eirik