Hex dump of no-cache, no-store files in memory cache listing
Dave Corsello
I'm new to this newsgroup. Daniel Veditz at Bugzilla recommended that I
post information here regarding bug request 477743, which I created
yesterday (https://bugzilla.mozilla.org/show_bug.cgi?id=477743).
To briefly summarize my request, Firefox 3.0.x displays a hex dump in
its memory cache listing of files that contain no-cache and no-store
headers. This renders these headers useless to the webmaster who intends
to use them as a means of preventing the permanent storage, via the
browser, of served files on the client computer. It does this by
providing text data that may be saved and converted to binary format.
Daniel points out in his response to my request that "suppressing the
hex-dump in the no-cache case makes sense," but would not "prevent
determined users from getting your content." This is true, but I
believe that the browser should not be the weak link where the security
of copyrighted material is concerned. As it is, Firefox 3 stands out as
the only browser (to my knowledge) that allows no-cache, no-store files
to be stored permanently on the client's hard drive. Firefox 2 did not
have this problem.
I'm not sure how this goes from here, but thanks for the opportunity to
voice my concern.
Sincerely,
Dave Corsello
L. David Baron
Are you talking about the memory cache...
On Tuesday 2009-02-10 18:37 -0500, Dave Corsello wrote:
> To briefly summarize my request, Firefox 3.0.x displays a hex dump in
> its memory cache listing of files that contain no-cache and no-store
... or the disk cache ...
> of copyrighted material is concerned. As it is, Firefox 3 stands out as
> the only browser (to my knowledge) that allows no-cache, no-store files
> to be stored permanently on the client's hard drive. Firefox 2 did not
?
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Boris Zbarsky
> To briefly summarize my request, Firefox 3.0.x displays a hex dump in
> its memory cache listing of files that contain no-cache and no-store
> headers. This renders these headers useless to the webmaster who intends
> to use them as a means of preventing the permanent storage, via the
> browser, of served files on the client computer.
Is this permanent storage the user wishes, or does not wish?
You have no way to prevent the former: it can be done via Firefox
extensions, via the "save as" functionality (if it's working correctly),
via a packet sniffer. That's off the top of my head; there are probably
more.
no-store and no-cache are used for situations where once the browser
_quits_ an attacker should not be able to get the data from the user's
hard drive. That is, they're designed to protect the user. The site is
already sending the bits out on the wire, so it clearly wants the
information given to the user.
> This is true, but I
> believe that the browser should not be the weak link where the security
> of copyrighted material is concerned.
I don't think the browser should be in the business of enforcing
copyrights at all. It sounds like you're using the wrong tool for the
job if that's what you want here. Perhaps you wish a dedicated viewer
that will do its own (encrypted) network access?
> As it is, Firefox 3 stands out as
> the only browser (to my knowledge) that allows no-cache, no-store files
> to be stored permanently on the client's hard drive. Firefox 2 did not
> have this problem.
Firefox 2 allows "save as" of no-cache, no-store content, last I
checked. So do a number of other browsers.
-Boris
Dave Corsello
> Are you talking about the memory cache...
>
> On Tuesday 2009-02-10 18:37 -0500, Dave Corsello wrote:
>> To briefly summarize my request, Firefox 3.0.x displays a hex dump in
>> its memory cache listing of files that contain no-cache and no-store
>
> ... or the disk cache ...
>
>> of copyrighted material is concerned. As it is, Firefox 3 stands out as
>> the only browser (to my knowledge) that allows no-cache, no-store files
>> to be stored permanently on the client's hard drive. Firefox 2 did not
>
> ?
>
> -David
>
Hi David,
The problem is with the memory cache listing. It exposes the entire
contents of a binary file in hex dump format. The dump text can be saved
and converted to binary data.
--Dave
Dave Corsello
Thanks for your response. My responses are below.
--Dave
Boris Zbarsky wrote:
> Dave Corsello wrote:
> Is this permanent storage the user wishes, or does not wish?
If the standard were honored in this case the way it is in every other
case, the user's wishes would be moot.
> You have no way to prevent the former: it can be done via Firefox
> extensions, via the "save as" functionality (if it's working correctly),
I should have mentioned the specifics of my application. I'm streaming
MP3 files that contain no-cache and no-store headers through Flash. I'm
sure that it's possible to record Flash-streamed audio in real-time
somehow. But if I'm not mistaken, there's currently no way for these
files to be saved directly to disk, other than by using Firefox 3,
saving the hex dump from its memory cache listing to a text file, and
converting that text into binary format.
It seems likely to me that Mozilla never intended for the dump listing
to be used as a means of saving protected files. It was probably
intended to be informational only. But it exposes Flash-streamed,
no-cache files, allowing them to be stored, which is something that no
other browser allows.
> via a packet sniffer. That's off the top of my head; there are
> probably more.
I don't think this lends strength to your argument. Security standards
of all kinds can be circumvented. But standards are needed and should be
adhered to. If there is a weak link, it should not be the browser.
> no-store and no-cache are used for situations where once the browser
> _quits_ an attacker should not be able to get the data from the user's
> hard drive. That is, they're designed to protect the user.
These standards are designed to keep files off of the user's hard drive.
Period. When honored, they protect both the user and the webmaster. In
every other browser, including older versions of Firefox, I don't need
to quit the browser in order to observe that no-cache files do not exist
on the hard drive.
> The site is already sending the bits out on the wire, so it clearly
> wants the information given to the user.
Universal Studios clearly wants its theater releases to be viewed by
audiences. But it doesn't want want people to bring camcorders to the
theater, and it doesn't want its releases leaked to the black market.
> I don't think the browser should be in the business of enforcing
> copyrights at all.
But it should be in the business of honoring standards. Firefox might
be honoring the "letter of the law" with regard to no-cache and no-store
standards, but it's clearly not honoring the "spirit of the law" when it
exposes the content of no-cache files in its memory cache listing.
> It sounds like you're using the wrong tool for the
> job if that's what you want here. Perhaps you wish a dedicated viewer
> that will do its own (encrypted) network access?
No, I just want Firefox to honor the standard the way it used to, by
returning to a memory cache listing that includes no hex dump of
no-cache files.
> Firefox 2 allows "save as" of no-cache, no-store content, last I
> checked. So do a number of other browsers.
Again, unless I'm mistaken, not in the case of MP3's streamed via Flash.
What value does the hex dump add for the Firefox user?
Benjamin Smedberg
> Boris Zbarsky wrote:
>> Dave Corsello wrote:
>> Is this permanent storage the user wishes, or does not wish?
>
> If the standard were honored in this case the way it is in every other
> case, the user's wishes would be moot.
That's the point. We are in the business of serving our users' wishes. If
they want to save data they have already downloaded, that's something we
should allow.
> I should have mentioned the specifics of my application. I'm streaming
> MP3 files that contain no-cache and no-store headers through Flash. I'm
> sure that it's possible to record Flash-streamed audio in real-time
> somehow. But if I'm not mistaken, there's currently no way for these
> files to be saved directly to disk, other than by using Firefox 3,
> saving the hex dump from its memory cache listing to a text file, and
> converting that text into binary format.
What happens if the users types the relevant URL into the browser window?
>> no-store and no-cache are used for situations where once the browser
>> _quits_ an attacker should not be able to get the data from the user's
>> hard drive. That is, they're designed to protect the user.
>
> These standards are designed to keep files off of the user's hard drive.
> Period. When honored, they protect both the user and the webmaster. In
> every other browser, including older versions of Firefox, I don't need
> to quit the browser in order to observe that no-cache files do not exist
> on the hard drive.
Fistly, this security standard is there to prevent saving sensitive
information to disk *inadvertantly*, especially in transparent proxies. If
the user wishes to save the content explicitly we definitely want to allow
that. You are trying to keep information from a user that the user has a
right to see, because you sent it to them.
Secondly, the information is not stored in the disk cache. It is only stored
in the memory cache.
>> The site is already sending the bits out on the wire, so it clearly
>> wants the information given to the user.
>
> Universal Studios clearly wants its theater releases to be viewed by
> audiences. But it doesn't want want people to bring camcorders to the
> theater, and it doesn't want its releases leaked to the black market.
You're saying that cache headers are a form of DRM. That's not true.
>> I don't think the browser should be in the business of enforcing
>> copyrights at all.
>
> But it should be in the business of honoring standards. Firefox might
> be honoring the "letter of the law" with regard to no-cache and no-store
> standards, but it's clearly not honoring the "spirit of the law" when it
> exposes the content of no-cache files in its memory cache listing.
We believe you misunderstand the spirit of the law.
--BDS
Boris Zbarsky
misconceptions here.
Dave Corsello wrote:
> If the standard were honored in this case the way it is in every other
> case, the user's wishes would be moot.
Actually, no. The HTTP standard explicitly says that the user's wishes
in various matters including, say, history navigation, trump the headers
sent over the network.
> I should have mentioned the specifics of my application.
I read the bug, so I know what the specifics are.
> It seems likely to me that Mozilla never intended for the dump listing
> to be used as a means of saving protected files.
It was meant to just be a dump listing. On our end there is no concept
of "protected files". Just data the user can see, which may or may not
be sensitive in terms of whether it's ok to expose it to others.
> I don't think this lends strength to your argument. Security standards
> of all kinds can be circumvented. But standards are needed and should be
> adhered to. If there is a weak link, it should not be the browser.
The security standard in question covers disclosure of information to
entities other than the user, not to the user.
> These standards are designed to keep files off of the user's hard drive.
> Period.
Uh... quoting RFC 2616, which defines no-store:
The purpose of the no-store directive is to prevent the inadvertent
release or retention of sensitive information (for example, on backup
tapes).
...
Even when this directive is associated with a response, users might
explicitly store such a response outside of the caching system (e.g.,
with a "Save As" dialog). History buffers MAY store such responses as
part of their normal operation.
That's a pretty clear description of what the header was designed to do,
seems to me. And it's not what you say.
> When honored, they protect both the user and the webmaster. In
> every other browser, including older versions of Firefox, I don't need
> to quit the browser in order to observe that no-cache files do not exist
> on the hard drive.
no-cache files exist on the hard drive just fine (with the exception of
SSL no-cache, which is treated as no-store unless "Cache-Control:
public" is sent). no-store data is kept in memory. See above for what
no-store means.
> Universal Studios clearly wants its theater releases to be viewed by
> audiences. But it doesn't want want people to bring camcorders to the
> theater, and it doesn't want its releases leaked to the black market.
Yes, but that's not the camcorder makers' problem. Nor should it be.
> But it should be in the business of honoring standards.
Yep. And we are. See the quote from the standard above.
> Firefox might be honoring the "letter of the law" with regard to no-cache and no-store
> standards, but it's clearly not honoring the "spirit of the law" when it
> exposes the content of no-cache files in its memory cache listing.
Uh... The spirit and intent of the law is described in the law in this
case. Again, see above. Looks to me like we're following it.
> What value does the hex dump add for the Firefox user?
That's a good question. Have you read the bug that added it? I
haven't, but presumably the reasons are described therein.
-Boris
Robert Kaiser
> Boris Zbarsky wrote:
>> Dave Corsello wrote:
>> Is this permanent storage the user wishes, or does not wish?
>
> If the standard were honored in this case the way it is in every other
> case, the user's wishes would be moot.
Mooting user wishes is so non-HTML and probably against the philosophy
of the open Internet that I'm not sure if Firefox even wants to enable
anything that could ignore the users wishes. Restricting freedom is not
what Firefox was made for or Mozilla is about.
Robert Kaiser
Dave Corsello
overwhelming bias in this discussion in favor of the user as a bit of a
logical disconnect. But I'm beginning to understand that where
copyrights are concerned, this discussion is a collision of two very
different world views, open-source and non-open-source. Mozilla's entire
existence is based on an open, sharing environment in which copyrights
don't exist. But as an entertainer, copyrights are essential to me. So,
because it's unlikely that this discussion will result in agreement if
pursued through the topic of copyrights, I'll leave that issue and
concentrate on standards.
Regarding the no-store directive, W3 states,
"The purpose of this directive is to meet the stated requirements of
certain users and service authors who are concerned about accidental
releases of information"
Please note the phrase "and service authors". Apparently, W3 considers
the wishes of the author worthy of consideration, as should Mozilla. The
standard continues:
"The purpose of the no-store directive is to prevent the inadvertent
release or retention of sensitive information (for example, on backup
tapes). The no-store directive applies to the entire message, and MAY be
sent either in a response or in a request. If sent in a request, a cache
MUST NOT store any part of either this request or any response to it. If
sent in a response, a cache MUST NOT store any part of either this
response or the request that elicited it. This directive applies to both
non-shared and shared caches."
And here is where I feel Mozilla is clearly not honoring the standard:
"'MUST NOT store' in this context means that the cache MUST NOT
intentionally store the information in non-volatile storage, and MUST
make a best-effort attempt to remove the information from volatile
storage as promptly as possible after forwarding it."
When I play a no-store MP3 through my web site, it is made available for
download through Firefox's memory cache listing long after it has
finished playing. In fact, it's available for the entire duration of my
session. This result is not allowed by the standard.
If an intruder were to search through Firefox's non-volatile cache, he
or she would not find my MP3's. But by browsing the memory cache
listing, the intruder could easily find and store my files to disk.
The standard continues:
"Even when this directive is associated with a response, users might
explicitly store such a response outside of the caching system (e.g.,
with a "Save As" dialog)."
One of the reasons why I stream my MP3's through flash is that it
removes the "Save As" dialog from the picture.
W3 concludes:
"History buffers MAY store such responses as part of their normal
operation."
This isn't applicable in my case, because Flash eliminates the ability
to scroll through a history of the individual files that it streams.
I _do not_ want my music stored to the user's hard drive, and have taken
every precaution that I know of, including the implementation of
standards that are designed explicitly to assist me towards that end.
Please fix this bug.
Boris Zbarsky
> Mozilla's entire existence is based on an open, sharing environment in which copyrights
> don't exist.
That's simply not true, as you can tell by reading the copyright license
for the source code...
> "The purpose of the no-store directive is to prevent the inadvertent
> release or retention of sensitive information (for example, on backup
> tapes). The no-store directive applies to the entire message, and MAY be
> sent either in a response or in a request. If sent in a request, a cache
> MUST NOT store any part of either this request or any response to it. If
> sent in a response, a cache MUST NOT store any part of either this
> response or the request that elicited it. This directive applies to both
> non-shared and shared caches."
>
> And here is where I feel Mozilla is clearly not honoring the standard:
>
> "'MUST NOT store' in this context means that the cache MUST NOT
> intentionally store the information in non-volatile storage, and MUST
> make a best-effort attempt to remove the information from volatile
> storage as promptly as possible after forwarding it."
>
> When I play a no-store MP3 through my web site, it is made available for
> download through Firefox's memory cache listing long after it has
> finished playing. In fact, it's available for the entire duration of my
> session. This result is not allowed by the standard.
The "MUST make a best-effort attempt to remove the information from
volatile storage as promptly as possible after forwarding it" is talking
about situations where forwarding is in fact happening. That would be
transparent caches, not client caches.
> "Even when this directive is associated with a response, users might
> explicitly store such a response outside of the caching system (e.g.,
> with a "Save As" dialog)."
>
> One of the reasons why I stream my MP3's through flash is that it
> removes the "Save As" dialog from the picture.
Yes, but the "save as" dialog is simply an example. The point of this
section is that no-store does NOT imply "the user shouldn't be able to
save this data".
> This isn't applicable in my case, because Flash eliminates the ability
> to scroll through a history of the individual files that it streams.
Again, just because you have adopted a technological solution that
prevents something the standard explicitly allows for doesn't mean that
a different technological solution must also prevent these things.
> I _do not_ want my music stored to the user's hard drive
Yes, I understand that. But we're in the business of implementing
standards, not serving your every wish. Especially if that wish happens
to be in conflict with the user's wishes. Please keep in mind that we
are a "User agent", right?
> and have taken
> every precaution that I know of, including the implementation of
> standards that are designed explicitly to assist me towards that end.
Except that the standard is NOT designed to prevent the client from
saving the data...
You're right that the existing HTTP infrastructure does not cater well
to your use case. I'm sorry about that, but the right solution to that,
as I said, is to either use an HTTP implementation that does exactly
what you want (e.g. a stand-alone client that imposes whatever
additional restrictons you desire but which HTTP does not require
clients to impose) or to live with the imperfect support for your use
case in existing HTTP clients. Of course if you go the "my own HTTP
client" route it's then up to your users whether to use said client.
-Boris
Mike Beltzner
I think that we should lay discussions of copyright and data ownership
aside, for the moment at least, and return to the actual issue at hand
which is that something has changed between Firefox 2 and Firefox 3
with respect to how we display this information in the about:cache page.
Dave asserts that this behavioural change (showing hex dump data for
no-cache / no-store files) is causing problems for him, and asks why
it was made. I haven't seen anyone answer that point directly. Do we
know when and why this changed?
cheers,
mike
Boris Zbarsky
> Do we know when and why this changed?
http://mxr.mozilla.org/firefox/find?string=aboutcache
Gets us to
http://bonsai.mozilla.org/cvslog.cgi?file=mozilla/netwerk/protocol/about/src/nsAboutCacheEntry.cpp
Which gets us to https://bugzilla.mozilla.org/show_bug.cgi?id=317627
Which is the necko module owner adding functionality to what is
basically a necko-debugging tool, as far as I can see.
-Boris
Mike Beltzner
*itself* is new, and from: https://bugzilla.mozilla.org/show_bug.cgi?id=317627
So, as I understand things we were always caching these files in
memory, and what we've changed is the ability to see that as a hex
dump. Next question is: without that capability, is there a way for a
user to get at that hex dump using other tools? Are were merely
simplifying access to that data or are we providing the only route to
it?
cheers,
mike
> it was made. I haven't seen anyone answer that point directly. Do we
> know when and why this changed?
>
> cheers,
> mike
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
Boris Zbarsky
> Next question is: without that capability, is there a way for a user to get
> at that hex dump using other tools? Are were merely simplifying access
> to that data or are we providing the only route to it?
I already answered that question before in this thread, but just to be
more explicit we also have a way for extensions to get at all HTTP data
as it comes in off the network. This is also new in Gecko 1.9, and is
used by at least Firebug. See bug 430155.
A user could also use at least the following methods to get this data,
sort of in order of how hard it is to separate the data from stuff
around it:
1) An extension that will just make a request to the URL in question.
Or heck, loading that URL in the browser. The URL is provided as
part of about:cache and has been all along. The extension might
be needed if we need to make absolutely sure we hit the cache only
and don't ask the server (e.g. if the server uses a one-time auth
token).
2) A packet sniffer (if not doing https).
3) A debugger (we provide symbols via the symbol server, so they don't
have to be particularly good with the debugger; just know which
data structures to poke).
4) Any of the various memory-examining tools that just let you dump
chunks of RAM to disk.
That's off the top of my head. I'm not sure it's worth me spending time
to come up with an "exhaustive" list.
-Boris
Dave Corsello
Thank you so much for continuing this dialog.
> That's simply not true, as you can tell by reading the copyright license
> for the source code...
Sorry about that. You're right. I was actually just trying to highlight
an issue that is extraneous to the discussion, and, as Mike suggested,
to set it aside.
> The "MUST make a best-effort attempt to remove the information from
> volatile storage as promptly as possible after forwarding it" is talking
> about situations where forwarding is in fact happening. That would be
> transparent caches, not client caches.
Mozilla (and all other browsers) must deem this standard applicable to
client caches in at least one sense, because without the directive, my
MP3's end up in disk cache, whereas with it, they do not.
--Dave
Dave Corsello
Sorry, this post might not lend weight to my argument, because honestly,
I'm not 100% sure that it's the no-store header that keeps the files out
of disk cache. For me to speak authoritatively, I'd have to test the
no-cache, no-store and expiration directives independently, and I
probably won't do that today.
Boris Zbarsky
>> The "MUST make a best-effort attempt to remove the information from
>> volatile storage as promptly as possible after forwarding it" is
>> talking about situations where forwarding is in fact happening. That
>> would be transparent caches, not client caches.
>
> Mozilla (and all other browsers) must deem this standard applicable to
> client caches in at least one sense, because without the directive, my
> MP3's end up in disk cache, whereas with it, they do not.
The requirement to not place in non-volatile storage at all applies to
all caches, since it doesn't involve forwarding.
-Boris
Dave Corsello
> Mike Beltzner wrote:
>> Next question is: without that capability, is there a way for a user
>> to get at that hex dump using other tools? Are were merely simplifying
>> access to that data or are we providing the only route to it?
>
> I already answered that question before in this thread, but just to be
> more explicit we also have a way for extensions to get at all HTTP data
> as it comes in off the network. This is also new in Gecko 1.9, and is
> used by at least Firebug. See bug 430155.
>
> A user could also use at least the following methods to get this data,
> sort of in order of how hard it is to separate the data from stuff
> around it:
>
> 1) An extension that will just make a request to the URL in question.
> Or heck, loading that URL in the browser. The URL is provided as
> part of about:cache and has been all along. The extension might
> be needed if we need to make absolutely sure we hit the cache only
> and don't ask the server (e.g. if the server uses a one-time auth
> token).
I've put code in place that makes this impossible. If the URL is copied
and pasted into the browser, the user receives the message, "An attempt
to download copyrighted material has been blocked. Error 101"
So in my case, the only way to get to the data is through a Firefox
feature that was intended for developers, not end users, whose
implications were not discussed beyond the statement "It's easy to add",
or through the non-end-user-oriented methods described below.
Shawn Wilsher
> Boris Zbarsky wrote:
>> Or heck, loading that URL in the browser. The URL is provided as
>> part of about:cache and has been all along. The extension might
>> be needed if we need to make absolutely sure we hit the cache only
>> and don't ask the server (e.g. if the server uses a one-time auth
>> token).
>
> I've put code in place that makes this impossible. If the URL is copied
> and pasted into the browser, the user receives the message, "An attempt
> to download copyrighted material has been blocked. Error 101"
something else? Chances are that an extension can easily make this look
like a legitimate request to you.
Honestly, it sounds like you are trying to secure your files through
obscurity...
Cheers,
Shawn
Boris Zbarsky
>> 1) An extension that will just make a request to the URL in question.
>> Or heck, loading that URL in the browser. The URL is provided as
>> part of about:cache and has been all along. The extension might
>> be needed if we need to make absolutely sure we hit the cache only
>> and don't ask the server (e.g. if the server uses a one-time auth
>> token).
>
> I've put code in place that makes this impossible. If the URL is copied
> and pasted into the browser, the user receives the message, "An attempt
> to download copyrighted material has been blocked. Error 101"
Like I said, one might have to use an extension that passes the "make
sure to load from the cache" flags down into necko. If it's in the
cache, an extension can always read it.
> So in my case, the only way to get to the data is through a Firefox
> feature that was intended for developers, not end users, whose
> implications were not discussed beyond the statement "It's easy to add",
> or through the non-end-user-oriented methods described below.
>
>> 2) A packet sniffer (if not doing https).
>> 3) A debugger (we provide symbols via the symbol server, so they don't
>> have to be particularly good with the debugger; just know which
>> data structures to poke).
>> 4) Any of the various memory-examining tools that just let you dump
>> chunks of RAM to disk.
Yep. about:cache is not user-oriented either, of course, as you note.
-Boris
Dave Corsello
> On 2/11/09 10:37 AM, Dave Corsello wrote:> Boris Zbarsky wrote:
> >> 1) An extension that will just make a request to the URL in question.
> >> Or heck, loading that URL in the browser. The URL is provided as
> >> part of about:cache and has been all along. The extension might
> >> be needed if we need to make absolutely sure we hit the cache only
> >> and don't ask the server (e.g. if the server uses a one-time auth
> >> token).
>
> > I've put code in place that makes this impossible. If the URL is copied
> > and pasted into the browser, the user receives the message, "An attempt
> > to download copyrighted material has been blocked. Error 101"
>
> And how did you do this? By checking for the referrer header or
> something else?
Something else.
> Chances are that an extension can easily make this look
> like a legitimate request to you.
Sorry, I don't understand.
> Honestly, it sounds like you are trying to secure your files through
> obscurity...
No, I want people to find my site and play my MP3's. I just don't
want them to be able to permanently download the files to the extent
that this is possible.
Dave Corsello
But packet sniffers, debuggers and memory-examining tools are not
browser features. Like I said early on, I don't expect to make it
impossible for users to get ahold of my files, but I don't expect the
browser to be the weak link.
The impact of this feature was not discussed thoroughly, and it was
intended for developers only. But it is, according to Daniel Veditz,
publicly advertised as a method of circumventing the kind of measures
that I've put in place.
Again, I ask, what value does it add for the end user? The answer is
simple: None. It was not intended for them.
I, as one kind of end user, an author, find this bug to be a problem.
It's a mistake that should be removed.
Boris Zbarsky
> The impact of this feature was not discussed thoroughly, and it was
> intended for developers only. But it is, according to Daniel Veditz,
> publicly advertised as a method of circumventing the kind of measures
> that I've put in place.
And if we took it out someone would just write an extension to dump the
data and advertise that.
The point is, users of our browser want access to the raw http data, for
various reasons, whether it be debugging their websites or something
else. We're not going to lock down access to such data (e.g. that would
break Firebug).
Given that, is this particular way of accessing the data worse than
others from your perspective, somehow?
-Boris
jsve...@gmail.com
> But packet sniffers, debuggers and memory-examining tools are not
> browser features. Like I said early on, I don't expect to make it
> impossible for users to get ahold of my files, but I don't expect the
> browser to be the weak link.
Honestly, if you're sending these files over the wire in the clear,
then the only weak link here is you. Adobe and other vendors offer
solutions specifically tailored to this situation that I'm sure they'd
be more than happy to provide or sell to you. Modern browsers also
offer you the APIs to roll your own streaming protocol and plugin if
you're so concerned users are going to be able to save your files from
a hex dump of all places...
HTTP is not aware of copyright or DRM and no client implementing it is
required to be either. The onus is solely on you to chose the best
protocol or format for the job. I as a content provider can understand
your plight, but I didn't just toss up the simplest solution and then
go after browsers, OSes and packet sniffers for simply allowing the
data I was transmitting to be shown. I created my own solution that is
end-to-end secure (I even allow browsers to cache the encrypted files
because it greatly improves user experience!) and any loss incurred by
ne'er do wells decompiling my SWF or capturing the output is
insignificant compared to the gains from having happy users.
I learned very quickly what does and doesn't work in this brave new
world of content distribution, but the basic takeaway is that there is
nothing you can do to stop the determined few from taking your
content. You do the best you can to prevent casual copying and any
easy means of doing so that opens up is no one's responsibility but
your own to close. Requesting others do the job you yourself should be
doing is simply ridiculous.
You either adapt or die in this business. I and countless others have
adapted. Will you?
- Johann S.
Justin Dolske
> But packet sniffers, debuggers and memory-examining tools are not
> browser features.
Yes, they are. Look at extensions such as Firebug, Live HTTP Headers,
Venkman, DOM Inspector, etc. They already exist, and there's demand for
more.
Justin
Dave Corsello
Hey Justin,
Thanks for your response. I do get what you're saying.
But I'm curious to know some things (Sorry, these are newbie questions...)
Currently, unless I'm mistaken, none of the browser plugins that have
been mentioned in this discussion offer the feature of displaying a hex
dump of the browser's memory cache (that is, unless necko is considered
a plugin). Is this because they don't have access to the data as necko
does? Or, is it just because no one has written one yet?
If the memory cache is available to any program that wants it, then are
there reasonabilty or practicality limits that would be prohibitive
towards developing a hex dump plugin that's similar to necko in terms of
its function, performance and usability? In other words, does necko's
"closeness" to Firefox offer advantages that make its existence vastly
more practical than some other hypothetical plugin that creates pretty
file dumps?
If the memory cache was not always universally accessible, when and why
did this change?
Thanks,
Dave
Dave Corsello
Like I've said before, I know that there are things like sniffers and
Flash decompilers. But I've thought of Firefox as being the weak link in
the _browsers_ arena, because it's the only one that I know of that has
a built-in feature that renders my current security measures worthless.
And it's frustrating, because except for the about:cache dumps, PHP
seems to offer me complete security, at least, again, within the
microcosm of web browsers. (I hope it's not naive to think that I'm
relatively secure with PHP as long as my server is healthy.)
You're absolutely right, though. If it's possible for me to put an
end-to-end security solution in place that doesn't detract from the
user's experience of my site, then I need to learn how to do that.
Serious food for thought, Johann. Thanks.
--Dave
Boris Zbarsky
> Currently, unless I'm mistaken, none of the browser plugins that have
> been mentioned in this discussion offer the feature of displaying a hex
> dump of the browser's memory cache (that is, unless necko is considered
> a plugin). Is this because they don't have access to the data as necko
> does? Or, is it just because no one has written one yet?
Firebug allows access to data that was fetched from the network. I
don't know the exact format it exposes it in, though.
> If the memory cache is available to any program that wants it,
The memory cache is available to any Firefox extension, not to any
program, though of course any program running on your computer can
insert itself into the running Firefox process if it wants to.
> then are there reasonabilty or practicality limits that would be
> prohibitive towards developing a hex dump plugin that's similar to
> necko in terms of its function, performance and usability?
necko is the networking library in Gecko; its primary purpose is to get
the data over the network. I'm not sure what you're asking here.
> In other words, does necko's "closeness" to Firefox offer advantages that make its existence vastly
> more practical than some other hypothetical plugin that creates pretty
> file dumps?
More practical for what purpose?
> If the memory cache was not always universally accessible, when and why
> did this change?
The memory cache could be read by any Firefox extension at any point in
time. Nothing here has changed.
-Boris
Dave Corsello
> Dave Corsello wrote:
>> Currently, unless I'm mistaken, none of the browser plugins that have
>> been mentioned in this discussion offer the feature of displaying a
>> hex dump of the browser's memory cache (that is, unless necko is
>> considered a plugin). Is this because they don't have access to the
>> data as necko does? Or, is it just because no one has written one yet?
>
> Firebug allows access to data that was fetched from the network. I
> don't know the exact format it exposes it in, though.
I just installed Firebug. Very cool extension. Under the Net tab, it
lists the files that have downloaded, and for each one it shows four
tabs: parameters, headers, post variables and response. It honors
no-store, no-cache and must-revalidate headers by displaying nothing
under the response tab.
This must be due to one of the following: 1) Firefox doesn't currently
make no-store data available to its extensions, 2) at the time when
Firebug was authored, Firefox didn't make no-store data available to its
extensions, 3) the author of Firebug didn't know that no-store data was
available, or 4) the author knew that the data was available and chose
not to display it.
Your clarification below eliminates possibilities 1 and 2, and option 3
seems very unlikely to me, especially in light of one of your earlier
posts, in which you stated that removing the hex dump of no-store data
from about:cache would break Firebug. So, I think that option 4 is most
likely, and this indicates to me that the author of Firebug acknowledges
that no-store headers are intended to keep data off of disk and out of
sight of the end user.
Would you mind clarifying something? Since it appears that Firebug does
nothing with no-store data, how would turning off the hex dump of that
specific kind of data in about:cache break Firebug?
>> If the memory cache is available to any program that wants it,
>
> The memory cache is available to any Firefox extension, not to any
> program, though of course any program running on your computer can
> insert itself into the running Firefox process if it wants to.
>
> > then are there reasonabilty or practicality limits that would be
> > prohibitive towards developing a hex dump plugin that's similar to
> > necko in terms of its function, performance and usability?
>
> necko is the networking library in Gecko; its primary purpose is to get
> the data over the network. I'm not sure what you're asking here.
Sorry about that. I was obviously unclear about what necko is.
>> In other words, does necko's "closeness" to Firefox offer advantages
>> that make its existence vastly more practical than some other
>> hypothetical plugin that creates pretty file dumps?
>
> More practical for what purpose?
Sorry again for being unclear. I was trying to find out if there's some
technical reason why none of the other plug-ins or extensions that have
been discussed in this thread display the contents of no-store files.
>> If the memory cache was not always universally accessible, when and
>> why did this change?
>
> The memory cache could be read by any Firefox extension at any point in
> time. Nothing here has changed.
Installing Firebug has made it even clearer to me that Mozilla's choice
to disregard no-store headers in Firefox's memory cache listing is, at
least, out of line with common practice.
Boris Zbarsky
> This must be due to one of the following: 1) Firefox doesn't currently
> make no-store data available to its extensions, 2) at the time when
> Firebug was authored, Firefox didn't make no-store data available to its
> extensions, 3) the author of Firebug didn't know that no-store data was
> available, or 4) the author knew that the data was available and chose
> not to display it.
My bet is on 3 (or more precisely on not knowing the right way to get
the data), but you might want to check with the author to make sure.
> Your clarification below eliminates possibilities 1 and 2, and option 3
> seems very unlikely to me, especially in light of one of your earlier
> posts, in which you stated that removing the hex dump of no-store data
> from about:cache would break Firebug.
Nowhere did I state this. Can you please point me to where I did?
I merely stated that Firebug has, as one of its stated goals, being able
to show you the data for all HTTP responses, and in particular for POST
responses. Last I checked they had this working using the new APIs in
Gecko 1.9.
In any case, if you want to know something about the decision-making
process of Firebug's authors, you should probably be contacting them
instead of trying to read tea-leaves...
> Would you mind clarifying something? Since it appears that Firebug does
> nothing with no-store data, how would turning off the hex dump of that
> specific kind of data in about:cache break Firebug?
It wouldn't, and at no point did I imply that it would. I just said
that writing an extension that would produce the data that you see in
that hex dump is quite doable. It could even not bother with the hex
dump and save directly to disk.
>> More practical for what purpose?
>
> Sorry again for being unclear. I was trying to find out if there's some
> technical reason why none of the other plug-ins or extensions that have
> been discussed in this thread display the contents of no-store files.
Almost certainly because if you try to naively load such a file the
cache won't hand it back. If you perform the load with certain flags
(e.g. "load only from cache", "don't revalidate", that sort of thing),
then you can get the data.
-Boris
Boris Zbarsky
> I just said that writing an extension that would produce the data that you see in
> that hex dump is quite doable. It could even not bother with the hex
> dump and save directly to disk.
And just to make this clearer... Your concern is that there is a way of
getting the data from the cache via about:cache and that this way is
being advertised in tutorials, so that your customers might learn about
it and use it to save the MP3s you stream to disk.
But if we took this functionality out of about:cache, all it would take
is one person writing an extension to expose the data (assuming that one
doesn't already exist; as I said, Firebug was working on this last I
checked) and then tutorials advertising it. At that point you're back
to square one, and about:cache is less useful for its primary audience...
-Boris
Robert Kaiser
> a built-in feature that renders my current security measures worthless.
Take care what you're saying here. This is not by any means about
security, it's about restriction in use of your content, which is
something completely different and in no way connected to security.
Robert Kaiser
John J. Barton
...
> I just installed Firebug. Very cool extension. Under the Net tab, it
> lists the files that have downloaded, and for each one it shows four
> tabs: parameters, headers, post variables and response. It honors
> no-store, no-cache and must-revalidate headers by displaying nothing
> under the response tab.
>
> This must be due to one of the following: 1) Firefox doesn't currently
> make no-store data available to its extensions, 2) at the time when
> Firebug was authored, Firefox didn't make no-store data available to its
> extensions, 3) the author of Firebug didn't know that no-store data was
> available, or 4) the author knew that the data was available and chose
> not to display it.
>
> Your clarification below eliminates possibilities 1 and 2, and option 3
> seems very unlikely to me, especially in light of one of your earlier
> posts, in which you stated that removing the hex dump of no-store data
> from about:cache would break Firebug. So, I think that option 4 is most
> likely, and this indicates to me that the author of Firebug acknowledges
> that no-store headers are intended to keep data off of disk and out of
> sight of the end user.
As one of the Firebug developers I appreciate all of the folks who have
contributed to this information thread. Now that we know about
'no-store' data we will investigate displaying it in the next Firebug
release.
jjb
Dave Corsello
>> 3 seems very unlikely to me, especially in light of one of your
>> earlier posts, in which you stated that removing the hex dump of
>> no-store data from about:cache would break Firebug.
>
> Nowhere did I state this. Can you please point me to where I did?
On 2/11/2009 at 3:47 you said:
"The point is, users of our browser want access to the raw http data,
for various reasons, whether it be debugging their websites or something
else. We're not going to lock down access to such data (e.g. that would
break Firebug)."
I see now that I misunderstood what you were saying. In my bug request,
I'm asking specifically that Firefox turn off the display of no-store
data. I misinterpreted your words "such data" to mean, specifically,
no-store data. Sorry, I certainly didn't mean to try to put words in
your mouth.
> In any case, if you want to know something about the decision-making
> process of Firebug's authors, you should probably be contacting them
> instead of trying to read tea-leaves...
I'm not so sure that I was trying to read tea leaves. If I hadn't
misinterpreted your post on the 2/11, my conclusion would have been
logical. But this is extraneous to the issue.
> [...] if you try to naively load such a file the
> cache won't hand it back. If you perform the load with certain flags
> (e.g. "load only from cache", "don't revalidate", that sort of thing),
> then you can get the data.
I see now that the essential clarification that you provided was when
you explained on 2/11, "The security standard in question covers
disclosure of information to entities other than the user, not to the
user." Yet, you continued to answer my questions, and for that I'm most
appreciative. The outcome of this discussion is not what I had hoped
for, but I did learn from it. Thanks.
Dave Corsello
Understood. Thanks.
Dave Corsello
> As one of the Firebug developers I appreciate all of the folks who have
> contributed to this information thread. Now that we know about
> 'no-store' data we will investigate displaying it in the next Firebug
> release.
>
> jjb
John, thanks for clarifying my "tea leaf" conclusions.
Daniel Veditz
> The impact of this feature was not discussed thoroughly, and it was
> intended for developers only. But it is, according to Daniel Veditz,
> publicly advertised as a method of circumventing the kind of measures
> that I've put in place.
It's not "publicly advertised", I had to dig for it. It was strictly in
the context of whether or not the bug should remain security-sensitive,
and I concluded that the people who would take advantage of that trick
already knew about it.
If someone's motivated enough to find the pages about using hexdumps in
Firefox 3 they're going to find a way to save the media, because that
information was on sites dedicated to ripping media along with all sorts
of tricks and tools that work on all sorts of browsers.