Improving download manager customizability
Matthew Gertner
functionality (mainly download acceleration). I use my own download
implementation entirely and hook it into the Download Manager UI by
adding the download, never starting it (from Firefox's perspective)
and then calling OnProgressChange64 as appropriate to update the UI.
This works as expected, but with a number of issues. In particular,
there is no obvious clean way to implement pause/resume (since my
implementation need to catch the commands from the UI) and retry. I've
discussed this with Shawn, and we see 3 possible solutions:
1) Overlay the download manager UI and implement custom functions for
pauseDownload and resumeDownload. The new functions check whether this
is a custom download (could be indicated by having a specific
interface implemented by the cancelable member of the download) and,
if so, do something different. This is the solution I am currently
using. I have my own interface with pause() and resume() methods, and
I change pause/resumeDownload to see if it is implement by the
canceable property and, if so, to call the appropriate method. If not,
the default implementation is used.
Two problems with this approach are a) it's a bit of a hack and is
very sensitive to changes to the download manager code and b) it's not
obvious how to implement retry (i.e. after a download in canceled). I
can probably get downloads to retry using my code with the same
approach as for pause/resume, but there is still the issue that if I
cancel a download and restart Firefox, the download starts again
automatically. I'm thinking I may have to go in and write something to
the download DB to make sure Firefox knows not to autoresume.
2) Use the mRequest member of the download for pause/resume. The idea
here is to set the download type to something new (e.g.
DOWNLOAD_TYPE_CUSTOM) and, if this is set, use the mRequest member
which is already in the download (but never used for anything right
now). The new download type might help with getting retry to work
properly since the download manager could be coded not to autoresume
if it is set.
3) Add a method to the download manager that takes a fully implemented
nsIDownload and adds it to the UI without doing any special
processing. This seems like the cleanest solution. The interface is
there and has all the necessary methods. Why not use it? The biggest
problem with this is that the current download manager implementation
has a lot of dependencies on the specific nsDownload class so it would
be a significant change.
I'd appreciate any opinions about which approach to take.
Cheers,
Matt
Shawn Wilsher
the only viable option here is (1), but it's really crappy from an
extension developer's perspective.
The main problem with both (2) and (3) is that there is no way to handle
retry. Additionally, (3) doesn't work since nsIDownload has no
functions to pause, resume, or cancel a download on it, and those would
have to be added. That ends up making nsIDownloads manage themselves,
instead of the download manager managing them.
Here's what I'm thinking the right approach to do this properly goes
something like the following. Note that for coming up with this, I'm
basically thinking how we could implement metalink downloads in an addon.
(1) register a content handler [1] which implements nsIContentHandler.
This is optional, and is only required if you want to handle a specific
mime type from content.
(2) register a component (could be the same as (1)) that will handle the
pausing, resuming, canceling, and retrying of downloads. We'd have a
contract id system setup much like content handlers work:
@mozilla.org/downloader;?type={mime type}
We would need to come up with a new interface for this.
(3) when calling addDownload (done manually, or done with the content
handler in (1)), pass in DOWNLOAD_TYPE_CUSTOM, which will be a new
constant that we'll have to add.
I'm not sure if we should use the mRequest that is given to the download
manager now for pause (suspect), resume, and cancel for (2) instead of
the custom interface. The custom handler has the advantage of giving
more control to the custom downloader. We'd probably at least need to
pass in mRequest from nsDownload to the handler so it can properly init
itself.
How does this sound so far?
Cheers,
Shawn
Boris Zbarsky
> Here's what I'm thinking the right approach to do this properly goes
> something like the following. Note that for coming up with this, I'm
> basically thinking how we could implement metalink downloads in an addon.
> (1) register a content handler [1] which implements nsIContentHandler.
> This is optional, and is only required if you want to handle a specific
> mime type from content.
Note that what this will give you is just the URI, basically. In
particular, if you want the data you'll have to get it yourself. That
is, your only options with nsIContentHandler are:
a) The load keeps happening, but no one gets the data.
b) The load is canceled
c) URILoader looks for someone else to dispatch the load to.
Is that acceptable for this use case?
-Boris
Shawn Wilsher
> Note that what this will give you is just the URI, basically. In
> particular, if you want the data you'll have to get it yourself. That
> is, your only options with nsIContentHandler are:
>
> a) The load keeps happening, but no one gets the data.
> b) The load is canceled
> c) URILoader looks for someone else to dispatch the load to.
>
> Is that acceptable for this use case?
we'd want to register an nsIURIContentListener [1] and we'll get the
data via doContent via the nsIStreamListener we provide. My biggest
concern there is that it seems like our code would be called every
single time some request returns data (I'm not sure on this), so I can
see addons creating a perf hit by doing this. It would be desiralbe to
be able to have an nsIURIContentListener for only certain MIME types for
this.
Cheers,
Shawn
Boris Zbarsky
> No, that's not going to work, and I missed that. Instead, it looks like
> we'd want to register an nsIURIContentListener [1] and we'll get the
> data via doContent via the nsIStreamListener we provide. My biggest
> concern there is that it seems like our code would be called every
> single time some request returns data (I'm not sure on this)
Your CanHandleContent method will be called any time we need to dispatch
a MIME type that is being loaded in a docshell and Gecko itself can't
handle.
> I can see addons creating a perf hit by doing this. It would be desiralbe to
> be able to have an nsIURIContentListener for only certain MIME types for
> this.
I doubt the CanHandleContent call will take all that much time...
Unless you're expecting hundreds of listeners registered here or something?
-Boris
Matthew Gertner
downloader. You are correct that this seems the best (if not the only)
way to resume downloads on restart. Very elegant. I do think we might
want to consider adding pause and resume methods to nsIDownload. It
seems like the appropriate interface in terms of existing methods/
attributes, and the underlying implementation nsDownload already
implements those methods. So this would be an extension of my proposal
(3). I tend to agree that nsIRequest probably isn't the right
interface since the semantics aren't correct and it has stuff we don't
need.
I'm not entirely clear on what you want to achieve with the content
listener. If you want to make it possible to selectively download
using different download implementations, couldn't this be better
accomplished by overlaying the unknownContentType UI? I would like to
refactor the JS code for that dialog a bit but it should be possible
to have a small number of hooks in there that extension developers
could overlay.
Matt
Anthony Bryan
> I've done some thinking about this over the weekend, and I realized that the
> only viable option here is (1), but it's really crappy from an extension
> developer's perspective.
>
> The main problem with both (2) and (3) is that there is no way to handle
> retry. Additionally, (3) doesn't work since nsIDownload has no functions to
> pause, resume, or cancel a download on it, and those would have to be added.
> That ends up making nsIDownloads manage themselves, instead of the download
> manager managing them.
>
> something like the following. Note that for coming up with this, I'm
> basically thinking how we could implement metalink downloads in an addon.
I know the DownThemAll! addon doesn't use Firefox's download manager
UI, but can some of it be re-used for pausing & canceling?
http://bugs.code.downthemall.net/trac/browser
I've also been wondering if some of DTA's metalink code could be used
in the FlashGot addon, just to do something simple like pass one URL
to either Firefox's native download manager or external download
managers. People wouldn't be able to use any of the extra useful info
in the metalinks (checksums, mirrors) but at least the download would
still proceed as normal.
http://bugs.code.downthemall.net/trac/browser/releases/1.0.3/chrome/content/dta/manager
--
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
)) Easier, More Reliable, Self Healing Downloads
Matthew Gertner
mentioned in my original post by using their own UI. I think that for
this type of extension it is vastly preferable from a user experience
standpoint to use the standard Download Manager UI.
Matt
On Nov 25, 8:11 pm, "Anthony Bryan" <anthonybr...@gmail.com> wrote:
> On Mon, Nov 24, 2008 at 4:10 PM, Shawn Wilsher <sdwi...@mozilla.com> wrote:
> > I've done some thinking about this over the weekend, and I realized that the
> > only viable option here is (1), but it's really crappy from an extension
> > developer's perspective.
>
> > The main problem with both (2) and (3) is that there is no way to handle
> > retry. Additionally, (3) doesn't work since nsIDownload has no functions to
> > pause, resume, or cancel a download on it, and those would have to be added.
> > That ends up making nsIDownloads manage themselves, instead of the download
> > manager managing them.
>
> > Here's what I'm thinking the right approach to do this properly goes
> > something like the following. Note that for coming up with this, I'm
> > basically thinking how we could implement metalink downloads in an addon.
>
> I know the DownThemAll! addon doesn't use Firefox's download manager
> UI, but can some of it be re-used for pausing & canceling?
>
> http://bugs.code.downthemall.net/trac/browser
>
> I've also been wondering if some of DTA's metalink code could be used
> in the FlashGot addon, just to do something simple like pass one URL
> to either Firefox's native download manager or external download
> managers. People wouldn't be able to use any of the extra useful info
> in the metalinks (checksums, mirrors) but at least the download would
> still proceed as normal.
>
> http://bugs.code.downthemall.net/trac/browser/releases/1.0.3/chrome/c...
Shawn Wilsher
On 11/25/08 11:32 AM, Matthew Gertner wrote:
> I really like the idea of registering a component for the custom
> downloader. You are correct that this seems the best (if not the only)
> way to resume downloads on restart. Very elegant. I do think we might
> want to consider adding pause and resume methods to nsIDownload. It
> seems like the appropriate interface in terms of existing methods/
> attributes, and the underlying implementation nsDownload already
> implements those methods. So this would be an extension of my proposal
> (3). I tend to agree that nsIRequest probably isn't the right
> interface since the semantics aren't correct and it has stuff we don't
> need.
Still not sold on adding pause and resume to nsIDownload. nsDownload
admittedly evolved without me thinking too much about the overall
design, which is something I've been wanting to take a look at for a
while now (someone clone me please?).
> I'm not entirely clear on what you want to achieve with the content
> listener. If you want to make it possible to selectively download
> using different download implementations, couldn't this be better
> accomplished by overlaying the unknownContentType UI? I would like to
> refactor the JS code for that dialog a bit but it should be possible
> to have a small number of hooks in there that extension developers
> could overlay.
I think we are trying to accomplish the same thing as the end result.
The idea is that the content listener, for lack of a better word,
magically updates the unkownContentType dialog so nobody has to update
it with an overlay manually. It also means things like Open With...
would work.
As for the unknownContentType UI...
We intended to merge it with the external protocol handling UI back in
Firefox 3, but we never got around to it. It'd be nice to get that done
(and to get better tested code for it).
Again, sorry for the delay in my response!
Cheers,
Shawn