nsIDownloadManager replaced by Downloads.jsm

[Paolo Amadini]

Hello,

if you are maintaining an add-on or a Mozilla product that interacts
with downloads, you should look into updating your code to use the new
Downloads.jsm module instead of nsIDownloadManager as soon as possible.

While other Mozilla products may migrate at different times, Firefox
for Desktop will do so starting from version 26, meaning that add-ons
that use methods of nsIDownloadManager will no longer be compatible
unless updated. Firefox 26 will reach the Beta channel on October 29th,
and will go to release on December 10th, 2013.

*Overview*

We have been working on this new module over the past few months, with
the goal of eliminating any temporary unresponsiveness that could be
observed when downloads are started, as well as making the browser more
responsive in general while there are downloads in progress.

To make this possible, we removed all access to the "downloads.sqlite"
database, replacing it with an in-memory representation of the state
of current downloads. In Firefox for Desktop, the history of past
downloads is handled separately, using the "places.sqlite" database.

*Changes*

The new API is fully asynchronous and works somewhat differently from
the old one. It has the advantage of being designed for JavaScript from
the start, and is much simpler than the old XPCOM API. In particular,
listing and handling current downloads may be done using JavaScript
objects, without any special code for database access.

The complete documentation of the module can be found here:

https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Downloads.jsm

We may still update existing methods to address specific needs or
new requirements, but the general mechanisms will remain the same.

*Testing*

A new about:config preference named "browser.download.useJSTransfer"
enables the browser and the Downloads Panel to use the Downloads.jsm
module instead of nsIDownloadManager as the back-end. The browser must
be restarted for the preference to take effect.

Support for this preference will be available in Nightly today or
tomorrow. This means that it will be ready for testing in the Aurora
channel starting from version 25, on August 8th.

In the Firefox 26 release train, nsIDownloadManager will not be used
anymore. The preference will be removed and there will be no way to
revert to the old system that caused potential performance issues.
We will finally be able to remove a lot of front-end code that is
complex to maintain and only needed for backwards compatibility.

*Feedback*

The version of the module in Firefox 25 is still in development, and
while the interface is complete, some features like restoring downloads
after a restart and the related prompts are not implemented yet. The
remaining work is tracked with dependencies of bug 847863:

https://bugzilla.mozilla.org/showdependencytree.cgi?id=847863&hide_resolved=1

If you notice any unexpected behavior with the new preference that is
not already listed there, feel free to file a new bug and mark it as
blocking bug 847863. For any other question or need, feel free to
reply to this thread in the relevant list (extensions or platform),
or contact myself directly. Any relevant information emerging from
the discussion will be summarized in a new project update.

Cheers,
Paolo

[Neil]

Paolo Amadini wrote:

>The complete documentation of the module can be found here:
>
>https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Downloads.jsm
>
>

... plus you'll need all the subpages for the other objects of course.

Do I have to watch the download lists globally in order to find out
about new downloads (e.g. from content-disposition: attachment)?

Do I have to emulate cleanUp by calling remove() on all the completed
downloads?

Do I have to maintain the active download count myself? (I don't think
that would be a problem.)

Does Downloads.createDownload(aFailedDownload) replace
aFailedDownload.retry()?

How do I watch for changes to a download if I don't know whether or not
it's part of a download list? (But given the first answer, it looks like
I have to watch the list whether I like it or not, and then filter on
the download in which I'm interested.)

P.S. I hope the misspelling of succeeded is a documentation error.

--
Warning: May contain traces of nuts.

[David E. Ross]

Will this break the Preserve Download Modification Timestamp extension
at <https://addons.mozilla.org/en-US/seamonkey/addon/222517/>?

--
David E. Ross
<http://www.rossde.com/>

Concerned about someone (e.g., the government)
snooping into your E-mail? Use PGP.
See my <http://www.rossde.com/PGP/>

[Paolo Amadini]

On 03/08/2013 17.26, Neil wrote:
> Do I have to watch the download lists globally in order to find out
> about new downloads (e.g. from content-disposition: attachment)?

Yes, you should add a view to the public and private download lists.

> Do I have to emulate cleanUp by calling remove() on all the completed
> downloads?

Good point. While you could do that for now, we'll likely add a method
to the DownloadList object, since it's an existing use case. Just filed
bug 901262.

> Do I have to maintain the active download count myself? (I don't think
> that would be a problem.)

Yes, as there are different counts that consumers may want to track.

> Does Downloads.createDownload(aFailedDownload) replace
> aFailedDownload.retry()?

No, it's replaced by aFailedDownload.start().

> How do I watch for changes to a download if I don't know whether or not
> it's part of a download list? (But given the first answer, it looks like
> I have to watch the list whether I like it or not, and then filter on
> the download in which I'm interested.)

If a download is not part of a DownloadList, it means it doesn't appear
in the user interface and is functional to some other part of the code,
and you cannot watch it (similar to the current situation when using
nsIWebBrowserPersist without linking it to an nsITransfer).

> P.S. I hope the misspelling of succeeded is a documentation error.

Fixed, thanks!

I'll use some of these questions as the base for a more detailed
migration documentation.

Paolo

[Paolo Amadini]

On 03/08/2013 19.27, David E. Ross wrote:
> Will this break the Preserve Download Modification Timestamp extension
> at <https://addons.mozilla.org/en-US/seamonkey/addon/222517/>?

Quite likely. In the new API we'll probably need to add explicit
support for this use case, something like a hook of some kind to
DownloadCopySaver and DownloadLegacySaver.

My suggestion to the add-on author would be to file a bug under the
Extension Compatibility component, blocking the more general bug 825588
for the Downloads API, with as much details as possible, and we'll get
to it as soon as the rest of the feature set is complete.

https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Extension%20Compatibility

Cheers,
Paolo

[Neil]

Paolo Amadini wrote:

>On 03/08/2013 17.26, Neil wrote:
>
>
>>Do I have to watch the download lists globally in order to find out about new downloads (e.g. from content-disposition: attachment)?
>>
>>
>Yes, you should add a view to the public and private download lists.
>
>

So nsIDownloadManagerUI is going away too?

>>Does Downloads.createDownload(aFailedDownload) replace aFailedDownload.retry()?
>>
>>
>No, it's replaced by aFailedDownload.start().
>
>

Sorry, I'd misread the note there. Thanks for clearing that up.

>I'll use some of these questions as the base for a more detailed migration documentation.
>
>

Or you could just edit the MDN pages for the interfaces that are being
removed and link to the replacements for all the relevant methods...

[Paolo Amadini]

On 04/08/2013 1.42, Neil wrote:
> So nsIDownloadManagerUI is going away too?

The Downloads back-end will not call any method of nsIDownloadManagerUI
anymore, new downloads should be detected using views. Technically, the
interface will still be there for a short time until we refactor the
code that opens the user interface (the Library window or the Downloads
tab in Firefox for Desktop).

> Or you could just edit the MDN pages for the interfaces that are being
> removed and link to the replacements for all the relevant methods...

Good idea, will do that also.

Cheers,
Paolo

[Robert Kaiser]

Paolo Amadini schrieb:

> The Downloads back-end will not call any method of nsIDownloadManagerUI
> anymore

So I cannot implement an alternative download manager any more?

Robert Kaiser

[gary...@gmail.com]

I flipped on 'browser.download.useJSTransfer' and downloaded some zip files from Fx inbound and they all were zero byte files. This was done in safe mode. Hope this is a WIP ;-)

[Paolo Amadini]

You can for sure :-) Anything we're changing and has a valid use case
will still be implemented in new ways, hopefully simpler to hook into.

Paolo

[Paolo Amadini]

On 06/08/2013 22.56, gary...@gmail.com wrote:
> I flipped on 'browser.download.useJSTransfer' and downloaded some zip files from Fx inbound

Thanks a lot for testing!

> and they all were zero byte files. This was done in safe mode. Hope this is a WIP ;-)

This is probably bug 901563, should be fixed in Nightly.

And yes, this is a work in progress :-)

Cheers,
Paolo

[Neil]

Paolo Amadini wrote:

>A new about:config preference named "browser.download.useJSTransfer" enables the browser and the Downloads Panel to use the Downloads.jsm module instead of nsIDownloadManager as the back-end. The browser must be restarted for the preference to take effect.
>
>Support for this preference will be available in Nightly today or tomorrow. This means that it will be ready for testing in the Aurora channel starting from version 25, on August 8th.
>
>In the Firefox 26 release train, nsIDownloadManager will not be used anymore. The preference will be removed and there will be no way to revert to the old system that caused potential performance issues. We will finally be able to remove a lot of front-end code that is complex to maintain and only needed for backwards compatibility.
>
>

I notice that this preference exists entirely in browser front-end code.
So how are other Mozilla products expected to test Downloads.jsm?

[Paolo Amadini]

On 16/08/2013 10.22, Neil wrote:
> Paolo Amadini wrote:
>> A new about:config preference named "browser.download.useJSTransfer"
>> enables the browser and the Downloads Panel to use the Downloads.jsm
>> module instead of nsIDownloadManager as the back-end. The browser must
>> be restarted for the preference to take effect.
>>

> I notice that this preference exists entirely in browser front-end code.
> So how are other Mozilla products expected to test Downloads.jsm?

The API itself can be used without changing the preference at all, if
you add items to the DownloadList using its methods. To switch downloads
started from content to use the new API, you should register its
nsITransfer component during "profile-after-change":

const cid = Components.ID("{1b4c85df-cbdd-4bb6-b04e-613caece083c}");
Components.manager.QueryInterface(Ci.nsIComponentRegistrar)
.registerFactory(cid, "", "@mozilla.org/transfer;1",
null);

http://mxr.mozilla.org/mozilla-central/source/browser/components/downloads/src/DownloadsStartup.js#96

At some point, we'll add a build-time switch to enable this component.

Cheers,
Paolo

[sam foster]

> if you are maintaining an add-on or a Mozilla product that interacts
> with downloads, you should look into updating your code to use the new
> Downloads.jsm module instead of nsIDownloadManager as soon as possible.

We're currently using nsIDownloadManager in Firefox for Metro and will be wanting to adopt Downloads.jsm as soon as possible (likely after our initial preview release). However, although Downloads.jsm itself is importable from toolkit, DownloadsCommon.jsm - which implements a lot of the goodies around it - is not. Looking though it I see only a couple of methods and specific-to-desktop-UI references that would prevent it moving to toolkit. Would you be interested in refactoring a little to enable it to move? I happy to help where I can.

Drop into #windev or ping me (sfoster) in #fx-team when you can to discuss.

/Sam

[teramako]

Hi

I want to notify to the user when the download is finished.

on nsIDownloadManager, I write like following:

Services.downloads.addListener({
onDownloadStateChange: function (state, download) {
if (download.state === Services.downloads.DOWNLOAD_FINISHED) {
openPopup(.....);
}
},
...
});

on Download.jsm, how to register a observer if can do ?
I read Download.jsm and related files, but couldn't find.

Best regards.

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform


--
teramako

[Paolo Amadini]

On 03/09/2013 18.08, teramako wrote:
> I want to notify to the user when the download is finished.
>
> on nsIDownloadManager, I write like following:
>
> Services.downloads.addListener({
> onDownloadStateChange: function (state, download) {
> if (download.state === Services.downloads.DOWNLOAD_FINISHED) {
> openPopup(.....);
> }
> },
> ...
> });
>
> on Download.jsm, how to register a observer if can do ?
> I read Download.jsm and related files, but couldn't find.

You need to add a view on the public (and private, if needed) download
lists (getPublicDownloadList, getPrivateDownloadList).

If you're just interested on completion, in the onDownloadAdded
callback of the object you provided to addView, you can register a
whenSucceeded callback on the download:

aDownload.whenSucceeded().then(function () { ... });

For other states, you need to handle them in the onDownloadChanged
callback. You should track your own previous state for the changes
you're interested into, if applicable.

Cheers,
Paolo

[Paolo Amadini]

On 28/08/2013 19.17, sam foster wrote:
> However, although Downloads.jsm itself is importable from toolkit, DownloadsCommon.jsm - which implements a lot of the goodies around it - is not. Looking though it I see only a couple of methods and specific-to-desktop-UI references that would prevent it moving to toolkit. Would you be interested in refactoring a little to enable it to move?

Most of the code and objects from DownloadsCommon.jsm will be completely
removed - ideally, up to the point where we have only Desktop front-end
code there. For example, what is currently a DownloadDataItem will
become a plain Download object.

We want to implement features like the DownloadSummary directly in
Downloads.jsm, as they make sense for all platforms:

Downloads.getPublicDownloadSummary().then(summary => summary.addView({
onSummaryChanged: function () {
setUIProgressElement(summary.progress);
},
}));

> I happy to help where I can.
> Drop into #windev or ping me (sfoster) in #fx-team when you can to discuss.

I'd _really_ welcome your help here! We can talk about the needs of the
Metro browser and what we can each do to help.

Cheers,
Paolo

[Eric Shepherd]

On 2013-08-03 12:21:57 +0000, Paolo Amadini said:

> The complete documentation of the module can be found here:
>
> https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Downloads.jsm

Paulo: On behalf of the documentation team, I want to thank you for
documenting this! We always appreciate it when the people who best know
new features and technologies are active in the documentation process,
and creating the docs like this is the absolute most effective way to
do that.

That you also went for it and documented Promise.jsm while you were at
it is a wonderful bonus.

You win the coveted Hero of the MDN Team award!

--
Eric Shepherd
Developer Documentation Lead
Mozilla
Blog: http://www.bitstampede.com/
Twitter: @sheppy

[andy.p...@gmail.com]

On Saturday, August 3, 2013 8:21:57 AM UTC-4, Paolo Amadini wrote:

I am trying to use the new module. It works fine in terms of downloading the file but how can I send the download to Firefox download manager so that user can see the progress same as normal downloads?

[jruby2bl...@gmail.com]

On Saturday, August 3, 2013 7:21:57 AM UTC-5, Paolo Amadini wrote:
> Hello,

I am nobody. But, I want file timestamps to be preserved when I download files. I guess I will use wget until this gets fixed.

Now I have no reason to use Firefox.

I was using this until todays upgrade to FF 26:

"Preserve Download Modification Timestamp" addon

I do hope someone is still working on fixing this.

https://bugzilla.mozilla.org/show_bug.cgi?id=918188

Thanks,
Jeff