March 9, 2014 at 23:27Firefox: download bar improvements
We will talk about the features of the new download panel in Firefox and the Download Panel Tweaker extension , which eliminates some of the unwanted features.
In particular, about the most controversial, in my opinion, innovation, because of which the completed downloads disappear from the list (although they remain visible in the corresponding section of the "library") - it just so happened that the time improvement took the most tofix .
The result looks like this (this is a "compact" option from the settings , "very compact" will save a little more space):
And here is how it was originally .
There will also be quite a lot of code examples (and then where without details?).
In general, if the limitation of the visible number of downloads could be configured using the built-in means, the extension might not have been, because the sizes are easily adjustable using userChrome.css or the Stylish extension .
In addition, it’s not possible (?) To display the download speed using styles alone - it is in the tooltip, but the pseudo-classes :: before and :: afterthey do not always work in XUL (apparently, these are the limitations of anonymous nodes ), so this does not help:
But the changes will only work if they were made when the browser started, that is, before the download bar was initialized.
Therefore, further searches resulted in the resource: //app/modules/DownloadsCommon.jsm file and in the DownloadsCommon.getSummary () function :
And what is actually happening in the DownloadsSummaryData constructor :
Everything is simple here - just do it like this:
That is, the limit for the already created regular and private instances of DownloadsSummaryData is adjusted .
The most difficult part is that the download list itself, in accordance with the new settings, will not be redrawn.
But here I had a head start: during the development of the Private Tab extension (which, incidentally, was also an article ), the exact same question arose, because I had to change the list of downloads from regular to private when switching tabs (and there were a lot of them experiments of varying degrees of success).
But, as usual, there were still some tricks: Firefox 28 removed the function for cleaning the download panel, completely (it used to be for switching between the old and the new download engine). So I had to write an analog - good, it's simple.
The result can be viewed at the link above or in the extension code .
And in the extension there is a feature: if you just do
, then a memory leak will occur, because the objects will be created in the scope of the extension, so commonly used constructor functions come in handy:
In this case, window is the window in which DownloadsView is located (i.e. DownloadsView === window.DownloadsView ).
As a result, when closing the browser, the Services.downloads.cleanUp () function was called .
(By the way, there are already traces of the new engine in the same place - checking the browser.download.useJSTransfer setting value .)
I had to redefine Services.downloads entirely, because it
(see resource: //gre/modules/Services.jsm ), and the properties of services cannot be changed:
If simplified, the result is as follows:
That is, our fake object contains its implementation of the cleanUp property function and inherits everything else from the present Services.downloads .
Well, from the fake function Services.downloads.cleanUp (), you can check the call stack, and if it is the same DownloadsStartup.js , then do nothing. Although not very reliable, it is easily restored when the extension is disabled. You can even complicate the task and add checks in case any other extension makes a similar wrapper.
resource: //gre/modules/DownloadStore.jsm
Then in resource: //gre/modules/DownloadIntegration.jsm the onsaveitem method is overridden:
Thus, there is a function DownloadStore.prototype.onsaveitem () , which always allows saving and is redefined for each specific implementation of new DownloadStore () .
(Looking ahead, I add that, unfortunately, not all comments are equally useful and truthful.)
Moreover, the source code of DownloadIntegration.jsm has an interesting conditional comment:
However, if you replace the DownloadIntegration.shouldPersistDownload () function (and don’t forget about DownloadIntegration._store.onsaveitem () in case the downloads have already been initialized) by analogy with the code from this conditional comment, a whole bunch of unpleasant surprises will pop up - it’s kind of documented, but it only works correctly in its original form, when completed downloads are not saved.
Firstly , after a restart, all completed downloads will show a zero size and browser start time (although everything is correctly saved in downloads.json ).
Invalid date caused by code from resource: //app/modules/DownloadsCommon.jsm :
Then this time does not change when calling DownloadsDataItem.prototype.updateFromJSDownload () (Firefox 26-27) and DownloadsDataItem.prototype.updateFromDownload () (Firefox 28+).
Fortunately, you can wrap around this function and make the necessary edits with each call .
Secondly , the code from the conditional comment about MOZ_B2G does not actually work: completed downloads will never be deleted at all. And I could not find other suitable conditional comments about MOZ_B2G (apparently, it doesn’t work there either), though I didn’t really try - it’s easy to fix everything there .
Then from here and thirdly: a large list of completed downloads cannot be loaded - the stack overflows (error " too much recursion "). Moreover, the problem can already be obtained on a list of 35 downloads.
Apparently, the used implementation of promises ( promises ) is not able to work correctly with actually synchronous calls.
For example, if in DownloadStore.prototype.load () ( resource: //gre/modules/DownloadStore.jsm ) a little tweak and replace in
last line on
, then the stack overflow will not go anywhere, but will happen with a little more saved downloads.
Fourth , somewhere else there are optimizations (?) , So deleting completed downloads through the interface may not start saving data to downloads.json (by default they are not saved), so after rebooting everything will remain as it was.
Fortunately, you can use a simple hack: add the preservation of current downloads when the browser shuts down.
Fifthly , when the browser starts, if there is something in the download panel (even if it is a suspended or even completed download) there will be a notification about the start of the download.
But we already have a wrapper forDownloadsDataItem.prototype.updateFromDownload () , so this is easily fixed .
Well, the downloads.json reading code , unfortunately, had to be rewritten . From what my opinion about promises has not improved at all - any technology should be applied wisely and only where it is really needed (and not shoved anywhere without just because it is fashionable and modern).
And there’s a strange problem with the list of downloads: if the dates are almost the same, the sorting will be inverted (the new ones will be at the bottom, not at the top), but if you open the download panel when the browser starts, before the delayed initialization starts, the order will be correct ( but only in this window).
In addition, problems with a large list of downloads are not limited to reading ... Although even with reading there is one more problem: after each addition of a new download, an interface update is called, synchronously. When restoring a saved list, an addition is also made, followed by notification of all those concerned. Here I had to make a couple more corrections .
Here the built-in profiler (Shift + F5) provided invaluable help , without it it would be almost impossible to figure out the reasons for the freeze - the logic is too complicated there (and the call stack is horrific).
Well, in addition to reading with a full list, at least the list cleaning function also works, so there too it can fall into a stack overflow if the list is large enough. This has not yet been fixed. But, in principle, this is not critical: if something should not be saved, there is always a private mode, and piecewise deletion (and the added cleaning item for all downloads in general) works.
What is most interesting, there are much fewer problems with incomplete downloads - their recovery for some reason will not go into recursion, even if there are quite a lot of them (I checked for 150 pieces). Apparently, this is due to the OS.File API allocated to a separate stream for working with files (and for paused downloads, the presence of a file is checked), because of it, inarticulate errors of the form
In addition, after the corrections made, the automatic download started working fine, but if you try to open the download panel right after launching the browser, it will work out some other code and, if there are a lot of downloads, it may freeze.
PS The new version of the extension is still being tested, “line item: 4 of 17”. Initially, I planned to wait (and the text, in terms of editing - and it was written more than a week ago - this is only good), but there is a limit to everything.
PPSI tried to write so as not to impose on the reader my opinion about the quality of the new code in Firefox, I hope I succeeded, and let everyone draw their own conclusions. However, I repeat, in its original form, no special problems arise.
In particular, about the most controversial, in my opinion, innovation, because of which the completed downloads disappear from the list (although they remain visible in the corresponding section of the "library") - it just so happened that the time improvement took the most to
The result looks like this (this is a "compact" option from the settings , "very compact" will save a little more space):
And here is how it was originally .
There will also be quite a lot of code examples (and then where without details?).
Instead of a preface, or long live compact!
For starters, all the elements of the download bar are huge by default! Firstly, I don’t have a touch monitor - thanks, but I can get to the right place anyway. And secondly, there are only three points, not more. That is, the place is spent, but something is of little use.In general, if the limitation of the visible number of downloads could be configured using the built-in means, the extension might not have been, because the sizes are easily adjustable using userChrome.css or the Stylish extension .
In addition, it’s not possible (?) To display the download speed using styles alone - it is in the tooltip, but the pseudo-classes :: before and :: afterthey do not always work in XUL (apparently, these are the limitations of anonymous nodes ), so this does not help:
.downloadDetails[tooltiptext]::after {
content: attr(tooltiptext) !important;
}
Increase Visible Downloads
As a result, the code responsible for the number of downloads in the panel was found quite quickly in the chrome file : //browser/content/downloads/downloads.js :const DownloadsView = {
//////////////////////////////////////////////////////////////////////////////
//// Functions handling download items in the list
/**
* Maximum number of items shown by the list at any given time.
*/
kItemCountLimit: 3,
But the changes will only work if they were made when the browser started, that is, before the download bar was initialized.
Therefore, further searches resulted in the resource: //app/modules/DownloadsCommon.jsm file and in the DownloadsCommon.getSummary () function :
/**
* Returns a reference to the DownloadsSummaryData singleton - creating one
* in the process if one hasn't been instantiated yet.
*
* @param aWindow
* The browser window which owns the download button.
* @param aNumToExclude
* The number of items on the top of the downloads list to exclude
* from the summary.
*/
getSummary: function DC_getSummary(aWindow, aNumToExclude)
{
if (PrivateBrowsingUtils.isWindowPrivate(aWindow)) {
if (this._privateSummary) {
return this._privateSummary;
}
return this._privateSummary = new DownloadsSummaryData(true, aNumToExclude);
} else {
if (this._summary) {
return this._summary;
}
return this._summary = new DownloadsSummaryData(false, aNumToExclude);
}
},
And what is actually happening in the DownloadsSummaryData constructor :
/**
* DownloadsSummaryData is a view for DownloadsData that produces a summary
* of all downloads after a certain exclusion point aNumToExclude. For example,
* if there were 5 downloads in progress, and a DownloadsSummaryData was
* constructed with aNumToExclude equal to 3, then that DownloadsSummaryData
* would produce a summary of the last 2 downloads.
*
* @param aIsPrivate
* True if the browser window which owns the download button is a private
* window.
* @param aNumToExclude
* The number of items to exclude from the summary, starting from the
* top of the list.
*/
function DownloadsSummaryData(aIsPrivate, aNumToExclude) {
this._numToExclude = aNumToExclude;
Everything is simple here - just do it like this:
var itemCountLimit = 5; // Для примера увеличим количество видимых загрузок с 3 до 5
if(DownloadsCommon._privateSummary)
DownloadsCommon._privateSummary._numToExclude = itemCountLimit;
if(DownloadsCommon._summary)
DownloadsCommon._summary._numToExclude = itemCountLimit;
That is, the limit for the already created regular and private instances of DownloadsSummaryData is adjusted .
The most difficult part is that the download list itself, in accordance with the new settings, will not be redrawn.
But here I had a head start: during the development of the Private Tab extension (which, incidentally, was also an article ), the exact same question arose, because I had to change the list of downloads from regular to private when switching tabs (and there were a lot of them experiments of varying degrees of success).
But, as usual, there were still some tricks: Firefox 28 removed the function for cleaning the download panel, completely (it used to be for switching between the old and the new download engine). So I had to write an analog - good, it's simple.
The result can be viewed at the link above or in the extension code .
And in the extension there is a feature: if you just do
DownloadsView._viewItems = {};
DownloadsView._dataItems = [];
, then a memory leak will occur, because the objects will be created in the scope of the extension, so commonly used constructor functions come in handy:
DownloadsView._viewItems = new window.Object();
DownloadsView._dataItems = new window.Array();
In this case, window is the window in which DownloadsView is located (i.e. DownloadsView === window.DownloadsView ).
Save downloads on exit
This turned out to be the most difficult, but not so much due to the fact that the internal implementation of downloads has changed in Firefox 26, but because of the many related problems. Many of these difficulties are reflected in the corresponding issue , but it’s better about everything in order.Old versions and download cleanup
In Firefox 25 and older, downloads were forcibly cleared ( resource: //app/components/DownloadsStartup.js ): case "browser-lastwindow-close-granted":
// When using the panel interface, downloads that are already completed
// should be removed when the last full browser window is closed. This
// event is invoked only if the application is not shutting down yet.
// If the Download Manager service is not initialized, we don't want to
// initialize it just to clean up completed downloads, because they can
// be present only in case there was a browser crash or restart.
if (this._downloadsServiceInitialized &&
!DownloadsCommon.useToolkitUI) {
Services.downloads.cleanUp();
}
break;
...
case "quit-application":
...
// When using the panel interface, downloads that are already completed
// should be removed when quitting the application.
if (!DownloadsCommon.useToolkitUI && aData != "restart") {
this._cleanupOnShutdown = true;
}
break;
case "profile-change-teardown":
// If we need to clean up, we must do it synchronously after all the
// "quit-application" listeners are invoked, so that the Download
// Manager service has a chance to pause or cancel in-progress downloads
// before we remove completed downloads from the list. Note that, since
// "quit-application" was invoked, we've already exited Private Browsing
// Mode, thus we are always working on the disk database.
if (this._cleanupOnShutdown) {
Services.downloads.cleanUp();
}
As a result, when closing the browser, the Services.downloads.cleanUp () function was called .
(By the way, there are already traces of the new engine in the same place - checking the browser.download.useJSTransfer setting value .)
I had to redefine Services.downloads entirely, because it
Components.classes["@mozilla.org/download-manager;1"]
.getService(Components.interfaces.nsIDownloadManager);
(see resource: //gre/modules/Services.jsm ), and the properties of services cannot be changed:
Object.defineProperty(Services.downloads, "cleanUp", {
value: function() {},
enumerable: true,
configurable: true,
writable: true
}); // Exception: can't redefine non-configurable property 'cleanUp'
If simplified, the result is as follows:
var downloads = Services.downloads;
var downloadsWrapper = {
__proto__: downloads,
cleanUp: function() { ... }
};
this.setProperty(Services, "downloads", downloadsWrapper);
That is, our fake object contains its implementation of the cleanUp property function and inherits everything else from the present Services.downloads .
Well, from the fake function Services.downloads.cleanUp (), you can check the call stack, and if it is the same DownloadsStartup.js , then do nothing. Although not very reliable, it is easily restored when the extension is disabled. You can even complicate the task and add checks in case any other extension makes a similar wrapper.
New versions, selective saving of downloads and many hacks
Then, in Firefox 26, they turned on by default a new download engine and transferred temporary downloads (namely, they are displayed in the download panel) to the downloads.json file in the profile. In addition, instead of cleaning, filtering was done while saving:resource: //gre/modules/DownloadStore.jsm
this.DownloadStore.prototype = {
...
/**
* This function is called with a Download object as its first argument, and
* should return true if the item should be saved.
*/
onsaveitem: () => true,
...
/**
* Saves persistent downloads from the list to the file.
*
* If an error occurs, the previous file is not deleted.
*
* @return {Promise}
* @resolves When the operation finished successfully.
* @rejects JavaScript exception.
*/
save: function DS_save()
{
return Task.spawn(function task_DS_save() {
let downloads = yield this.list.getAll();
...
for (let download of downloads) {
try {
if (!this.onsaveitem(download)) {
continue;
}
Then in resource: //gre/modules/DownloadIntegration.jsm the onsaveitem method is overridden:
this.DownloadIntegration = {
...
initializePublicDownloadList: function(aList) {
return Task.spawn(function task_DI_initializePublicDownloadList() {
...
this._store.onsaveitem = this.shouldPersistDownload.bind(this);
...
/**
* Determines if a Download object from the list of persistent downloads
* should be saved into a file, so that it can be restored across sessions.
*
* This function allows filtering out downloads that the host application is
* not interested in persisting across sessions, for example downloads that
* finished successfully.
*
* @param aDownload
* The Download object to be inspected. This is originally taken from
* the global DownloadList object for downloads that were not started
* from a private browsing window. The item may have been removed
* from the list since the save operation started, though in this case
* the save operation will be repeated later.
*
* @return True to save the download, false otherwise.
*/
shouldPersistDownload: function (aDownload)
{
// In the default implementation, we save all the downloads currently in
// progress, as well as stopped downloads for which we retained partially
// downloaded data. Stopped downloads for which we don't need to track the
// presence of a ".part" file are only retained in the browser history.
// On b2g, we keep a few days of history.
//@line 319 "c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\toolkit\components\jsdownloads\src\DownloadIntegration.jsm"
return aDownload.hasPartialData || !aDownload.stopped;
//@line 321 "c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\toolkit\components\jsdownloads\src\DownloadIntegration.jsm"
},
Thus, there is a function DownloadStore.prototype.onsaveitem () , which always allows saving and is redefined for each specific implementation of new DownloadStore () .
(Looking ahead, I add that, unfortunately, not all comments are equally useful and truthful.)
Moreover, the source code of DownloadIntegration.jsm has an interesting conditional comment:
shouldPersistDownload: function (aDownload)
{
// In the default implementation, we save all the downloads currently in
// progress, as well as stopped downloads for which we retained partially
// downloaded data. Stopped downloads for which we don't need to track the
// presence of a ".part" file are only retained in the browser history.
// On b2g, we keep a few days of history.
#ifdef MOZ_B2G
let maxTime = Date.now() -
Services.prefs.getIntPref("dom.downloads.max_retention_days") * 24 * 60 * 60 * 1000;
return (aDownload.startTime > maxTime) ||
aDownload.hasPartialData ||
!aDownload.stopped;
#else
return aDownload.hasPartialData || !aDownload.stopped;
#endif
},
However, if you replace the DownloadIntegration.shouldPersistDownload () function (and don’t forget about DownloadIntegration._store.onsaveitem () in case the downloads have already been initialized) by analogy with the code from this conditional comment, a whole bunch of unpleasant surprises will pop up - it’s kind of documented, but it only works correctly in its original form, when completed downloads are not saved.
Firstly , after a restart, all completed downloads will show a zero size and browser start time (although everything is correctly saved in downloads.json ).
Invalid date caused by code from resource: //app/modules/DownloadsCommon.jsm :
/**
* Represents a single item in the list of downloads.
*
* The endTime property is initialized to the current date and time.
*
* @param aDownload
* The Download object with the current state.
*/
function DownloadsDataItem(aDownload)
{
this._download = aDownload;
...
this.endTime = Date.now();
this.updateFromDownload();
}
Then this time does not change when calling DownloadsDataItem.prototype.updateFromJSDownload () (Firefox 26-27) and DownloadsDataItem.prototype.updateFromDownload () (Firefox 28+).
Fortunately, you can wrap around this function and make the necessary edits with each call .
Secondly , the code from the conditional comment about MOZ_B2G does not actually work: completed downloads will never be deleted at all. And I could not find other suitable conditional comments about MOZ_B2G (apparently, it doesn’t work there either), though I didn’t really try - it’s easy to fix everything there .
Then from here and thirdly: a large list of completed downloads cannot be loaded - the stack overflows (error " too much recursion "). Moreover, the problem can already be obtained on a list of 35 downloads.
Apparently, the used implementation of promises ( promises ) is not able to work correctly with actually synchronous calls.
For example, if in DownloadStore.prototype.load () ( resource: //gre/modules/DownloadStore.jsm ) a little tweak and replace in
/**
* Loads persistent downloads from the file to the list.
*
* @return {Promise}
* @resolves When the operation finished successfully.
* @rejects JavaScript exception.
*/
load: function DS_load()
{
return Task.spawn(function task_DS_load() {
let bytes;
try {
bytes = yield OS.File.read(this.path);
} catch (ex if ex instanceof OS.File.Error && ex.becauseNoSuchFile) {
// If the file does not exist, there are no downloads to load.
return;
}
let storeData = JSON.parse(gTextDecoder.decode(bytes));
// Create live downloads based on the static snapshot.
for (let downloadData of storeData.list) {
try {
let download = yield Downloads.createDownload(downloadData);
last line on
let {Download} = Components.utils.import("resource://gre/modules/DownloadCore.jsm", {});
let download = Download.fromSerializable(downloadData);
, then the stack overflow will not go anywhere, but will happen with a little more saved downloads.
Fourth , somewhere else there are optimizations (?) , So deleting completed downloads through the interface may not start saving data to downloads.json (by default they are not saved), so after rebooting everything will remain as it was.
Fortunately, you can use a simple hack: add the preservation of current downloads when the browser shuts down.
Fifthly , when the browser starts, if there is something in the download panel (even if it is a suspended or even completed download) there will be a notification about the start of the download.
But we already have a wrapper forDownloadsDataItem.prototype.updateFromDownload () , so this is easily fixed .
Well, the downloads.json reading code , unfortunately, had to be rewritten . From what my opinion about promises has not improved at all - any technology should be applied wisely and only where it is really needed (and not shoved anywhere without just because it is fashionable and modern).
And there’s a strange problem with the list of downloads: if the dates are almost the same, the sorting will be inverted (the new ones will be at the bottom, not at the top), but if you open the download panel when the browser starts, before the delayed initialization starts, the order will be correct ( but only in this window).
In addition, problems with a large list of downloads are not limited to reading ... Although even with reading there is one more problem: after each addition of a new download, an interface update is called, synchronously. When restoring a saved list, an addition is also made, followed by notification of all those concerned. Here I had to make a couple more corrections .
Here the built-in profiler (Shift + F5) provided invaluable help , without it it would be almost impossible to figure out the reasons for the freeze - the logic is too complicated there (and the call stack is horrific).
Well, in addition to reading with a full list, at least the list cleaning function also works, so there too it can fall into a stack overflow if the list is large enough. This has not yet been fixed. But, in principle, this is not critical: if something should not be saved, there is always a private mode, and piecewise deletion (and the added cleaning item for all downloads in general) works.
What is most interesting, there are much fewer problems with incomplete downloads - their recovery for some reason will not go into recursion, even if there are quite a lot of them (I checked for 150 pieces). Apparently, this is due to the OS.File API allocated to a separate stream for working with files (and for paused downloads, the presence of a file is checked), because of it, inarticulate errors of the form
Error: Win error 2 during operation open (Cannot find the file specified.(this is if you delete the file paused by the download)
)
Source file: resource: //gre/modules/commonjs/sdk/core/promise.js
Line: 133
In addition, after the corrections made, the automatic download started working fine, but if you try to open the download panel right after launching the browser, it will work out some other code and, if there are a lot of downloads, it may freeze.
PS The new version of the extension is still being tested, “line item: 4 of 17”. Initially, I planned to wait (and the text, in terms of editing - and it was written more than a week ago - this is only good), but there is a limit to everything.
PPSI tried to write so as not to impose on the reader my opinion about the quality of the new code in Firefox, I hope I succeeded, and let everyone draw their own conclusions. However, I repeat, in its original form, no special problems arise.