Performance Issues (Host I/O Causes Unresponsive UI?)

[idfubar (Rishi Ugersain Chopra)]

I have been using Shelve with a configuration that makes its
functionality very similar to that of the old Slogger plug-in (i.e.
one large XML log file and subfolders for each day and further each
unique time of access). The problem which I've encountered is that
Auto-Save configuration results in a browser which freezes the host
(an iMac) and causes the UI to become unresponsive: simply visiting a
page (e.g. 'economist.com') and issuing a few 'Open In A New Tab'
commands causes the problem. Although the content is being saved as
expected ("Web page, complete (HTML)") the performance degradation is
simply atrocious (i.e. the browser becomes unresponsive as the tabs
load and even Finder navigation slows). Is the observed behavior as
expected or could there be some other problem? The only thing which I
could think of that might account for what I've observed would be code
that is attempting to read the log file (i.e. scanning to the end of
the XML file) before writing to the file... though performance when
the plug-in was first installed was also impacted (i.e. degraded) and
is reminiscent of what I observed when running Slogger on Windows XP
and Linux hosts.

PS: I have decided to ask before digging into the code as I'd like to
know if I'm doing something which is obviously wrong (i.e. not as
expected) before asking additional questions; my apologies if another
means is preferred.

PPS: I can share some pictures of Disk Activity from Activity Monitor
(or even a video) if it would be of any help...

[lith]

Hi,

I currently have no time to look into this.

Since shelve's main use case is to manually save a document, performance wasn't a top priority. I never found shelve's performance annoying though.

Does the problem persist when

• when changing the file format to plain text,
• when using a vanilla profile with no other addons installed, or
  
• when switching of writing to the log file?

though performance when
the plug-in was first installed was also impacted (i.e. degraded) and
is reminiscent of what I observed when running Slogger on Windows XP
and Linux hosts.

Could you please explain/describe your observations in greater detail.

[lith]

Am Donnerstag, 28. Juli 2011 08:44:11 UTC+2 schrieb Rishi Chopra:

The problem which I've encountered is that
Auto-Save configuration results in a browser which freezes the host
(an iMac) and causes the UI to become unresponsive: simply visiting a
page (e.g. 'economist.com') and issuing a few 'Open In A New Tab'
commands causes the problem.

How big is your log file? I tested your scenario with ff 10.0.2 & shelve 1.25. I used a shelf with a regexp that always matches "." and with "Check this RegExp while browsing" selected. I opened about 20 pages in tabs from the economist simultaneously, which caused the fan of my laptop to make some noise but didn't cause any freezes or ff becoming unreponsive. I didn't use the ShelveLogger template but I don't see how this should make any difference.

Did you test this with no other addon than shelve enabled? Did you test it with a fresh profile? This problem also occurs under linux with no virus scanner or any other process that could try to scan html files on accesses/writes running?

Regards,
Tom

[lith]

Am Donnerstag, 8. März 2012 05:32:15 UTC+1 schrieb lith:

Am Donnerstag, 28. Juli 2011 08:44:11 UTC+2 schrieb Rishi Chopra:

The problem which I've encountered is that
Auto-Save configuration results in a browser which freezes the host
(an iMac) and causes the UI to become unresponsive: simply visiting a
page (e.g. 'economist.com') and issuing a few 'Open In A New Tab'
commands causes the problem.

How big is your log file? I tested your scenario with ff 10.0.2 & shelve 1.25

I appended to a file as log file that is 1.8M big.

Regards,
Tom

[Rishi Chopra]

A brief update: I made the suggested configuration change, i.e. disabling the "After program start save documents automatically" option on the 'Auto-Save' tab (and replaced the functionality by selecting "Save a document to this shelf if the URL matches this RegExp" under the 'Auto-Save' tab of the 'Shelve: Edit Shelf' menu of the "ShelveLogger" shelf) and... it worked!  The performance is markedly improved and the system no longer becomes unresponsive after a few tabs are loaded.  While I should note that performance is also affected (albeit to a lesser degree) by writing to a smaller log file I am confident that the suggestion has made the difference as no other extensions were enabled, the scenario was tested with a fresh profile, and the performance has been checked on systems which are only running a default installation of the operating system and with no additional background scanning processes.

PS:  I confirmed the performance on a Solaris 11 instance running in a VM with 1GB of RAM allocated to it (on my iMac host); I would be willing to check using other configurations but my intuition tells me the suggestion is sufficient for addressing what I observed.

PPS:  Is there something in the codepath that can be changed so that the suggestion doesn't make such a big difference?  I apologize as I haven't looked at the code yet (I assumed soliciting the author's expertise might be a more logical path to solution) but I would nevertheless assume that documenting the suggestion in a more accessible place is wise.