Out of Memory Error Crashes, Overwrites Existing TW File

[Scott Simmons]

Yikes!  This is the first time I've encountered this error, and it freaked me out a bit.  =:-[

 

Early this morning I encountered the dreaded "Out of memory" error dialog in my browser (Maxthon 2) when I tried to save one of my TiddlyWikis:

 

http://tiddlywiki.secret-hq.com/group/images/twError_OutOfMemory.png

 

According to Process Explorer, I wasn't using much memory.  That made sense:  I was browing in four tabs (three of them local TWs) and not running any other apps, except maybe Notepad++.

 

When I dismissed the dialog, I got this dialog (which I'd never seen before):

 

http://tiddlywiki.secret-hq.com/group/images/twError_FailedToSave.png

 

This was a bit distressing but not especially concerning.  I have my TWs set to autosave, so I simply grabbed the contents of the tiddler I'd been working on and threw them into a .TXT file.  Then I attempted another save and got the usual ActiveX approval dialog:

 

http://tiddlywiki.secret-hq.com/group/images/twError_NotPossibleToSaveChanges.png (plus the yellow IE bar to approve ActiveX)

 

... and then the "Are you sure you want to leave this page?" dialog (since I still hadn't saved that tiddler I'd been working on when the problem occurred):

 

http://tiddlywiki.secret-hq.com/group/images/twError_LeaveOrStay-Saving.png

 

I chose to leave the page, knowing I could paste in the tiddler content after getting over the save hurdle — and something unusual happen.  My TW refreshed to an empty page.  Completely empty -- blank, white.  I glanced over at the folder containing my TW and immediately spotted the file size was "0 bytes."  Sure enough, there wasn't a single character inside the TW -- no source code, not even a stray space.  :_(

 

Thankfully, I back up my TWs to a network drive about once a day, so I didn't lose the entire file -- though I did lose a few hours' worth of work.  (That hasn't happened to me in a while, and I have to admit: It stung!)

 

After that, I got curious and tried to save the other open TWs, which behaving similarly.  I was able to bring over the back-up of my main TW from the network drive and successfull save a few changes in it (including re-enabling automatic back-ups!) -- but after several minutes, I got the same out of memory error and lost that copy of TW on attempting to save it.

 

After scouring the cache for a copy of my lost TW (to no avail), I closed the browser, dumped the cache, and restarted -- and haven't had any troubles since.

 

—

 

I'm curious what happened, though.  Has anyone else encountered this behavior?  It's my first time in nearly a year and a half of using TiddlyWiki every day.

 

I'd have thought my real-time changes were made in a temp copy of TW and then saved (over the existing TW in my directory) when I click on "Save Changes" or (in my case) click "done" in a tiddler and invoke the autosave.  When I encountered the "failed to save" error, I expected TW had simply failed to save and that my last-saved version was still sitting safely in its folder.  Instead, it looks like TW overwrote itself with an empty file.

 

Most importantly:  Any ideas on how to safeguard against this?  As noted, I've re-enabled automatic back-ups and am using Tyler Akins's BackupOptionsPlugin (http://rumkin.com/tools/tiddlywiki/#BackupOptionsPlugin) to limit the number of back-ups it's making.  If TW overwrote itself with an empty HTML file, would it not also similarly overwrite the back-ups?

 

Could my having multiple TWs open at the same time have contributed to the issue?  It's not uncommon for me to have multiple TWs open at once (running similar sets of plugins), but it is a little unusual for me to have multiple tiddlers open in edit view at the same time across multiple TWs.  (I think I had one tiddler each open in edit view in two of the three TWs when the problem occurred.  I don't think any of them had the same tiddler name.)

 

It may have been a freak accident, but I'd love to hear from anyone with any insight.  :)

[Yakov]

Hi Scott.

Did you make any tiddler really big before you saved TW? There is
(was?) also a bug which appeared in Opera [1], the whole content of
wiki was smashed, but it was not emptied. I'm not sure whether these
two are connected.

[1] https://groups.google.com/group/tiddlywiki/browse_thread/thread/628ae4abdc7a95ec?hl=ru

[Scott Simmons]

Thanks for the link, Yakov!

 

At the time of the crash, I was working on a large-ish tiddler.  I looked to see if I still had the .TXT file I used to back it up around so I could post exactly how large — but, alas, I don't.  As best I can recall, it was around 800 to 1,000 words at the time.  (I'm not sure how many bytes.)  [That's based on the final version being around 1,600 words and a sketchy memory of where I was when the crash occurred.]

 

Going over the circumstances in my mind, I think I remember having two instances of the same TiddlyWiki open during that browser session.  If I'm right, the second instance had been closed for several minutes (at least), and the remaining one had gone through at least two or three different saves (without any refreshing) — but (if I'm remembering correctly) the presence of conflicting temp files may have played some role in my crash.

 

Or at least that's the best theory I've been able to come up with — sketchy though it is.  :)

[Yakov]

Well, you can track how many symbols are needed. In the link I cited
you can see my number. How to count:

1) make a line with nine symbols and press enter (that would be ten
symbols)
2) ctrl+a (select all), {ctrl+c (copy), ctrl+v (paste), say 10 times},
that would be 100 symbols
3) repeat 2) twice -- that would be 10000 symbols
4) copy and paste all 4 times, so that there'll be 40000 symbols and
try saving -- I expect it would be enough for crash

Than you can
- count more precisely by repeating process (make 30000) and than add
less
- try with other wikis opened (I didn't try such a thing)

I think this will help tracking this. That time Jeremy created a
ticket [1], but it looks like investigation didn't go deep. Perhaps if
these two issues are connected, this new one will clarify it a bit.

[1] http://trac.tiddlywiki.org/ticket/1276

[Scott Simmons]

Good strategy, Yakov.  I created one string ("123456789" + \n) and copy-pasted it over and over to generate an easy-to-track raw number of characters.  Since I use the WordCountMacro by Simon Baird, the line break made it easy to know how many 9-letter "words" and 10-letter blocks of characters were being saved in the tiddler.

 

Unfortunately, I couldn't replicate the crash.  I got to up to as many as 100,000 10-character blocks with no errors (http://tiddlywiki.secret-hq.com/group/images/tw1Mcharacters.png) -- although, as you might expect, the TW's performance bogged down a bit.

[Yakov]

Yeap, that's bad news. A nonreplicable bug is close to nontrackable.
Though, you can also try to play with two opened TWs (or open and
close one, as you did that time). Anyway, remember the link of this
thread so you can add more observations later.

[Scott Simmons]

Bookmarked.  I'll definitely come back and resurrect this thread if I discover anything useful.

 

Thanks for the help, Yakov.  I'll try to keep your thread on my radar too, in case I encounter anything that looks like your bug over here ... .

[Scott Simmons]

Happened to me again today, for what it's worth.

 

This time, only one TW had been open during the entire browser session, and I'd just finished saving a rather large tiddler.  I clicked into another tiddler, made a minor edit, and then clicked "done" — at which point I got the "Out of Memory" dialog, saw that the backup had been saved, and realized both my TW and its backup had been wiped out to 0 bytes.  I tried recovering the TW from the cache (0 bytes) and using "View Source" to grab it before it was too late (empty file), but neither worked.

 

Sadly, I've been on the run this week and haven't made any manual back-ups of this TW (my main notebook), so I'll be spending Friday night trying to reconstruct everything I've lost since Monday morning.  :_(

 

—

 

The upshot is that I've just discovered that Tyler Rumkin updated his life-saving BackupOptionsPlugin (http://rumkin.com/tools/tiddlywiki/#BackupOptionsPlugin) earlier this year to allow for rolling back-ups — meaning that I can now have two alternating back-ups created on save, preventing the heart-breaking loss I just experienced without having to have 200 back-ups trailing behind me.  :)

[PMario]

With win7, it should be possible to use compressed folders. Just
compress the backups folder and you are fine, even with 1000 backups.
-m

[Yakov]

Hi Scott,

can you specify what browser do you use? You mentioned Maxthon 2 but
then also some ActiveX dialogues. Is this Maxthon 2 based on IE? I
wonder if it is recognized as IE and vise versa, if it is, whether all
the functions used in TW do what they are supposed to do when it is IE.