SST files being created nonstop - how to analyze them and determine cause? (potentially generated by IndexedDB actions)

[Oren Blasberg]

Hi folks,

My Chrome app seems to be causing a slow but steady generation of .sst files in my ~/.config/Default/Extension\ State folder, even when my app isn't actively (in the foreground) running.

Over time this amounts to a pretty significant leak that takes up more and more hard drive space as I use Chrome (but not necessarily while running my app).

(Linux x64, Version 23.0.1271.64 beta)

These files never seem to be removed from the disk, even after Chrome is closed or upon reboot, so I ended up accumulating some 16 gigs of these files after a month or so of use.

I believe these .sst files are generated by IndexedDB actions.

How can I analyze them to determine their contents? They seem to be just binary files... does anyone have any insight on what I could do to isolate the source of my problem?

Thanks!

Oren

[Antony Sargent]

Is your app one of our fancy new v2 packaged apps, or a hosted / "legacy" packaged app?

We use the Extension State file (which is a leveldb database, which we use for a few things in chrome such as the backing store for IndexedDb) for apps and extensions to store things like remembering event listeners and alarms registered for transient background pages, the size/position of apps v2 windows, context menu items, and declarativeWebRequest rules. It's possible this is a bug in leveldb or our usage of it (not garbage collecting properly, etc.). Or it might be that your use of extension/app features is causing a lot of writes into it. 

If you can reproduce this reliably, feel free to create a bug (preferably with as minimal a test case as possible) and we can continue discussion there.

Oren

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Alec Flett]

On Thu, Nov 8, 2012 at 11:59 AM, Oren Blasberg <or...@chromium.org> wrote:

Hi folks,

My Chrome app seems to be causing a slow but steady generation of .sst files in my ~/.config/Default/Extension\ State folder, even when my app isn't actively (in the foreground) running.

Over time this amounts to a pretty significant leak that takes up more and more hard drive space as I use Chrome (but not necessarily while running my app).

(Linux x64, Version 23.0.1271.64 beta)

These files never seem to be removed from the disk, even after Chrome is closed or upon reboot, so I ended up accumulating some 16 gigs of these files after a month or so of use.

I believe these .sst files are generated by IndexedDB actions.

To clarify: IndexedDB data is stored in a directory called "IndexedDB" - if it doesn't have that, it's not IDB. But .sst files are almost certainly LevelDB's SSTable files.

 

How can I analyze them to determine their contents? They seem to be just binary files... does anyone have any insight on what I could do to isolate the source of my problem?

if you want to get your hands dirty and dig around those files, I'd suggest http://code.google.com/p/py-leveldb/ - but I think Antony is right, there's something screwy with someone's use of LevelDB, as LevelDB does a pretty decent job recombining SST's and not allowing the directory to get bigger than 2-3x the size of the actual data being stored.

[Steve McKay]

This would be a "fancy new v2 packaged" app.

A couple followup questions.

• Are individual SST files associated with specific apps? If so, is there a way to determine ownership on the command line?
• Can a poorly behaved/written app cause the leak of these handlers, or should that be transparent to the app?

[Antony Sargent]

A couple followup questions.

• Are individual SST files associated with specific apps? If so, is there a way to determine ownership on the command line?

I don't think so. It looks like we just have a single database with all app/extension data in it (extension_id is used as a prefix on all keys). 

• Can a poorly behaved/written app cause the leak of these handlers, or should that be transparent to the app?

I think a poorly behaved/written app could cause us to write a lot of data (eg a large number of differently named context menu items or  window id's), but any sort of resource leaks are likely bugs in our code or usage of leveldb.

BTW, the relevant code here is:

chrome/browser/extensions/state_store.cc

chrome/browser/value_store/leveldb_value_store.cc

[Pirate Radio]

Same thing happened to me. 400,000 .sst files in the user appData hidden dir. locallow google,  Cant comment what really started it, but Earth 3d flyby crashed on me  a couple times, I went on to other things.  Noticed the my 86GB free had gone to 0.  I hadn't been using chrome,  Only google earth and google drive.  Google drive had some sync errors, that is what made me notice the space was gone.
Just fyi, maybe useless info. 
I un-installed chrome, drive, and earth, none erased the google appdata files.  Then deleted all the appdata locallow google files manually.  . 

[David Grogan]

Sorry, which directory were the sst files in? Do you know the full path?

--