PSA: Pref JSON files are no longer pretty-printed, becoming 30% smaller

187 views
Skip to first unread message

Ryo Hashimoto

unread,
Mar 4, 2015, 11:13:48 PM3/4/15
to chromium-dev
After r319211, pref JSON files are no longer pretty-printed with human-friendly line breaks and indents.
This should result in about 30% size reduction of those files and it should contribute to slightly faster startup/shutdown.

If you like these files to be pretty-printed for development, you can use the new commandline switch "--pretty-print-prefs".

- Ryo

Daniel Bratell

unread,
Mar 5, 2015, 10:05:47 AM3/5/15
to chromium-dev, Ryo Hashimoto, hash...@google.com
On Thu, 05 Mar 2015 05:13:10 +0100, Ryo Hashimoto <hash...@chromium.org> wrote:

After r319211, pref JSON files are no longer pretty-printed with human-friendly line breaks and indents.
This should result in about 30% size reduction of those files and it should contribute to slightly faster startup/shutdown.

I like!

/Daniel

Mattias Nissler

unread,
Mar 5, 2015, 10:58:54 AM3/5/15
to bra...@opera.com, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
Have you measured timing impact? Do we have any reason to believe that size of pref files is a problem in practice?

IMHO, pref files being less readable to devs is a factor to include in the equation that I haven't seen considered in the discussion on the bug.

-----------------------------------------------
Google Germany GmbH
Dienerstr. 12
80331 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Graham Law, Christine Elizabeth Flores

Steuernummer: 48/725/00206 
Umsatzsteueridentifikationsnummer: DE813741370

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

Gabriel Charette

unread,
Mar 5, 2015, 11:56:39 AM3/5/15
to mnis...@chromium.org, bra...@opera.com, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 10:58 AM Mattias Nissler <mnis...@chromium.org> wrote:
Have you measured timing impact? Do we have any reason to believe that size of pref files is a problem in practice?

IMHO, pref files being less readable to devs is a factor to include in the equation that I haven't seen considered in the discussion on the bug.

This was my main pushback and one of the major discussion topics on the CL/bug and, as mentioned in the original post, we added --pretty-print-prefs for devs who want to pretty print (and would leave that option there even if we eventually switch to another format than JSON for the default case IMO).

We could even consider a chrome://flags if adding the command-line flag all the time is deemed too cumbersome, but we shouldn't incur ~30% file size increase for whitespace just to make it slightly easier for devs IMO.

If this 30% size decrease proves worthwhile performance wise, I'd like to investigate other storage formats than JSON to reduce it further.


Cheers,
Gab

Alexei Svitkine

unread,
Mar 5, 2015, 12:02:05 PM3/5/15
to Gabriel Charette, mnis...@chromium.org, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
Note that if you just want to view the contents for debugging, rather than edit them, there's also chrome://local-state debug url which I believe will show the contents pretty printed. Also works on mobile and other platforms where you can't get at the file easily. (If there's interest, a similar url can be added for profile prefs.)

Gabriel Charette

unread,
Mar 5, 2015, 12:03:23 PM3/5/15
to Alexei Svitkine, Gabriel Charette, mnis...@chromium.org, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
Oh sweet, didn't know about that one, yea that could be a very nice solution for prefs.

What do others think?

Brett Wilson

unread,
Mar 5, 2015, 12:31:18 PM3/5/15
to Gabriel Charette, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
This sounds like a great improvement. I remember being quite surprised when I opened my prefs file and found it to be meticulously formatted despite not being something users should edit.

IMO I don't think we need to worry about making it more convenient for developers, it seems like finding data in this file is rare and easy enough if you need to.

Brett

Mike Frysinger

unread,
Mar 5, 2015, 12:41:01 PM3/5/15
to Alexei Svitkine, Gabriel Charette, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
python -mjson.tool < foo.json | less
-mike

Benjamin Kalman

unread,
Mar 5, 2015, 12:44:00 PM3/5/15
to vap...@chromium.org, Alexei Svitkine, Gabriel Charette, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
I'd find it interesting to measure JSON vs IPC pickling here both for storage and parsing efficiency. Two concerns:
  • I've found during both Extension development and debugging Extension failures (e.g. random user complaining "My AdBlock isn't working!") that being able to read Preferences is invaluable. If there were a chrome: URL to dump a readable version of Preferences that would go a long way to mitigating it - unless the bug is causing a crash on startup :-)
  • I also worry about how Preferences modifying malware (and users) would react/adapt. I wouldn't expect it go well. Proteus team would be important to consult.
Btw, I once tried switching Extension sendMessage from using window.JSON + std::string, to v8 API + base::Value pickling, and the pickling solution was a significant regression (on the order of 100% slower) and allocated a lot of memory - but I think much of that is the v8 API's (and/or our use of it) fault.

Mattias Nissler

unread,
Mar 5, 2015, 1:17:11 PM3/5/15
to Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Gabriel Charette, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

-----------------------------------------------
Google Germany GmbH
Dienerstr. 12
80331 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Graham Law, Christine Elizabeth Flores

Steuernummer: 48/725/00206 
Umsatzsteueridentifikationsnummer: DE813741370

Gabriel Charette

unread,
Mar 5, 2015, 1:40:20 PM3/5/15
to Mattias Nissler, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Gabriel Charette, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 1:16 PM Mattias Nissler <mnis...@chromium.org> wrote:
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

I don't think it's reasonable to 1.5X a file (i.e. undoing a 33% reduction is a 50% increase) with whitespace just a for few debugging scenarios.

Especially since you're just one --pretty-print-prefs step away from getting back to a debuggable state (and even closer if we add a chrome://preferences URL as Alexei proposed above) -- assuming the problem is not a startup problem (a drawing problem is fine because the prefs would still get pretty printed back to disk in the background even if Chrome can't render) and worst case the one line JSON is still searchable in notepad so it's not that hard.
 

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

These are the factors that were weighed, the measurement that was collected was the fact that we expect this to drop preferences size by roughly 33% and that nothing in my opinion justifies having 33% whitespace in a file opened by a very slim margin of users (especially with the addition of workarounds for those who really need it).

Asking the question backwards, if we weren't pretty printing today, would you say it's reasonable to enable pretty-printing (and 1.5X all existing files) for everyone all-the-time just because it helps in a few edge case debugging scenarios?

IMO this is a clear yes (I had been considering this for a while, but hadn't realized it was a 33% reduction or I would have done it a while ago), performance improvements coming out of this (if any) will dictate IMO whether we should explore other storage types than JSON to reduce size/serializing complexity further.

Nico Weber

unread,
Mar 5, 2015, 1:44:47 PM3/5/15
to Gabriel Charette, Mattias Nissler, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 10:39 AM, Gabriel Charette <g...@chromium.org> wrote:


On Thu, Mar 5, 2015 at 1:16 PM Mattias Nissler <mnis...@chromium.org> wrote:
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

I don't think it's reasonable to 1.5X a file (i.e. undoing a 33% reduction is a 50% increase) with whitespace just a for few debugging scenarios.

Especially since you're just one --pretty-print-prefs step away from getting back to a debuggable state (and even closer if we add a chrome://preferences URL as Alexei proposed above) -- assuming the problem is not a startup problem (a drawing problem is fine because the prefs would still get pretty printed back to disk in the background even if Chrome can't render) and worst case the one line JSON is still searchable in notepad so it's not that hard.
 

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

These are the factors that were weighed, the measurement that was collected was the fact that we expect this to drop preferences size by roughly 33% and that nothing in my opinion justifies having 33% whitespace in a file opened by a very slim margin of users (especially with the addition of workarounds for those who really need it).

Asking the question backwards, if we weren't pretty printing today, would you say it's reasonable to enable pretty-printing (and 1.5X all existing files) for everyone all-the-time just because it helps in a few edge case debugging scenarios?

IMO this is a clear yes (I had been considering this for a while, but hadn't realized it was a 33% reduction or I would have done it a while ago), performance improvements coming out of this (if any) will dictate IMO whether we should explore other storage types than JSON to reduce size/serializing complexity further.

I agree with all of this, but to put the fabled 33% in context: According to https://code.google.com/p/chromium/issues/detail?id=463366, these 33% we're talking about are less than 100kB of disk space total. With a warm disk, that probably needs close to no time to read and skipping whitespace while parsing should be very fast too. With a cold disk and a very fragmented file system, it might make a difference though. Looking forward to perf numbers.

Nico

Brett Wilson

unread,
Mar 5, 2015, 1:47:34 PM3/5/15
to Gabriel Charette, Mattias Nissler, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 10:39 AM, Gabriel Charette <g...@chromium.org> wrote:

On Thu, Mar 5, 2015 at 1:16 PM Mattias Nissler <mnis...@chromium.org> wrote:
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

I don't think it's reasonable to 1.5X a file (i.e. undoing a 33% reduction is a 50% increase) with whitespace just a for few debugging scenarios.

Especially since you're just one --pretty-print-prefs step away from getting back to a debuggable state (and even closer if we add a chrome://preferences URL as Alexei proposed above) -- assuming the problem is not a startup problem (a drawing problem is fine because the prefs would still get pretty printed back to disk in the background even if Chrome can't render) and worst case the one line JSON is still searchable in notepad so it's not that hard.
 

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

These are the factors that were weighed, the measurement that was collected was the fact that we expect this to drop preferences size by roughly 33% and that nothing in my opinion justifies having 33% whitespace in a file opened by a very slim margin of users (especially with the addition of workarounds for those who really need it).

Asking the question backwards, if we weren't pretty printing today, would you say it's reasonable to enable pretty-printing (and 1.5X all existing files) for everyone all-the-time just because it helps in a few edge case debugging scenarios?

IMO this is a clear yes (I had been considering this for a while, but hadn't realized it was a 33% reduction or I would have done it a while ago), performance improvements coming out of this (if any) will dictate IMO whether we should explore other storage types than JSON to reduce size/serializing complexity further.

+1, personally I wouldn't even bother to do perf numbers. Every such operation is almost free on a developer machine, it can be very different on a laptop with a 5K RPM drive and slow memory.

Brett 

Peter Kasting

unread,
Mar 5, 2015, 1:52:37 PM3/5/15
to Gabriel Charette, Alexei Svitkine, mnis...@chromium.org, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 9:02 AM, Gabriel Charette <g...@chromium.org> wrote:
Oh sweet, didn't know about that one, yea that could be a very nice solution for prefs.

What do others think?

On Thu, Mar 5, 2015 at 12:00 PM Alexei Svitkine <asvi...@chromium.org> wrote:
Note that if you just want to view the contents for debugging, rather than edit them, there's also chrome://local-state debug url which I believe will show the contents pretty printed. Also works on mobile and other platforms where you can't get at the file easily. (If there's interest, a similar url can be added for profile prefs.)

Putting on my command-line-flags-owner hat: we shouldn't have long-lived command-line flags to do things in general.  Flags should be temporary things.  So if at all possible, please use/implement something like what Alexei proposes and remove the flag you've added.

(There are standalone utilities to pretty-print JSON files, aren't there?  Some combination of telling people to use those, plus the facilities described above, seems sufficient to obviate the need for a flag to me.)

PK 

Mattias Nissler

unread,
Mar 5, 2015, 2:01:39 PM3/5/15
to Brett Wilson, Gabriel Charette, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 10:46 AM, Brett Wilson <bre...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 10:39 AM, Gabriel Charette <g...@chromium.org> wrote:

On Thu, Mar 5, 2015 at 1:16 PM Mattias Nissler <mnis...@chromium.org> wrote:
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

I don't think it's reasonable to 1.5X a file (i.e. undoing a 33% reduction is a 50% increase) with whitespace just a for few debugging scenarios.

Especially since you're just one --pretty-print-prefs step away from getting back to a debuggable state (and even closer if we add a chrome://preferences URL as Alexei proposed above) -- assuming the problem is not a startup problem (a drawing problem is fine because the prefs would still get pretty printed back to disk in the background even if Chrome can't render) and worst case the one line JSON is still searchable in notepad so it's not that hard.
 

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

These are the factors that were weighed, the measurement that was collected was the fact that we expect this to drop preferences size by roughly 33% and that nothing in my opinion justifies having 33% whitespace in a file opened by a very slim margin of users (especially with the addition of workarounds for those who really need it).

Asking the question backwards, if we weren't pretty printing today, would you say it's reasonable to enable pretty-printing (and 1.5X all existing files) for everyone all-the-time just because it helps in a few edge case debugging scenarios?

Here's my answer:
  • If the only performance impact is 33% more disk space usage for a file that's <100KB, I'd totally pretty print it. The end result is that the file is human readable with no relevant drawbacks.
  • If the larger file causes encode/decode and/or read/write performance degradation, then I'm very much convinced that better performance is more important than readable pref files. I haven't seen any substantial arguments for this though, which is why I asked.

Dirk Pranke

unread,
Mar 5, 2015, 2:02:56 PM3/5/15
to Peter Kasting, Gabriel Charette, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
+1. This doesn't seem like it should've gotten a command-line flag to me, either. There are plenty of other ways one could read the data.

-- Dirk

Scott Hess

unread,
Mar 5, 2015, 2:04:06 PM3/5/15
to Nico Weber, Gabriel Charette, Mattias Nissler, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
Gzip'ing my Preferences file cuts 85% of the size, and the result
would probably be about the same regardless of whitespace. Depending
on how much of latency is CPU time versus I/O time, it could end up
faster.

Of course, if it's really important another solution would be to use a
serialization format geared towards high performance and compact
representation, rather than tweaking around the edges.

-scott

Gabriel Charette

unread,
Mar 5, 2015, 2:08:50 PM3/5/15
to Peter Kasting, Gabriel Charette, Alexei Svitkine, mnis...@chromium.org, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
I debated that when making this request on the CL.

Here's my reasoning (put in context of now having the chrome://preferences possibility which I hadn't initially thought of):
  • A flag allows us to pretty print and diagnose even if Chrome drawing is otherwise borked (e.g. to toggle hardware acceleration off or something).
  • Pretty printing to file also makes it easier for edits (chrome://preferences would be read-only).
  • If we move to another format than JSON, pretty printing to JSON can only be done from Chrome (not a third-party JSON tool).
If we decide not to move away from JSON after all then I agree that we could consider relying on third-party tools for JSON debugging.

Cheers,
Gab

Gabriel Charette

unread,
Mar 5, 2015, 2:19:53 PM3/5/15
to Mattias Nissler, Brett Wilson, Gabriel Charette, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 2:01 PM Mattias Nissler <mnis...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 10:46 AM, Brett Wilson <bre...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 10:39 AM, Gabriel Charette <g...@chromium.org> wrote:

On Thu, Mar 5, 2015 at 1:16 PM Mattias Nissler <mnis...@chromium.org> wrote:
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

I don't think it's reasonable to 1.5X a file (i.e. undoing a 33% reduction is a 50% increase) with whitespace just a for few debugging scenarios.

Especially since you're just one --pretty-print-prefs step away from getting back to a debuggable state (and even closer if we add a chrome://preferences URL as Alexei proposed above) -- assuming the problem is not a startup problem (a drawing problem is fine because the prefs would still get pretty printed back to disk in the background even if Chrome can't render) and worst case the one line JSON is still searchable in notepad so it's not that hard.
 

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

These are the factors that were weighed, the measurement that was collected was the fact that we expect this to drop preferences size by roughly 33% and that nothing in my opinion justifies having 33% whitespace in a file opened by a very slim margin of users (especially with the addition of workarounds for those who really need it).

Asking the question backwards, if we weren't pretty printing today, would you say it's reasonable to enable pretty-printing (and 1.5X all existing files) for everyone all-the-time just because it helps in a few edge case debugging scenarios?

Here's my answer:
  • If the only performance impact is 33% more disk space usage for a file that's <100KB, I'd totally pretty print it. The end result is that the file is human readable with no relevant drawbacks.
  • If the larger file causes encode/decode and/or read/write performance degradation, then I'm very much convinced that better performance is more important than readable pref files. I haven't seen any substantial arguments for this though, which is why I asked.

I see your point.

Pref file size was theorized to be the cause of some CrOS shutdown crashes: http://crbug.com/418627#c32.

I haven't given much thought into whether I believed this to be the case because the size decrease alone was sufficient to convince me (as I'm currently trying to reduce all startup IO as part of the ongoing speed efforts and while I don't expect this to be huge I consider it completely superfluous).

Also, pref file size has come up before (e.g., http://crbug.com/355722 and recently http://crbug.com/424057).

ReadFileToString() reads files in chunks to a 65K buffer, and thus reducing the pref file from 180KB to 120KB (that's what it was in my local experiment) gets rid of an entire IO operation during startup (where spin disks are already going nuts seeking like crazy).

I will track PrefService.CreateProfilePrefsTime in UMA to see the perf impact in practice.

Cheers,
Gab

Mattias Nissler

unread,
Mar 5, 2015, 2:25:31 PM3/5/15
to Gabriel Charette, Brett Wilson, Benjamin Kalman, Mike Frysinger, Alexei Svitkine, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 11:19 AM, Gabriel Charette <g...@chromium.org> wrote:


On Thu, Mar 5, 2015 at 2:01 PM Mattias Nissler <mnis...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 10:46 AM, Brett Wilson <bre...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 10:39 AM, Gabriel Charette <g...@chromium.org> wrote:

On Thu, Mar 5, 2015 at 1:16 PM Mattias Nissler <mnis...@chromium.org> wrote:
So let me clarify the motivation for my reply: I'm aware of all the workarounds available to work with non pretty-printed files. However, there are case where even workarounds increase friction, such as asking bug reporters "can you please open this file and past the line labeled such-and-such". Asking for the entire file is also a step back as that might contain PII that we don't want them to share.

I don't think it's reasonable to 1.5X a file (i.e. undoing a 33% reduction is a 50% increase) with whitespace just a for few debugging scenarios.

Especially since you're just one --pretty-print-prefs step away from getting back to a debuggable state (and even closer if we add a chrome://preferences URL as Alexei proposed above) -- assuming the problem is not a startup problem (a drawing problem is fine because the prefs would still get pretty printed back to disk in the background even if Chrome can't render) and worst case the one line JSON is still searchable in notepad so it's not that hard.
 

That said, I don't think the above is a strong reason to reject the idea of reducing pref file size, I was just interested to hear what factors where considered in the decision that led to the change and whether we've collected measurements to properly weigh these factors.

These are the factors that were weighed, the measurement that was collected was the fact that we expect this to drop preferences size by roughly 33% and that nothing in my opinion justifies having 33% whitespace in a file opened by a very slim margin of users (especially with the addition of workarounds for those who really need it).

Asking the question backwards, if we weren't pretty printing today, would you say it's reasonable to enable pretty-printing (and 1.5X all existing files) for everyone all-the-time just because it helps in a few edge case debugging scenarios?

Here's my answer:
  • If the only performance impact is 33% more disk space usage for a file that's <100KB, I'd totally pretty print it. The end result is that the file is human readable with no relevant drawbacks.
  • If the larger file causes encode/decode and/or read/write performance degradation, then I'm very much convinced that better performance is more important than readable pref files. I haven't seen any substantial arguments for this though, which is why I asked.

I see your point.

Pref file size was theorized to be the cause of some CrOS shutdown crashes: http://crbug.com/418627#c32.

I haven't given much thought into whether I believed this to be the case because the size decrease alone was sufficient to convince me (as I'm currently trying to reduce all startup IO as part of the ongoing speed efforts and while I don't expect this to be huge I consider it completely superfluous).

Also, pref file size has come up before (e.g., http://crbug.com/355722 and recently http://crbug.com/424057).

ReadFileToString() reads files in chunks to a 65K buffer, and thus reducing the pref file from 180KB to 120KB (that's what it was in my local experiment) gets rid of an entire IO operation during startup (where spin disks are already going nuts seeking like crazy).

I will track PrefService.CreateProfilePrefsTime in UMA to see the perf impact in practice.

Cool, thanks - also for providing the additional background.

John Abd-El-Malek

unread,
Mar 5, 2015, 9:21:34 PM3/5/15
to Dirk Pranke, Peter Kasting, Gabriel Charette, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
+2

It's not worth paying this cost for a billion users because a few times a week we might ask some of them to give us some values. We can point them at the countless JSON pretty printing websites. Chrome devs can do that as well (or we can add code to pretty print it in debug builds).
 

-- Dirk

Gabriel Charette

unread,
Mar 6, 2015, 7:50:09 AM3/6/15
to John Abd-El-Malek, Dirk Pranke, Peter Kasting, Gabriel Charette, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Thu, Mar 5, 2015 at 9:21 PM John Abd-El-Malek <j...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 11:01 AM, Dirk Pranke <dpr...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 10:52 AM, Peter Kasting <pkas...@chromium.org> wrote:
On Thu, Mar 5, 2015 at 9:02 AM, Gabriel Charette <g...@chromium.org> wrote:
Oh sweet, didn't know about that one, yea that could be a very nice solution for prefs.

What do others think?

On Thu, Mar 5, 2015 at 12:00 PM Alexei Svitkine <asvi...@chromium.org> wrote:
Note that if you just want to view the contents for debugging, rather than edit them, there's also chrome://local-state debug url which I believe will show the contents pretty printed. Also works on mobile and other platforms where you can't get at the file easily. (If there's interest, a similar url can be added for profile prefs.)

Putting on my command-line-flags-owner hat: we shouldn't have long-lived command-line flags to do things in general.  Flags should be temporary things.  So if at all possible, please use/implement something like what Alexei proposes and remove the flag you've added.

(There are standalone utilities to pretty-print JSON files, aren't there?  Some combination of telling people to use those, plus the facilities described above, seems sufficient to obviate the need for a flag to me.)

+1. This doesn't seem like it should've gotten a command-line flag to me, either. There are plenty of other ways one could read the data.

+2

It's not worth paying this cost for a billion users because a few times a week we might ask some of them to give us some values. We can point them at the countless JSON pretty printing websites. Chrome devs can do that as well (or we can add code to pretty print it in debug builds).

Okay, you convinced me, hadn't considered the alternative of online JSON pretty-printing tools (e.g., jsonprettyprint.com).

Removing the switch since I asked for it, please review: https://codereview.chromium.org/982293002/

We may still need a Chrome provided switch if we one day decide to move away from JSON to a custom format, but it's not required yet.

Cheers,
Gab

Xiaohan Wang (王消寒)

unread,
Mar 10, 2015, 6:27:36 PM3/10/15
to Gabriel Charette, John Abd-El-Malek, Dirk Pranke, Peter Kasting, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
(Found this thread because the flag doesn't work for me any more :)

My 2-cents:
  • Preference file contain a lot of private data (e.g. browsing history, settings etc). Probably we shouldn't recommend people (even developers) to paste it to a random website for pretty printing. There are a lot of ways to do this locally, e.g. "python -mjson.tool Preferences".
  • The Preference file is bloated not only because it's pretty printed, but also because it has a lot of deprecated preferences synced from the sync server. Do we have any plan to clean that up? For example, in a newly synced Preference of one of my test accounts, I have the following, even though libnetflixplugin2 has been deprecated for years. BTW, I don't know what nested 1.0.4.195 entry is; it looks weird to me :(
            "plugin_whitelist": {
                "1": {
                    "0": {
                        "4": {
                            "195": true
                        }
                    }
                },
                "libnetflixhelper": {
                    "so": true
                },
                "libnetflixplugin2": {
                    "so": false
                },
            ......

Xiaohan

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

Gabriel Charette

unread,
Mar 11, 2015, 2:17:35 PM3/11/15
to Xiaohan Wang (王消寒), Gabriel Charette, John Abd-El-Malek, Dirk Pranke, Peter Kasting, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
On Tue, Mar 10, 2015 at 6:26 PM Xiaohan Wang (王消寒) <xhw...@chromium.org> wrote:
(Found this thread because the flag doesn't work for me any more :)

My 2-cents:
  • Preference file contain a lot of private data (e.g. browsing history, settings etc). Probably we shouldn't recommend people (even developers) to paste it to a random website for pretty printing. There are a lot of ways to do this locally, e.g. "python -mjson.tool Preferences".

Right code/CL comment says to use your "favorite command-line or online tool" IIRC. I don't really care uploading my preferences to a server but that's a individual choice IMO, I doubt anybody tracks me this way..
 
  • The Preference file is bloated not only because it's pretty printed, but also because it has a lot of deprecated preferences synced from the sync server. Do we have any plan to clean that up?

Agreed, but cleaning it up is much more complicated than disabling pretty print. If this 30% size reduction proves worthwhile, we could explore more cleanup efforts. But right now, no one is working on such a cleanup AFAIK.
 
  • For example, in a newly synced Preference of one of my test accounts, I have the following, even though libnetflixplugin2 has been deprecated for years. BTW, I don't know what nested 1.0.4.195 entry is; it looks weird to me :(

Looks like a bug this code should use DictionaryValue::SetBooleanWithoutPathExpansion() from the "plugin_whitelist" dictionary, but probably doesn't. Happy to review a fix.

Elliott Sprehn

unread,
Mar 11, 2015, 6:11:59 PM3/11/15
to Gabriel Charette, Xiaohan Wang (王消寒), John Abd-El-Malek, Dirk Pranke, Peter Kasting, Alexei Svitkine, Mattias Nissler, Daniel Bratell, chromium-dev, Ryo Hashimoto, Ryo Hashimoto
What about gzip? If disk read performance is the issue making the file smaller at the expense of some more CPU time will be a win.
Reply all
Reply to author
Forward
0 new messages