navigator.cores proposal

[Eli Grey]

I am proposing a very simple JavaScript API that already exists in PNaCl: navigator.cores, a read-only integer.

Webapps and JS libraries need this in order to allocate an appropriately sized worker threadpool for CPU-intensive multithreaded tasks. Such an API would make these examples possible without a requiring the user to interact with a GUI slider or radio list before processing:

1. Porting multithreaded LZMA2 to JavaScript without having to prompt the user for worker thread count.
2. Running realtime object/face/movement/etc. detection algorithms on webcam input or video file input, without prompting the user for a worker thread count.
   Of note: There are many web worker based webcam webapps, and almost every single one of them includes a GUI for worker thread count or just a hardcoded value.
3. Multithreaded automatic OCR, similar to http://projectnaptha.com/ (single-threaded) but faster and more responsive.
   If Project Naptha is ever to add multithreaded Ocrad support at this time (without the PNaCl workaround), it would need to prompt the user to enter their number of cores by opening a popup window or injecting HTML into the current document, as it is, for the most part, a silent background extension.
   Users would not know what to do if a popup window came up after installing Project Naptha, asking how many cores they have. If the user bought a "gaming" computer, they may even assume the best choice is the maximum option, which will lead to performance problems (e.g. trying to run 32 threads on a dual core laptop).
   

Current web worker-based applications must hardcode a threadpool size and may offer a GUI for adjusting the size. The problem with this is that most users do not know how many cores their computer has, nor should they have to.

FAQ

• This already exists?
  
  Yes. PNaCl run from any webpage can use sysconf(_SC_NPROCESSORS_ONLN) in order to get the core count.
  
  
• As some developers in #chromium have brought up, why not an event based model that gives a suggested "optimum thread count" based on current load?
  
  There are many reasons why this is a bad idea:
  
• This introduces a large amount of threadpool management overhead into the application.
  
• Depending on the application complexity, the setup and unload overhead caused by frequently resizing the threadpool could easily outweigh any potential performance benefits gained by this model.
  
• The load caused by the application itself will confound the data reported by the event model, further complicating the issue.
  
• We already trust OS schedulers to do this for us:
  Most multithreaded native desktop applications (especially compression software like xz) do not attempt to dynamically resize their threadpools based on CPU load. Instead, they use all available cores and rely on the OS scheduler to determine CPU resource allocation.

Potential security concerns

CPU core count is not private information, as demonstrated with the above PNaCl code snippet. If for some reason you believe that CPU core count should be private information, know this:

I have created a timing attack that uses worker threads on any CPU in order to estimate the number of cores it has, as demonstrated at http://wg.oftn.org/projects/core-estimator/demo/. There is no defense against this timing attack other than completely removing support for web workers.

[Adam Barth]

Hi Eli,

Would you be willing to send your proposal to the WHATWG?  That's a better forum for getting feedback on ideas like this because it lets multiple vendors participant on an equal footing.

Off-hand, this sounds like a reasonable feature request.

Adam

[Eli Grey]

Ian Hixie (not 100% sure, but doesn't he have final say in WHATWG?) ended the last navigator.cores-like proposal. He disagrees with using the CPU core/thread count, citing dynamic load concerns, but doesn't acknowledge OS scheduler responsibilities or capabilities. He also thinks that this is a privacy leak. As I have proven above with my timing attack, this information is already leaked on all browsers, so it is irrelevant if it is a privacy leak.

Instead of having this proposal being shut down again, I thought that Chrome might want to lead the change.

[Elliott Sprehn]

I agree this is not a pricacy leak. Native has this capability and so does PNacl. Can you post a patch that exposes this on navigator? We should understand how expensive this is to implement. I suspect it's trivial.

In the worst case we should add a requestHardwareInfo() promise that guards the values and make it always resolve to success for now. If we decide this is actually sensitive later we can add a user prompt or other security guard. That's the model for exposing new information that's future proof.

[Adam Barth]

It's pretty trivial to implement:

https://codereview.chromium.org/262723002/

Eli, would you be willing to write up a proposal on the WHATWG wiki <http://wiki.whatwg.org/wiki/Category:Proposals> that explains how this feature works?

Adam

[Eli Grey]

Yeah, sure. I will later today.

[Jonas Sicking]

For what it's worth, in Firefox we've avoided implementing this due to
the increased fingerprintability. Obviously we can't forbid any APIs
which increase fingerprintability, however in this case we felt that
the utility wasn't high enough given that the number of cores on the
machine often does not equal the number of cores available to a
particular webpage.

A better approach is an API which enables the browser to determine how
much to parallelize a particular computation.

/ Jonas

> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+...@chromium.org.

[Adam Barth]

For me, it's more of an issue of parity with native frameworks.  Cocoa apps get to learn the number of cores.  So do Android apps:

http://developer.android.com/reference/java/lang/Runtime.html#availableProcessors()

How can the web compete with these frameworks if we hold back this sort of basic information from developers?

Adam

> email to blink-dev+unsubscribe@chromium.org.

[Eli Grey]

Jonas: People are already using web workers, and this provides a drop-in upgrade for everyone who is currently hardcoding their threadpool size (all you have to do is replace threads = # with threads = navigator.cores || #). Of course a parallizeability worker API would be great in the future, but it will require large changes to existing applications using worker threads that wish to use the new API. It has been 5 years since discussions have began on creating such an API, and we still don't have anything like that in any browsers. All I want is to know how many threads I need to allocate in the old worker model, and this is a very trivial way to add support for that.

As for fingerprintability, run the Core Estimator demo I linked to in Firefox. As you can see, I can already fingerprint you based on CPU cores. Even if Chrome and Firefox did implement the API you describe, what's to stop me from running Core Estimator anyways to help fingerprint you?

[Jonas Sicking]

We are aware of these arguments. However so far we have not felt that
the added benefits to the user is greater than the added risks to the
user. Obviously others may disagree.

/ Jonas

[Adam Barth]

To make sure I understand, the risks you're referring to are fingerprinting risks, correct?

Adam

> email to blink-dev+unsubscribe@chromium.org.

[Boris Zbarsky]

On 4/30/14, 7:48 PM, Eli Grey wrote:
> As for fingerprintability, run the Core Estimator demo I linked to in
> Firefox. As you can see, I can already fingerprint you based on CPU
> cores.

So I tried running this.

I consistently (5 runs in a row) get it reporting "4" in Chrome. In
Firefox 29 I have so far gotten 5, 5, 4, 9, 5. In a Firefox nightly I
consistently (5 runs in a row) get 10.

I have 4 cores, fwiw, if you ignore hyperthreading and whatnot.

That said, I accept in principle that you can extract some information
about the number of cores this way, if you're willing to throw enough
compute time at it.

-Boris

[Eli Grey]

I currently have the demo tuned in order to run in a short (usually under 10 seconds) amount of time. By doubling the samples taken and increasing the workload, I can get much more accurate core counts at the expense of you waiting a minute or two instead of seconds.

Also, as for the "5" results you are getting, please be aware that this algorithm is not specifically intended to extract an accurate core count, and instead extracts a useful core count in a short amount of time. It does this by erring on the side of more cores than less, as a quad core processor can get more done with 5 threads faster than it can with 3, for example. If my intention was only to get a core count for fingerprinting, I could increase the accuracy as described above and remove the code that errs on having more cores.

[Eli Grey]

Here is a draft of my proposal on the wiki: http://wiki.whatwg.org/wiki/NavigatorCores

[Domenic Denicola]

Is this kind of information generally available synchronously? I would guess yes, as it could be cached at process start-up time and the same number would be returned without any delay when requested. But I wanted to check.

[Adam Barth]

It's available synchronously in POSIX.  You just ask the kernel.  Looks like Windows works the same way:

http://msdn.microsoft.com/en-us/library/ms724381%28VS.85%29.aspx

Adam

[Darin Fisher]

Yes, it is available synchronously.

I'll also note that I think this API seems useful. I can imagine wanting to know how many workers I should create when performing a parallel computation. I don't think the concerns around fingerprinting are well founded.

-Darin

[Si Robertson]

Regarding fingerprinting, "navigator.cores" doesn't seem any worse than knowing a user's screen resolution or input devices (mouse, touchscreen, etc), or even the output sample rate used by their audio hardware (detected by Web Audio).

Even with all of those variables combined they still wouldn't be reliable enough to fingerprint an individual, the variables are no where near variable enough.

[Darin Fisher]

It is the entire collection of useragent varying features of the platform that in total provide a fingerprint. When considering a new useragent varying feature, we ask whether it makes the situation worse. In this case, the number of cores seems reliably detectable, especially for pages that the user visits for a length of time (e.g., a background tab left open).

-Darin

[Si Robertson]

Fair enough.

In that case isn't there some other generic value that could be exposed to developers, instead of the exact number of cores, to indicate the hardware's threshold for multiple web workers?

For example, Worker.cpuThreshold = "low" || "medium" || "high"

That would allow developers to balance the number of worker instances without knowing anything [more] about the CPU architecture.

[Mike Lawther]

That doesn't seem useful. On the one hand you've got APIs that accept a number for the count of workers they'll start. And on the other, you're getting a non-number value that you need to interpret to be able to provide a number..

What does 'high' (or 'low') imply? 4 workers? 8? 32? What will it mean next year?

[Darin Fisher]

Hmm, I wasn't arguing against revealing the exact number of cores (or the exact number of cores available to a web page / origin). I was just trying to explain why folks worry about such APIs from a fingerprinting perspective. It's the sum of entropy sources that may yield a fingerprint. If some useragent varying feature is already detectable (though perhaps requiring some effort or time), then providing a direct and simple API to get at the same information does not seem so problematic from a fingerprinting point of view.

-Darin

[Si Robertson]

No it isn't very useful :-)

It is however one possible solution for those who are concerned about fingerprinting. Personally I think exposing the number of CPU cores will be more beneficial than detrimental, and JS development is reaching a point where this kind of information is going to be required to optimize the performance of CPU intensive games and applications.

[Anne van Kesteren]

On Thu, May 1, 2014 at 8:03 AM, Darin Fisher <da...@chromium.org> wrote:
> If some useragent varying feature is already detectable (though perhaps
> requiring some effort or time), then providing a direct and simple API to
> get at the same information does not seem so problematic from a
> fingerprinting point of view.

Why not? An analytics script embedded on many pages throughout the web
would likely use the latter, but would probably not use the former as
it would impact performance too much.


--
http://annevankesteren.nl/

[Eli Grey]

Anna: Remember, Chrome already exposes this number publicly to all web content through PNaCl.

As for performance of the timing attack: On various recent (desktop TDP) Intel CPUs over 3GHz ranging from 4 to 12 logical cores, the estimate always finishes in under 5 seconds for me.

Even the relatively slow 1.4GHz dual core Haswell part in all of the new Chromebooks reproducibly finishes Core Estimator in ~7.5 seconds, after 4 complete core tests of varying sizes (1,2,4,3).

[Darin Fisher]

In my experience, CPU burn isn't much of a deterrent. There are beaconing "techniques" that involve issuing an async request in unload followed up by half second of CPU burn. If N trackers do this it really adds up :-(

In this case the CPU burn can happen in a non blocking fashion, which would surely make it even less of a deterrent--especially on desktop.

-Darin

[Alex Russell]

I'm sympathetic to Jonas' argument here.

I'd like for us to do the request*() style for this API as it allows us to selectively expose the information to apps based on trust level.

[Joshua Bell]

And though we should be designing for the future, making it request*() style (as Elliott and Alex suggest) also makes it more easily polyfilled using a variant of Eli's estimation script.

[Darin Fisher]

That's clever, but hmm...

It all just seems a bit gratuitous to me. I don't see why we would bother ever rejecting such a request. It's a static property we can determine before letting script run. Doesn't seem like it needs to be more than a property getter.

-Darin

[Eli Grey]

If a request style API is used in JavaScript, malicious websites will just hook into PNaCl for your core count instead, since it doesn't use a permission dialog. If PNaCl is modified to use request style API for sysconf(_SC_NPROCESSORS_ONLN), they will hook into Core Estimator instead. As Darin Fisher put it, malicious websites simply don't care. This is going to be used for fingerprinting whether we like it or not, so let's at least spare the users their CPU cycles. I'd rather not be "that guy" who made the library that you hate for wasting your CPU cycles on malicious websites. If a request style API is used, I'm going to be that guy.

The only possible solution to closing this leak, other than removing web worker support entirely, is to add a request style API to the instantiation of worker threads themselves.

[Eric Seidel]

I suspect Academics have studied the finger-printing problem and can
tell us exactly how many bits of data they need before reliably
finger-printing across a population of size N.

My gut tells me that the browser likely already exposes plenty of
information to reliably fingerprint users.

This discussion is about exposing a few more bits of data. If my gut
is correct, we're already several orders of magnitude beyond the
necessary number of bits to reliably fingerprint, at which point this
discussion is moot.

Does someone have real data as to how close we are to the "sufficient
data to reliably fingerprint users" browsers currently are?
Presumably there are good papers on this topic?

Can we move this discussion to data/research instead of gut feels?
Otherwise we're just offering competing hunches. If we're left only
with hunches, mine goes with Adam/Elliott/Eli/Darin that the
finger-printing game is long-since lost and we should expose this
directly today.

[Jeremy Roman]

EFF's Panopticlick suggests that many users are already easy to fingerprint (see both the tool and paper).

https://panopticlick.eff.org/

[Jeremy Roman]

Forgot to link to the paper itself:

https://panopticlick.eff.org/browser-uniqueness.pdf

[Ian Hickson]

On Wed, 30 Apr 2014, Eli Grey wrote:
>
> As for fingerprintability, run the Core Estimator demo I linked to in
> Firefox. As you can see, I can already fingerprint you based on CPU cores.

The difference between exposing the number and being able to estimate the
number is the value of the number.

For example, I ran the estimator just now while my computer was busy, and
I got a wrong number. Then I ran it again while it was less busy and I got
a different number. This means that you couldn't use this to reliably add
bits to fingerprint me over multiple visits (which is the whole point of
fingerprinting). (My load varies wildly during a day, for example based on
whether I've got GMail open, or G+ open, or both, or whether I'm watching
videos, or listening to music, etc.)

On the other hand, the API would give you a precise unchanging number,
which has a high value for the purposes of fingerprinting. It would give
you probably 2 bits these days, probably up to 3 bits in the medium term.
That's about as many bits as you get from looking at the user's timezone.

It also demonstrates that the API is a bad API for the purpose of the API.
When my machine is busy, the core estimator correctly figures out how many
cores it should use -- if I run multiple copies in parallel, they each
figure out that there's a load going on, and thus they all end up deciding
to use fewer cores. If they instead relied on a static number form an API,
many Web pages would end up all competing to use all my cores, instead of
managing the load intelligently.

I'm all for exposing an API that would let pages dynamically manage how
many threads they want to use, but that's not a raw "number of cores".

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Boris Zbarsky]

On 5/1/14, 12:55 PM, Eli Grey wrote:
> If a request style API is used in JavaScript, malicious websites will
> just hook into PNaCl for your core count instead

Just as a note, that's true for Chrome, but obviously not for other
browsers. If/when you try to sell other UAs on this, I suggest not
using "we already expose this information via PNaCl" as an argument,
since their (reasonable, imo) response would be that that's your problem
with exposing too much via PNaCl.

-Boris

[Eli Grey]

As I explained previously in the thread, the Core Estimator I linked to is the official version, configured to actually be useful for parallel algorithms (i.e. it doesn't try to take longer to run than your algorithm itself). Here is a differently configured version that takes 12 times as long and should give you the same result every time (given that you're not trying to run multiple of the demo at once or intentionally consuming all CPU resources in order to prove a point): http://wg.oftn.org/projects/customized-core-estimator/demo/

As it stands, workers are not useful for scalable parallel tasks, and there have been no successful proposals in the meantime. Meanwhile, the change I propose only takes a few lines of code and actually makes workers useful for parallel tasks, without exposing anything I didn't already know.

> When my machine is busy, the core estimator correctly figures out how many 
> cores it should use

When your machine is busy, do your video games start reducing threads used for their physics engines? Does xz resize it's threadpool during compression under system load? Do any of your user space desktop applications react specially to high system load? No.

When your machine is busy, your OS scheduler allocates CPU resources based on thread priorities. This is not a job for your hypothetical API that requires complete rewrites of current worker algorithms. It's a job for your OS scheduler.

[Boris Zbarsky]

On 5/1/14, 2:18 PM, Eli Grey wrote:
> Here is a differently configured version that takes
> 12 times as long and should give you the same result every time (given
> that you're not trying to run multiple of the demo at once or
> intentionally consuming all CPU resources in order to prove a
> point): http://wg.oftn.org/projects/customized-core-estimator/demo/

Just fyi, I got "2" two times in a row in Chrome, "3" two times in a row
in a Firefox nightly, 5 just now in release Firefox. Still on the same
4-core system. This is still useful info for fingerprinting, obviously:
the number of cores I have is clearly not "1" and not "12 or more"
according to these numbers. It's just not as useful as you claim it is.

-Boris

[Eli Grey]

I only improved precision in the fingerprint version, not the accuracy. The only way to improve accuracy (on Windows) is to fix the high res timer https://code.google.com/p/chromium/issues/detail?id=158234. As long as you get the same result (±1) every time in the same browser, the number can still be used for fingerprinting.

I agree that when this is actually used for fingerprinting you would still want to store a range of ±1 from the estimate in order to have a more robust fingerprint.

[Alex Russell]

On Thu, May 1, 2014 at 9:54 AM, Darin Fisher <da...@chromium.org> wrote:

That's clever, but hmm...

It all just seems a bit gratuitous to me. I don't see why we would bother ever rejecting such a request. It's a static property we can determine before letting script run. Doesn't seem like it needs to be more than a property getter.

If fingerprinting is the risk, we should enable that UAs have leeway about when/what to expose this information. The request*() style gives us that.

[Domenic Denicola]

From: sligh...@google.com <sligh...@google.com> on behalf of Alex Russell <sligh...@chromium.org>

> If fingerprinting is the risk, we should enable that UAs have leeway about when/what to expose this information. The request*() style gives us that.

I don't see any advantages of the promise-returning request*() style over having the .cores getter throw a synchronous exception.

[Alex Russell]

Exceptions suck?

[Domenic Denicola]

No more than rejected promises? Async and sync exceptions seem equivalent.

---

From: sligh...@google.com <sligh...@google.com> on behalf of Alex Russell <sligh...@chromium.org>

Sent: Thursday, May 01, 2014 15:55
To: Domenic Denicola
Cc: Darin Fisher; Joshua Bell; Jonas Sicking; Eli Grey; blink-dev; Elliott Sprehn; Adam Barth
Subject: Re: [blink-dev] navigator.cores proposal

 

[Jonas Sicking]

On Wed, Apr 30, 2014 at 5:42 PM, Adam Barth <aba...@google.com> wrote:
> To make sure I understand, the risks you're referring to are fingerprinting
> risks, correct?

Yes.

Do note that the fact that you can already approximate this API using
workers is just as much an argument for that there are no additional
fingerprinting entropy exposed here, as it is an argument for that
this use case has already been resolved.

Additionally many of the people that are fingerprinting right now are
unlikely to be willing to peg the CPU for 20 seconds in order to get a
reliable fingerprint. Though obviously there are exceptions.

Another relevant piece of data here is that we simply haven't gotten
high priority requests for this feature. This lowers the relative
value-to-risk ratio.

This doesn't mean that Blink can't ship this feature though. I believe
that the blink rules allow shipping features to solve use cases which
other vendors aren't interested in solving. Likewise I would not be
surprised if other vendors are interested in implementing this
feature.

/ Jonas

[Elliott Sprehn]

We can't synchronously throw an exception since that would prevent us showing a permission prompt without adding a sync IPC to the browser process.