Intent to Ship: RFC 7616 Digest auth: Support SHA-256, SHA-512-256 and user hashing

[Deomid "rojer" Ryabkov]

Contact emails

roj...@gmail.com

Explainer

None

Specification

https://datatracker.ietf.org/doc/html/rfc7616

Summary

https://datatracker.ietf.org/doc/html/rfc7616 specifies SHA-256 and SHA-512-256 algorithms for the HTTP digest authentication scheme, in addition to the obsolete and insecure MD5. It also specifies way of concealing the username, provided that server supports it. Firefox supports algorithm=SHA-256 since 93, but not SHA-512-256 or username hashing. https://chromium-review.googlesource.com/c/chromium/src/+/4611879 is the pending CL.

Blink component

Blink>Network

TAG review

None

TAG review status

Not applicable

Risks

Interoperability and Compatibility

Gecko: Shipped/Shipping (https://bugzilla.mozilla.org/show_bug.cgi?id=472823)

WebKit: No signal

Web developers: No signals

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

Debuggability

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

No

Flag name

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1160478

Estimated milestones

Shipping on desktop

116

Shipping on Android

116

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5139896267702272

Links to previous Intent discussions

This intent message was generated by Chrome Platform Status.

--

Deomid "rojer" Ryabkov

ro...@rojer.me

[Daniel Bratell]

I am missing the compatibility picture here. How will this affect existing web pages, and what happens to browsers that do not support this is we add support?

/Daniel

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwr1XJbEU-yWbe2Whx%2Bago2njJFg-gOOdKzEj0%3DGVzP%3D0g%40mail.gmail.com.

[Deomid "rojer" Ryabkov]

Hi Daniel,

I do not anticipate immediate impact. By and large, this is not currently used because the most popular browser engine doesn't support it. So people just use MD5, as in the bad old days.

If we ship this tomorrow, nothing changes: servers still send legacy MD5 challenges, which we continue to support.

As awareness of SHA-256 support by Chrome/Blink-based browser spreads, I expect service administrators to gradually start transitioning away from MD5.

The RFC specifies use of multiple algorithm challenges by sending multiple WWW-Authenticate headers, so there's a good transition path there for service admins (send both SHA-256 and MD5 challenges, phase out MD5 eventually).

I verified that with my current code this works as specified (order of headers specifies server's preference).

Firefox already supports SHA-256, so there's already a capable browser in the wild, but since it's a minority there isn't much use yet.

As far as deviating from Firefox in also supporting SHA-512-256 and userhashing, it's again a no-op initially: these won't be used unless servers explicitly specify that they can be used (for example, for lighttpd there's an explicit option to enable username hashing, off by default).

If Firefox catches up and also implements these, then we might start seeing adoption.

[Rick Byers]

Hi Deomid,

Thanks for the contribution! Do you know if chromium has any metrics on how common digest auth is? I took a quick look and didn't find one myself. +David Benjamin also. Technically all new APIs (which includes protocols) require a killswitch flag since we've seen even new APIs break websites sometimes (eg. in this case presumably it would be from a misconfigured server). But if we have data showing the usage of digest auth is small / insignificant enough, I'd personally be receptive to an argument that that is unnecessary extra bureaucracy. 

Also it would be nice to know if WebKit has an opinion on this, could you file a position request please?

Thanks,

   Rick

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwof4uGQYUB%3Dac00NisuQG%3Di1JrDT7BcvJtPzMReWcZyxQ%40mail.gmail.com.

[Deomid "rojer" Ryabkov]

Hi Rick,

thanks for your response. I've files a WebKit issue: https://github.com/WebKit/standards-positions/issues/212

as for metrics, i don't have any. we have a 2-year old comment from ericlaw@microsoft that usage is extremely low.

Perhaps i should clarify why I am bothering with this at all: embedded device usage. Use of built-in auth mechanisms is attractive for extremely low footprint devices, we happen to be shipping one of those and currently use it.

Use of SSL/TLS there is also usually not an option (for lack of trusted certificates, if not footprint concerns), so basic+TLS is not a viable alternative, hence the desire to have digest auth with a modern algorithm.

This - use in embedded environments - could also be masking some of the usage. But in general I tend to agree that current use is most likely negligible overall.

I do not have access to any metrics though.

[David Benjamin]

Not an expert in our HTTP auth code overall, but I don't see any metrics for specifically digest auth vs. other HTTP auth types either.

As an upper bound, I wouldn't expect HTTP auth of any form to be common. This whole space is mostly a legacy feature as far as the web is concerned. I doubt usage would ever completely fall off, but it's not exactly a preferred path, or a space that sees much active work. Making modern hashes available for Digest seems reasonable enough, I suppose, but I wouldn't want to spend more time on it than that. :-)

But I couldn't find good metrics for the upper bound either. Net.HttpResponseCode says how often we see 401 and 407 response codes (very rarely), but that's a little misleading. If I recall (not an HTTP auth expert), for the simpler auth types (Basic and Digest), cached credentials are sufficient to preemptively send a (Proxy-)Authorization header. That means we'll only see a 401 or 407 on the first go around, up to some interesting rules around which paths an auth entry is scoped to.

In terms of breakage risk, since this protocol is server-offer / client-select, the client never sends the list of values it supports. That means only servers that actually claim to support the new hashes will see any change at all. So the risk is limited to early-adopting servers that have adopted the new thing but gotten it wrong. Or if we got it wrong. :-) (No particular opinions on my end as to whether that makes it worth flag-protecting.)

[Deomid "rojer" Ryabkov]

+ericlaw@microsoft

Eric, could you perhaps provide updated numbers of digest auth usage? (i'm referring to your previous comment here: https://bugs.chromium.org/p/chromium/issues/detail?id=1160478#c6)

[Yoav Weiss]

Thanks for working on this!

Would you be able to put this behind a flag, to reduce any risk this might hit in the wild?

It'd also be good to add use-counters to know how common these digests are, but given that the code is fully contained in net/ this might be a bit tricky. Maybe +Adam Rice and team would be able to help with adding some metrics there? That's not a blocker for this intent, but would help shed light on overall usage of this feature.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwryoCQ8j4sfPFh3Ozs%3DKaMDt0xEHFMo61mQXXzVw0eM%3DQ%40mail.gmail.com.

[Rick Byers]

Agreed that given we don't have metrics proving the usage is really rare, we should err on the side of caution and follow the "all new APIs need a killswitch" guideline here. It should be straightforward to do so, but please let us know otherwise. 

Rick

[Rick Byers]

Thanks for the details and your guidance by the way David! It's helpful.

[Adam Rice]

While plumbing use counters from //net back into Blink would be technically possible, it would be a lot of work and we'd need some compelling user benefit to justify it.

In this case, just having a kill switch seems sufficient.

[Yoav Weiss]

On Fri, Jun 30, 2023 at 8:38 AM Adam Rice <ri...@chromium.org> wrote:

While plumbing use counters from //net back into Blink would be technically possible, it would be a lot of work and we'd need some compelling user benefit to justify it.

Could we simply add UMA instead?

[David Benjamin]

What question are we trying to answer with UMA?

UMA came up in order to answer "Is HTTP auth rare enough to not need a kill switch?" But it's too late to answer that question usefully because we've already decided to make a kill switch.

We could preemptively gather metrics for the next such HTTP auth change, but HTTP auth is a legacy feature, so the likelihood that we'll have a use for it before the metric expires and causes some busywork is low. Conversely, the cost of adding a kill switch is low, so that busywork isn't buying that much.

As it's a legacy feature, we could answer "is HTTP auth or specifically digest auth so uncommon that we can remove it?". But we don't need metrics to know the answer to that is no. This is a legacy feature that's been around basically since HTTP started. It's not going anywhere. :-(

We could answer "how common is HTTP auth or specifically digest auth, and should we be investing in it after all?" Again, we don't need metrics to know the answer to that is no.

We could answer "how often is MD5 Digest auth used and can we remove it?". That is also almost certainly a no, and will likely remain a no for a long, long time. And as digest auth's security benefit is pretty minimal, I don't think it's really worth the effort to track that and try to deprecate it later.

We could answer "was this new feature we added ever used / was it worth the time?" a few years down the line. And, honestly, I bet the answer to that will also be no. But adding these hashes costs nothing, and so if someone is enthusiastic about adding it, I think it's fine. I don't think we'd bother unshipping it should the answer turn out to be no. I suppose we could have said "we'll only add SHA-256 and not SHA-512-256 because having extra options is not a good idea but whatever". :-)

So the benefit of metrics seems marginal at best. I don't think it's worth the (also marginal) cost in busywork and engineering time. Honestly, I think we've already spent far more time on this mechanism in pondering this than the feature justifies. ;-)

[Deomid "rojer" Ryabkov]

ok, added a flag (enabled by default).

[Yoav Weiss]

On Fri, Jun 30, 2023 at 8:49 PM David Benjamin <davi...@chromium.org> wrote:

What question are we trying to answer with UMA?

I was thinking that UMA will enable us to answer questions around HTTP auth deprecation aspirations. But you make a good point about UMA metrics expiring.

So I'm convinced we should only add them in the future if we have a specific goal in mind. Thanks!

[David Benjamin]

On Mon, Jul 3, 2023, 00:15 Yoav Weiss <yoav...@chromium.org> wrote:

On Fri, Jun 30, 2023 at 8:49 PM David Benjamin <davi...@chromium.org> wrote:

What question are we trying to answer with UMA?

I was thinking that UMA will enable us to answer questions around HTTP auth deprecation aspirations. But you make a good point about UMA metrics expiring.

So I'm convinced we should only add them in the future if we have a specific goal in mind. Thanks!

Ah. Yeah, maybe I'm just pessimistic, but I would be shocked if HTTP auth is viable to deprecate and remove. Though I certainly will not grieve anything if I'm wrong here! :-)

At the least, any metrics for that are probably orthogonal to this Intent, since either way HTTP auth is solidly a legacy feature. So, my (personal and non-authoritative!) answer to Apple's question is mostly apathy: this is not a feature worth investing in, but I won't stand in the way if folks want to upgrade some hash functions in it.

Though we shouldn't give the impression that this change is some meaningful security upgrade. It is not. HTTP digest auth is a dead end and should not be used.

What that means for Chromium accepting or rejecting this proposal, I'll defer to others. The longer this conversation goes on, the more inclined I am to switch my apathy to "no" because we've already spent more energy on it than this feature warrants! :-)

(Hmm, I see Firefox only added SHA-256 and not SHA-512-256. Should we accept this proposal, we may want to trim that down. Having extra choices is bad, *and* we'd be adding, in a tiny way, to platform divergence.)

David

[David Benjamin]

Oh. If the story here is to login to servers without TLS, we definitely should say no to this. That is just not a coherent threat model. So many things go wrong in that context that there's no point. The only thing we should be communicating about passwords and insecure origins is "don't".

On Mon, Jul 3, 2023, 12:10 Deomid "rojer" Ryabkov <ro...@rojer.me> wrote:

actually, my next step would be to go add SHA-512-256 and username hashing to FF for completeness but i'm fine with dropping it, my main goal is to have SHA-256.

yes, HTTP auth is old and cruaty but it does the job with very little server support and is better than basic in the absence of TLS, which is not an option in certain embedded applications.

[Deomid "rojer" Ryabkov]

actually, my next step would be to go add SHA-512-256 and username hashing to FF for completeness but i'm fine with dropping it, my main goal is to have SHA-256.

yes, HTTP auth is old and cruaty but it does the job with very little server support and is better than basic in the absence of TLS, which is not an option in certain embedded applications.

On Mon, 3 Jul 2023 at 11:24, David Benjamin <davi...@chromium.org> wrote:

[Deomid "rojer" Ryabkov]

i don't want to take this too far off track but what is the practical alternative for management UIs, in the absence of a trusted certificate?

every single piece of equipment i've encountered that has a management web UI uses insecure authentication, either basic or simple form + cookie-based, with password sent in plain, no challenge/response or nonce counter mechanism.

it's a problem we face as producers of such equipment, except we at least make an effort and avoid sending password in plain (currently by implementing the sha-256 digest on the client side ourselves, because browsers only support md5).

but if there is a better solution i'm very much interested.

[Alex Russell]

This should be sent to the TAG for review. What criteria were you using to claim review wasn't necessary?

As a general matter, I'm happy for us to remove MD5 and other insecure ciphers everywhere we can, so I'm nominally positive, but can't LGTM until we dot I's and cross T's.

Best,

Alex

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwr1XJbEU-yWbe2Whx%2Bago2njJFg-gOOdKzEj0%3DGVzP%3D0g%40mail.gmail.com.

--

Deomid "rojer" Ryabkov

ro...@rojer.me

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwof4uGQYUB%3Dac00NisuQG%3Di1JrDT7BcvJtPzMReWcZyxQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwryoCQ8j4sfPFh3Ozs%3DKaMDt0xEHFMo61mQXXzVw0eM%3DQ%40mail.gmail.com.

--

Deomid "rojer" Ryabkov

ro...@rojer.me

[Jeffrey Yasskin]

On Wed, Jul 5, 2023 at 8:45 AM Alex Russell <sligh...@chromium.org> wrote:

This should be sent to the TAG for review. What criteria were you using to claim review wasn't necessary?

This pretty clearly falls under the first point in https://www.chromium.org/blink/guidelines/api-owners/process-exceptions/: it's standardized in RFC 7616, and it's shipped in one other browser.

That said, I think David might be right about whether we should add this to Chromium, unless we're missing part of the threat model here. I haven't done a full security analysis of digest auth, but it sounds like an active attacker can always replace digest auth with basic auth, and so get the user to send the raw password to the attacker without showing any UI difference. So changes to digest auth can only protect against eavesdroppers.

Eavesdroppers also can't change Javascript, so implementing a better hash there gives equivalent security to doing it in the browser. Doing it in client-side Javascript is likely to be better, since developers can pick a hash like Argon2 that's meant to resist a compromise of the password database. It's possible that digest auth is enough easier to adopt than a client-side security system that it's worth improving digest auth anyway, but if, as Deomid says, "every single piece of equipment i've encountered that has a management web UI uses insecure authentication, either basic or simple form", that's not actually happening.

The lack of a TLS mechanism to talk to non-public devices is definitely a hole in the platform.

Jeffrey

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwr1XJbEU-yWbe2Whx%2Bago2njJFg-gOOdKzEj0%3DGVzP%3D0g%40mail.gmail.com.

--

Deomid "rojer" Ryabkov

ro...@rojer.me

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwof4uGQYUB%3Dac00NisuQG%3Di1JrDT7BcvJtPzMReWcZyxQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABFmFwryoCQ8j4sfPFh3Ozs%3DKaMDt0xEHFMo61mQXXzVw0eM%3DQ%40mail.gmail.com.

--

Deomid "rojer" Ryabkov

ro...@rojer.me

--

Deomid "rojer" Ryabkov

ro...@rojer.me

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/04fea32d-6e9f-44ac-8204-4ee6e7376eddn%40chromium.org.

[Daniel Bratell]

I think shipping SHA-256, the hashing function already supported by Mozilla would be a trivial step. It might be tempting to wrinkle one's nose at people using such archaic things as Digest Auth, but I think giving those that are still using Digest Auth a way to avoid MD5 is an unqualified improvement, especially when it makes Chromium more compatible.

Expanding to SHA-512-256 could, IMHO, come later or as a seperate intent, just to keep the trivial step trivial.

/Daniel

[Deomid "rojer" Ryabkov]

Thanks, Daniel!

I agree, SHA-256 support is non-controversial.

What about username hashing? Again, it should be of no practical consequence on introduction as it's only used if explicitly advertised by the server but Firefox doesn't support it at present, so with that we'd be putting one foot forward.

It does have the practical benefit of concealing the username, if supported, so it is an improvement and is trivial in terms of code (literally 3 lines).

[Deomid "rojer" Ryabkov]

ping?

[Yoav Weiss]

LGTM1 to ship the 256 variant that Firefox are already shipping, as that part seems non-controversial. Also LGTM to hash to username as well, even if that part is not supported by Firefox.

While I agree with David that we shouldn't encourage folks to send auth credentials over non-secure networks, this change does seem like an improvement in practice (even if it's insufficient).

It seems to me that if we wanted to encourage internal-network web applications (e.g. management UIs) to use TLS, we should provide easy means for them to do so. That seems like a separate effort.

[Daniel Bratell]

LGTM2, the subset outlined by Yoav

/Daniel

[Mike Taylor]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/93016ecf-4755-58e0-a1d7-b35ba8ddc6ac%40gmail.com.

[Deomid "rojer" Ryabkov]

ok, thank you. i've updated https://chromium-review.googlesource.com/c/chromium/src/+/4611879 to disable SHA-512-256.

i didn't remove the code completely so it can be re-enabled easily if needed.

there's also a flag to disable it if necessary and is ready for review.

[David Benjamin]

I noticed an issue with the SHA-512-256 variant. Omitting it is definitely the right move, and we probably need to make that stance permanent. Those auth types will likely forever be an interop nightmare and probably should be considered lost.

The specification cites SHA-512/256. SHA-512/256 is a 256-bit hash function based on SHA-512. Confusingly, the value on the wire is "SHA-512-256", but the hash function is normally named "SHA-512/256" and the spec never seems to explicitly say one maps to the other, but that's clearly the intent of the text.

https://www.rfc-editor.org/rfc/rfc7616.html#section-3.2

However the test vectors are not for SHA-512/256. They are for SHA-512 truncated down to 256 bits. This is not the same thing. SHA-512/256 has a different initial state from SHA-512. (SHA-384 and SHA-512 have the same relationship. SHA-384 is SHA-512 with different initial state and then truncated. Just the name "SHA-256" already refers to its own hash function, not based on SHA-512, so SHA-512/256 needed a longer name.) Likely the reference implementation that generated those test vectors did not realize this and simply truncated SHA-512./

Looks like someone noticed this in 2016 and filed an errata, but the WG was closed shortly afterwards and no one did anything about it. The errata remains untriaged.
https://mailarchive.ietf.org/arch/msg/http-auth/mSghM0Y8akYjhygZFUYJHdgeMvg/
https://mailarchive.ietf.org/arch/msg/http-auth/oZwVQ1vNhqAMi8gfCE6x0zzgLm0/
https://mailarchive.ietf.org/arch/msg/http-auth/D3HNF8ftO-Mq_tMePcX9CrBdkpo/
https://mailarchive.ietf.org/arch/msg/http-auth/09-9tYS4hvI4C6Uhv1GEWSZ35W4/

https://www.rfc-editor.org/errata/rfc7616

The SHA-256 one is fine and probably shippable, but this does not appear to be a healthy mechanism. Given the lack of any actual security benefits here, and the inconsistency with our stance throughout the web platform, I don't think it's worth the effort to try to save the SHA-512/256 one. (I also don't think it's worth shipping the SHA-256, but don't care strongly enough to stand in the way. :-) Like I said, this legacy mechanism is just not worth spending time on.)

David

[Mike Taylor]

Thank you David for sharing your expert opinion on SHA-512-256 (or SHA-512/256 or whatever). :)

[Deomid "rojer" Ryabkov]

yeah, was a surprise to me too, great detective work!

the simple truncation did seem weird to me (like - why?) but it passed the test vector and i hadn't looked into it further.

David if you're reading this, can you help me land the change? being an outsider, i cannot trigger the CQ.