Some clarity on SCRAM-SHA1?

[Matt Prelude]

I'm implementing SCRAM-SHA1 in order to update the Common Lisp mongo driver.

I can't seem to find a few things in the specification, regarding the client final message.

Particularly:

1. Many parts of this call for creating a HMAC digest.
   These digests are later fed into other HMAC & SHA functions.
   Should these digests be binary, decimal, hex? Obviously SHA("16") isn't the same as SHA("10"). 
2. When generating the client proof, I need to do clientKey XOR clientSignature.
   Presumably this means XORing the raw numbers, but what format should the output be as this is base64-encoded?

[Craig Wilson]

Hi Matt,

I assume you are looking at the specification here (https://github.com/mongodb/specifications/blob/master/source/auth/auth.rst#plain) and the assocaited RFC here: http://tools.ietf.org/html/rfc4616.

1) I believe it's SHA("1"), as in the name. They are computed on bytes. 

2) This needs to be using the bytes. The output would also be in bytes. Once you have the XORed bytes, it will get converted into Base64 and assigned to the "p" key.

All of the MongoDB drivers already support SCRAM-SHA1, so it might be easiest to read some of their source code to see how it's done. I'm not sure of another language you are comfortable with, so if you can tell me, I can point you in the right direction. .NET's is here: https://github.com/mongodb/mongo-csharp-driver/blob/master/src/MongoDB.Driver.Core/Core/Authentication/ScramSha1Authenticator.cs#L145. We've simply used the same function names as exist in the RFC so hopefully it's easy enough to follow.

Craig

[Matt Prelude]

Hi Craig, thanks for the fast reply.

I'm quite familiar with C, C++, PHP & Ruby. I've read the RFC, but I seem not to have implemented it correctly.

This is my implementation so far: https://github.com/mprelude/cl-scram

The RFC says:

   This is a simple example of a SCRAM-SHA-1 authentication exchange
   when the client doesn't support channel bindings (username 'user' and
   password 'pencil' are used):

   C: n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL
   S: r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,
      i=4096
   C: c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,
      p=v0X8v3Bz2T0CJGbJQyF0X+HI4Ts=
   S: v=rmF9pqV8S7suAoZWja4dJRkFsKQ=

But when I try using the example response to generate a final response, I get:

   * (cl-scram:gen-client-final-message :password "pencil" :client-nonce "fyko+d2lbbFgONRv9qkxdawL" :client-initial-message "n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL" :server-response "r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096")

   ((CL-SCRAM::SERVER-SIGNATURE

     . #(33 115 21 228 67 190 35 238 223 122 117 125 222 242 209 136 175 228 67

         151))

    (CL-SCRAM::FINAL-MESSAGE 

     . 

     "c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,p=MTQ0MDkzNTE3MzIzNzQ3MDgzMTE2Mzk5NDU4MjQ1NTY1NDkxMTczMDk5MjU4NDgw"))

Unfortunately, the RFC example doesn't specify what each intermediate variable should be as we generate the message, so it's hard for me to see exactly where I'm going wrong.

[Matt Prelude]

Just adding some more info, this is what I have for the intermediate vars, in case it allows you to see which step is wrong:

* (cl-scram:gen-client-final-message :password "pencil" :client-nonce "fyko+d2lbbFgONRv9qkxdawL" :client-initial-message "n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL" :server-response "r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096")

((CL-SCRAM::SERVER-SIGNATURE

  . #(33 115 21 228 67 190 35 238 223 122 117 125 222 242 209 136 175 228 67

      151))

 (CL-SCRAM::SERVER-KEY

  . #(15 224 146 88 179 172 133 43 165 2 204 98 186 144 62 170 205 191 125 49))

 (CL-SCRAM::CLIENT-PROOF

  . #*1100100111101011000000111010100000010101011001000101011100110001111100001100100010001101001000110101001010101010001011111000100011100001001110100001001110000)

 (CL-SCRAM::CLIENT-SIGNATURE

  . #(251 9 164 14 244 111 236 112 227 116 148 143 243 255 231 75 58 114 21

      88))

 (CL-SCRAM::AUTH-MESSAGE

  . "n,,n=user,r=fyko+d2lbbFgONRv9qkxdawL,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,s=QSXCR+Q6sek8bf92,i=4096,c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j")

 (CL-SCRAM::STORED-KEY

  . #(233 217 70 96 195 157 101 195 143 186 217 28 53 143 20 218 14 239 43

      214))

 (CL-SCRAM::CLIENT-KEY

  . #(226 52 196 123 246 195 102 150 221 109 133 43 153 170 162 186 38 85 87

      40))

 (CL-SCRAM::SALTED-PASSWORD

  . #(29 150 238 58 82 155 90 95 158 71 192 31 34 154 44 184 166 225 95 125))

 (CL-SCRAM::FINAL-MESSAGE-BARE

  . "c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j")

 (CL-SCRAM::FINAL-MESSAGE

  . "c=biws,r=fyko+d2lbbFgONRv9qkxdawL3rfcNHYJY1ZVvWVs7j,p=IyoxMTAwMTAwMTExMTAxMDExMDAwMDAwMTExMDEwMTAwMDAwMDEwMTAxMDExMDAxMDAwMTAxMDExMTAwMTEwMDAxMTExMTAwMDAxMTAwMTAwMDEwMDAxMTAxMDAxMDAwMTEwMTAxMDAxMDEwMTAxMDEwMDAxMDExMTExMDAwMTAwMDExMTAwMDAxMDAxMTEwMTAwMDAxMDAxMTEwMDAw"))

[Craig Wilson]

Hi Matt,

First, I posted the wrong links above (both to our specification and the RFC. These are correct: https://github.com/mongodb/specifications/blob/master/source/auth/auth.rst#scram-sha-1 and http://tools.ietf.org/html/rfc5802.

Couple of things that may or may not be of value. If you are still having issues, I can step through my code and capture all the intermediate values.

https://github.com/mprelude/cl-scram/blob/master/src/scram.lisp#L51

Why are you base64 encoding the client-initial-message? How are you using saslStart? This might be ok, just want to confirm.

https://github.com/mprelude/cl-scram/blob/master/src/scram.lisp#L23

The password here is not the plain text password, but rather a special mongo hashed password. This particular function is defined here: https://github.com/mongodb/specifications/blob/master/source/auth/auth.rst#scram-sha-1 relating to how to compute the mongo_hashed_password and, subsequently, the salted password.

You are using ascii-string-to-byte-array in a number of places. It might or might not matter, but UTF-8 is the proper encoding.

https://github.com/mprelude/cl-scram/blob/master/src/scram.lisp#L32

This should be the client-initial-message-bare. It should not include the "n,,"

Craig

[Matt Prelude]

Hi Craig,

I managed to get the library working based on Maarten Bodewes reply to my StackOverflow thread.

The intermediate values were a massive help as they gave me something to compare to.

Thanks for your assistance. Next up: implementing Mongo-specific stuff in cl-mongo library.

[Matt Prelude]

http://stackoverflow.com/questions/32468600/implementing-scram-sha1-client-getting-it-wrong-somewhere

[Craig Wilson]

Awesome. Ping back if you need some more help.

[Matt Prelude]

Trying to do the mongo-specific part now -- I've forked cl-mongo from fons as he no longer appears to maintain it.

This is the old login code:

(let* ((nonce (get-element "nonce" (car (docs (db.run-command 'getnonce :mongo mongo)))))

       (pwd (concatenate 'string username ":mongo:" password))

       (md5-pwd (hex-md5 pwd))

       (md5-pwd-str (ironclad:byte-array-to-hex-string md5-pwd))

       (md5-key (hex-md5 (concatenate 'string nonce username md5-pwd-str)))

       (md5-key-str (ironclad:byte-array-to-hex-string md5-key))

       (request (kv (kv "authenticate" 1)

                    (kv "user" username)

                    (kv "nonce" nonce)

                    (kv "key" md5-key-str)))

       (retval (get-element "ok" (car (docs (db.find "$cmd" request :limit 1 :mongo mongo))))))

  (if retval t nil)))

I'm trying to, based on this, implement sending the initial message.

I'm struggling (again) with the binary data types:

(let* ((nonce (cl-scram:gen-client-nonce))

       (pwd (concatenate 'string username ":mongo:" password))

       (md5-pwd (hex-md5 pwd))

       (md5-pwd-str (ironclad:byte-array-to-hex-string md5-pwd))

       (initial-message (cl-scram:gen-client-initial-message :username username

                                                             :nonce nonce))

       (request (kv (kv "saslStart" 1)

                    (kv "mechanism" "SCRAM-SHA-1")

                    (kv "payload"

                        (bson-binary :generic (ironclad:ascii-string-to-byte-array

                                                (cl-scram:base64-encode initial-message))))))

       (response (car (docs (db.find "$cmd" request :limit 1 :mongo mongo))))

       (retval (pairlis '(errmsg ok code message binary-message)

                         (list (get-element "errmsg" response)

                               (get-element "ok" response)

                               (get-element "code" response)

                               initial-message

                               (ironclad:ascii-string-to-byte-array (cl-scram:base64-encode initial-message))))))

            (list request retval)))

Because this is a binary communication, I'm not 100% sure how best to represent the data.

This is what's being sent:

saslStart: 1

mechanism: SCRAM-SHA-1

payload: Binary (type 0x00) (#(98 105 119 115 98 106 49 104 100 88 74 49 98 88 74 51 76 72 73 57 83 87 116 122 101 84 100 78 101 71 100 97 90 71 52 53 85 69 86 113 87 108 104 85 89 108 78 75 89 106 74 80 79 87 78 84 99 49 108 84 82 68 99 61))

The payload is "n,,n=aurumrw,r=Iksy7MxgZdn9PEjZXTbSJb2O9cSsYSD7", first base64-encoded, then converted to octets.

If you have any suggestions, please let me know.

[Matt Prelude]

Have added more details here: http://stackoverflow.com/questions/32486579/implementing-mongodb-sasl-authentication-in-cl-mongo