How to negotiate authentication over HTTP ? (the Go way)

[mi...@ubo.ro]

I find myself in need to handle various authentication schemes such "proprietary" oauth2 schemes and NTLM. The easy and clean way to do it (API wise) would be using a http.RoundTripper but looks like it forbids you from even reading the response headers.

 In the end I just made a ``func DoHTTPRequestWithAuthenticationHandling(cl *http.Client, req *http.Request)(*http.Response, error)`` function that just wraps  net/http.Client.Do(), clones the request and response if it's necessary and negotiates the authentication scheme. It's basically what I wanted to do within http.RoundTripper except now the user of the http.Client needs to always remember to execute the requests the right way (i.e. using a different function) instead of http.Do.

Is there a better way to do it? Would it be a good idea  in Go 2.0  make http.Client an interface to prevent this kind of limitation/workarounds?

[ben...@gmail.com]

I'm not sure what these proprietary auth schemes look like (and don't know much about oauth2 or NTLM), but for many kinds of auth you'd just set headers in the request and read them in the body. For example:

    request, err := http.NewRequest("GET", "https://httpbin.org/get", nil)

    // handle err

    request.Header.Set("X-My-Auth", "abcd1234")
    response, err := client.Do(request)

    // handle err

    // handle response

Runnable code example here:

https://play.golang.org/p/cocv1avzNCo

And of course you can roll your own function to create a new request with auth headers already applied. Would this kind of thing work for you?

-Ben

[mi...@ubo.ro]

Thanks for your help. I'm aware I can set the headers that way but the authentication transport also needs to inspect the response headers and optionally re-submit the request based on the headers received. That's part of the "negotiation" mechanism. That's because we don;t know what authentication scheme is supported by the remote party until we receive the actual response/headers from a blind request. 

Below is a simple use case:

  - the http client executes a reques with no authentication.

  - we receive a status code 403 along with a www-header indicating that oauth2 authentication is supported. 

   - we resend the request with appropriate oauth2 headers.

Note that we had to read the response code and response headers so that we can re-send the request.  This is the part that I wish I could automate using the http.RoundTripper. 

 The alternative is to either handle the response manually after each request or create a custom endpoint < i.e. func ExecuteHTTP(*http.Client, *http.Request)(*http.Response, error) >  that handles the response headers and retries the requests using the proper authentication headers.

In both cases you loose the flexibility to use a http client that does all this in the background. 

Some packages (i.e ElasticSearch package) support configuration using your own http client. However you cannot pass a custom `ExecuteHTTP` function to ElasticSearch so it becomes quite hard to hack the authentication/negotiation.

- Mihai.

[mi...@ubo.ro]

That would work great but the documentation on RoundTriper specifically forbids it[0]. Unless I get it wrong(do I?) you are not allowed to read the response within RoundTrip. The request needs to be .Clone-ed as well but that's not an issue.


[0] https://pkg.go.dev/net/http#RoundTripper  
// RoundTrip should not attempt to interpret the response. In 

// particular, RoundTrip must return err == nil if it obtained

// a response, regardless of the response's HTTP status code. 

// A non-nil err should be reserved for failure to obtain a

// response. Similarly, RoundTrip should not attempt to

// handle higher-level protocol details such as redirects, 

// authentication, or cookies.

// RoundTrip should not modify the request

On 27/10/2021 02:09, Ben Hoyt wrote:

Oh, I see. An "ExecuteHTTP" function seems reasonable, but yeah, if you need an *http.Client, I think you have to customize the Transport/Roundtripper. You mention in your initial email that RoundTripper "forbids you from even reading the response headers", but I don't think that's the case, right?

For example, here's a custom "AuthTransport":

type AuthTransport struct {
     AuthHeader string
}

func (t *AuthTransport) RoundTrip(request *http.Request) (*http.Response, error) {
     request.Header.Set("X-My-Auth", t.AuthHeader)
     response, err := http.DefaultTransport.RoundTrip(request)
     if err != nil {
         return nil, err
     }
     if response.StatusCode == 403 || response.Header.Get("WWW-Authenticate") != "" {
         fmt.Println("Would retry here")
     }
    return response, nil
}

and then you'd instantiate your http.Client like so:

client := &http.Client{
    Timeout:   5 * time.Second,
    Transport: &AuthTransport{AuthHeader: "abcd1234"},
}

Did that not work for you?

-Ben

[Ben Hoyt]

Ah, thanks for that info -- glad somebody is reading the docs ... :-) Seems like you're well ahead of me here. I'd be interested to hear what others have done, or what you end up with. -Ben

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/nTQ7F4b_wvI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a9a6f788-6a45-49cf-9381-4abb98888864n%40googlegroups.com.

[Amnon]

The design of the client side of net/http is not great, and requires a lot of boilerplate.

I don't know of any approach which is better than your DoHTTPRequestWithAuthenticationHandling (though I would have 
chosen a very much shorter name). 

Every request to a service  would normally require payload serialisation, authentication, error handling, response code checking,
and response unmarshalling. I normally encapsulate all of these in a single method which is called by the functions which are used to 
call the endpoints. 

I think it is generally accepted that the net/http needs some love, from the client point of view.
But it is solid and well tested. We have got used to its quirks and have learned to live with it, and nobody has yet come up with a compelling alternative.

Perhaps we should try to convince Kenneth Reitz to switch to Go?