Capture Outgoing Port for a HTTP Request

[ryan...@gmail.com]

Is it possible to capture the outgoing port for a given HTTP request?

I'm using a knockoff of ab that I wrote in go to send repeated requests to a given web service. Sometimes we get an error and I want to look at a packet trace of it. The problem is it's really hard to find one failed request in 1,000 in a tcp dump. If I can see the source port, that would help me narrow it down.

The code I'm doing is effectively this (forgive any typos, this is a quick & dirty recopy, not a cut & paste):

tlsConfig := &tls.Config{
    InsecureSkipVerify: true,
}

transport := &http.Transport{
    DisableKeepAlives: true,
    TLSClientCOnfig: tlsCOnfig,
    ResponseHeaderTimeout: time.Duration(headerTimeout) * time.Second,
}

client := &http.Client{
    Timeout: time.Duration(timeOut) * time.second,
    Transport: transport,
}

response, err :=client.Get(*targetURL)    // How can I capture the outgoing port from this?

[urji...@gmail.com]

Hi Ryan,

You can get it via httptrace (https://blog.golang.org/http-tracing)

Example:

  req, _ := http.NewRequest("GET", "http://example.com", nil)

  trace := &httptrace.ClientTrace{

              GotConn: func(connInfo httptrace.GotConnInfo) {

                              fmt.Printf("Got Conn: %s\n", connInfo.Conn.LocalAddr().String()) <------------------------- This has the local outgoing port

              },

   }

  req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))

  if _, err := http.DefaultTransport.RoundTrip(req); err != nil {

             log.Fatal(err)

  }

[ryan...@gmail.com]

That was exactly what I was looking for. Thank you!

[ryan...@gmail.com]

One problem came up. Sorry for the false positive.

How can I capture this information for a request which failed to connect?

If a connection times out with, say "context deadline exceeded (Client.Timeout exceeded while awaiting headers)", the local outgoing port is not captured.

I'm not seeing anything in the httptrace library for for a "FailedConnInfo" equivalent to "GotConnInfo." Am I overlooking something?

On Thursday, October 8, 2020 at 1:12:39 PM UTC-5 urji...@gmail.com wrote:

[Urjit Singh Bhatia]

It depends on how far down the connection process it reached before timing out...
What is the timeout setting your http client?

Try this example:

func main() {

req, _ := http.NewRequest("GET", "https://deelay.me/300/https://google.com", nil)

trace := &httptrace.ClientTrace{

GetConn: func(hostPort string) {

fmt.Printf("Get Conn: %s\n", hostPort)

},

GotConn: func(connInfo httptrace.GotConnInfo) {

fmt.Printf("Got Conn: %s\n", connInfo.Conn.LocalAddr().String())

},

ConnectStart: func(network, addr string) {

fmt.Printf("Conn start: %s %s\n", network, addr)

},

ConnectDone: func(network, addr string, err error) {

fmt.Printf("Conn done: %s %s\n", network, addr)

},

}

req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))

client := http.Client{

Timeout: time.Millisecond * 250,

}

if _, err := client.Do(req); err != nil {

log.Fatal(err)

}

}

-----------------------
Output:

Get Conn: deelay.me:443

Conn start: tcp 167.99.174.142:443

Conn done: tcp 167.99.174.142:443

Got Conn: 192.168.0.103:52450

2020/10/14 14:11:04 Get "https://deelay.me/300/https://google.com": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

--------------------

If you play with the timeout value and set the timeout on the client to be too small, GotConn is not hit because the timeout triggers before acquiring a connection for example due to a TLS handshake delay etc.

>>> I'm not seeing anything in the httptrace library for for a "FailedConnInfo" equivalent to "GotConnInfo." Am I overlooking something?

the GotConn documentation says that for errors look for the error returned by the roundtrip call

[ryan...@gmail.com]

My timeout is configurable, but I usually set it to 5 or 10 seconds. Plenty of time for a TLS handshake.

Based on the code you gave me, increasing the timeout to 5 seconds, and changing the target endpoint to match the behavior I'm seeing, this is the output.

Get Conn: <<servername>>:<<port>>
Conn start: tcp <<ipaddress>>:<<port>>
Conn done: tcp  <<ipaddress>> :<<port>>
2020/10/14 21:30:40 Get "https:// <<servername>> :<<port>> ": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

It looks like it's not getting far enough in the connection process to capture what I am hoping to capture.