Server connection pool

1,327 views
Skip to first unread message

Max

unread,
Oct 3, 2013, 6:25:47 PM10/3/13
to golan...@googlegroups.com
I know http.Client has connection pool in type Transport

But there is no Transport in http.Server

I wish to access server's pool. 

1. List all connections
2. Close connections/remove them from pool

Thanks

Max

unread,
Oct 4, 2013, 10:22:15 AM10/4/13
to golan...@googlegroups.com
Seems I made the bad subject. I had to say http.Server instead.

Update

I am learning Go source code https://code.google.com/p/go/source/browse/src/pkg/net/http/server.go#1087

It seems that this function has the answer that there is no Pool.

May be someone knows how to override that function? Do I have to create a hole copy of package?

chris dollin

unread,
Oct 4, 2013, 11:50:39 AM10/4/13
to Max, golang-nuts
What actual problem are you trying to solve?

Chris

--
Chris "allusive" Dollin

Max

unread,
Oct 4, 2013, 12:15:31 PM10/4/13
to golan...@googlegroups.com, Max, ehog....@googlemail.com
When I run Go Webserver with default settings then some connections are not closed. (never)
After 30 days server keeps more then 1000 connections and stop to work

lsof -n -p 12633 | wc -l

So if I create server with 

http1 := &http.Server{
Addr:           ":8080",
Handler:        nil,
ReadTimeout:    20 * time.Second,
WriteTimeout:   20 * time.Second,
MaxHeaderBytes: 1 << 15,
}


Fix that problem.

Usually I have 30-50 connections now.

But what will happen if load goes up?

My server should not open more connections then it is allowed to do.

I wish to make code that will make timeout less when server will have 500 connections.
even more less when server has 700 connections.
Server should close 10 oldest connections when 810 connections is opened.

I wish to count number of requests per connection.
I wish to give unique ID to every connection.

I wish to make report like lighttpd Mod_Status module does
Here is example from google http://kiserai.net/server-status

From report I wish to be able to close connections as well. 

Kyle Lemons

unread,
Oct 4, 2013, 4:20:32 PM10/4/13
to Max, golang-nuts, chris dollin
It sounds like you want to implement your own http.RoundTripper.  I don't think you can get everything you want with what you're given in http.Transport.


--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Max

unread,
Oct 4, 2013, 4:38:22 PM10/4/13
to golan...@googlegroups.com, Max, chris dollin

It sounds like you want to implement your own http.RoundTripper.  I don't think you can get everything you want with what you're given in http.Transport.


I have not noticed that http.Server uses http.RoundTripper. Does it ?

Kyle Lemons

unread,
Oct 4, 2013, 5:12:04 PM10/4/13
to Max, golang-nuts, chris dollin
Oh, right, server.  You'll want to make your own net.Listener that you give to Server.Serve, probably.


On Fri, Oct 4, 2013 at 1:38 PM, Max <max.se...@gmail.com> wrote:

It sounds like you want to implement your own http.RoundTripper.  I don't think you can get everything you want with what you're given in http.Transport.


I have not noticed that http.Server uses http.RoundTripper. Does it ?

--

Max

unread,
Oct 4, 2013, 7:17:35 PM10/4/13
to golan...@googlegroups.com, Max, chris dollin

Oh, right, server.  You'll want to make your own net.Listener that you give to Server.Serve, probably.




Thank You, Kyle! That might work. It will not allow me to do everything I wish but at least I can count connections and may be close old connections when server hit high load.


What is a the best way to modify http packge ?
I am debating to create http2 copy of http and modify `func (c *conn) serve() {`
Is that good idea?
Kyles solution resolves 90% of my problems.

Tango

unread,
Oct 4, 2013, 7:57:02 PM10/4/13
to golan...@googlegroups.com, Max, chris dollin

LimitListener?

Kyle Lemons

unread,
Oct 4, 2013, 8:09:05 PM10/4/13
to Max, golang-nuts, chris dollin
What is the other 10%?  Do you know about Hijacker?

The http server is one package that I don't suggest forking, as it is likely to evolve and grow and you probably don't want to maintain a patch set.


--

Max

unread,
Oct 5, 2013, 1:09:06 PM10/5/13
to golan...@googlegroups.com, Max, chris dollin
LimitListener is good point for start. Thank You! It is very helpful. 

On Saturday, October 5, 2013 2:57:02 AM UTC+3, Tango wrote:

LimitListener?

Max

unread,
Oct 5, 2013, 1:11:39 PM10/5/13
to golan...@googlegroups.com, Max, chris dollin
No. Hijack will not help me.

But I think I can pass query info from handler and cast net.Conn to my own my.Conn
So I will be able to count requests per connection and I will know the last connection.

Thanks
Reply all
Reply to author
Forward
0 new messages