Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Reverse Proxies: It's been a while since someone mentioned them :)
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  10 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
OmarShariffDontLikeIt  
View profile  
 More options Apr 26 2012, 9:58 am
From: OmarShariffDontLikeIt <omarshariffdontlik...@gmail.com>
Date: Thu, 26 Apr 2012 06:58:45 -0700 (PDT)
Local: Thurs, Apr 26 2012 9:58 am
Subject: Reverse Proxies: It's been a while since someone mentioned them :)
Hi Gophers!

Messing about with a small project: it's basically a reverse proxy
that will convert json responses from the origin server to lovely html
in the client request. It decodes the json response from the origin
and passes this struct to the mustache templating engine which outputs
html.

The server has a config file allowing the user to specify url patterns
to match and destination urls to proxy to:
{
   "pattern":"/hello",
   "origin":"http://example.com/some/destination",
   "template":"./hello.template"

} // etc, etc

I'm having difficulty understanding our friend
httputil.NewSingleHostReverseProxy(). It may have something to do with
the way I have approached this though.

Ideally I have a http server. This assigns a handler for each entry in
the config file, matched by pattern. The handler creates a new proxy
server using httputil.NewSingleHostReverseProxy() with the origin
config parameter as the target argument. I then have a custom
http.RequestWriter that reads the json body in response, parses it,
passes it to mustache and writes the output:

...
        for _, route := range config.Routes {
                url, _ := url.Parse(route.origin)
                MustacheHandler := NewMustacheHandler(route.TemplateCache, url)
                http.Handle(route.Pattern, MustacheHandler)
        }
        err = http.ListenAndServe(address, nil)
        if err != nil {
                log.Fatal("ListenAndServe: ", err)
        }
...

The mustache handler looks a little bit like this:

func NewMustacheHandler(template []byte, url *url.URL)
http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        mw := NewMustacheWriter(w, template)
        rw := mustacheResponseWriter{Writer: mw, ResponseWriter: w}

        proxy := httputil.NewSingleHostReverseProxy(url)
        proxy.ServeHTTP(rw, r)
        }

}

And the mustache writer looky likey thisy:

type MustacheWriter struct {
        Template []byte
        w io.Writer

}

func (w MustacheWriter) Write(b []byte) (int, error) {
        var f interface{}
        err := json.Unmarshal(b, &f)
        if err != nil {
                        log.Fatal(err, string(b))
        }

        m := f.(map[string]interface{})
        d := mustache.Render(string(w.Template), m)

    return w.Write([]byte(d))

}

The problem is that I keep getting "File not found" passed into the
Write(b []byte) argument, even though if I curl the URL being passed
as the origin I get valid JSON back.

So, questions!

1) Am I using NewSingleHostReverseProxy() correctly?
2) Is there a problem with my approach to this problem?
3) Is there a better way?

Cheers!
Ben


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kyle Lemons  
View profile  
 More options Apr 26 2012, 1:50 pm
From: Kyle Lemons <kev...@google.com>
Date: Thu, 26 Apr 2012 10:50:36 -0700
Local: Thurs, Apr 26 2012 1:50 pm
Subject: Re: [go-nuts] Reverse Proxies: It's been a while since someone mentioned them :)

On Thu, Apr 26, 2012 at 6:58 AM, OmarShariffDontLikeIt <

The actual URL is created by combining r with url.  Just do an http.Get on
URL unless you actually want a reverse proxy for a directory or something.

2) Is there a problem with my approach to this problem?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
OmarShariffDontLikeIt  
View profile  
 More options Apr 26 2012, 2:13 pm
From: OmarShariffDontLikeIt <omarshariffdontlik...@gmail.com>
Date: Thu, 26 Apr 2012 11:13:20 -0700 (PDT)
Local: Thurs, Apr 26 2012 2:13 pm
Subject: Re: Reverse Proxies: It's been a while since someone mentioned them :)
Ah, but a want a FULL HTTP proxy, passing though POSTs, GET, DELETE
etc;

Are you suggesting that NewSingleHostReverseProxy is not suitable for
this task? I may have misunderstood but I thought that this is exactly
what NewSingleHostReverseProxy is for.

On Apr 26, 6:50 pm, Kyle Lemons <kev...@google.com> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kyle Lemons  
View profile  
 More options Apr 26 2012, 3:27 pm
From: Kyle Lemons <kev...@google.com>
Date: Thu, 26 Apr 2012 12:27:20 -0700
Local: Thurs, Apr 26 2012 3:27 pm
Subject: Re: [go-nuts] Re: Reverse Proxies: It's been a while since someone mentioned them :)

Yes, that's what it does.

You said that you can CURL the route URL, so I assumed that the route URL
was the one that was returning the JSON.  The route URL is being modified
by ReverseProxy, so that CURL is not going to reproduce the behavior.  I
believe you can wrap the default Director with your own and log the new
path if you want to see what it's doing.

On Thu, Apr 26, 2012 at 11:13 AM, OmarShariffDontLikeIt <


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
OmarShariffDontLikeIt  
View profile  
 More options Apr 26 2012, 4:02 pm
From: OmarShariffDontLikeIt <omarshariffdontlik...@gmail.com>
Date: Thu, 26 Apr 2012 13:02:34 -0700 (PDT)
Local: Thurs, Apr 26 2012 4:02 pm
Subject: Re: Reverse Proxies: It's been a while since someone mentioned them :)

> The route URL is being modified by ReverseProxy

Ah, interesting! Why does it do that!?

> believe you can wrap the default Director with your own and log the new
> path if you want to see what it's doing.

Ooo that sounds just what I need to do to work out the problem. Could
you give me an example of what you mean?

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Kyle Lemons  
View profile  
 More options Apr 26 2012, 4:28 pm
From: Kyle Lemons <kev...@google.com>
Date: Thu, 26 Apr 2012 13:28:08 -0700
Local: Thurs, Apr 26 2012 4:28 pm
Subject: Re: [go-nuts] Re: Reverse Proxies: It's been a while since someone mentioned them :)

On Thu, Apr 26, 2012 at 1:02 PM, OmarShariffDontLikeIt <

omarshariffdontlik...@gmail.com> wrote:

> > The route URL is being modified by ReverseProxy
> Ah, interesting! Why does it do that!?

From the documentation: "NewSingleHostReverseProxy returns a new
ReverseProxy that rewrites URLs to the scheme, host, and base path provided
in target. If the target's path is "/base" and the incoming request was for
"/dir", the target request will be for /base/dir."

So, thus, if your route is http://example.org/generate/gophers.js and the
request is for http://localhost:4242/gophers.js the outgoing URL is
http://example.org/genrate/gophers.js/gophers.js which is probably not what
you intended :).

> > believe you can wrap the default Director with your own and log the new
> > path if you want to see what it's doing.
> Ooo that sounds just what I need to do to work out the problem. Could
> you give me an example of what you mean?

I think it looks something like

shrp := httputil.NewSingleHostReverseProxy(url)
director := shrp.Director
shrp.Director = func(req *http.Request) {
   director(req)
   log.Printf("Redirected URL: %s", req.URL)


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
OmarShariffDontLikeIt  
View profile  
 More options Apr 26 2012, 5:19 pm
From: OmarShariffDontLikeIt <omarshariffdontlik...@gmail.com>
Date: Thu, 26 Apr 2012 14:19:01 -0700 (PDT)
Local: Thurs, Apr 26 2012 5:19 pm
Subject: Re: Reverse Proxies: It's been a while since someone mentioned them :)
Aha! That makes perfect sense! I will try that out tmrw.

Thanks for your help.

Cheers,
Ben


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
d...@fit.ly  
View profile  
 More options Apr 26 2012, 7:29 pm
From: d...@fit.ly
Date: Thu, 26 Apr 2012 16:29:26 -0700 (PDT)
Local: Thurs, Apr 26 2012 7:29 pm
Subject: Re: Reverse Proxies: It's been a while since someone mentioned them :)

You could have a look at falcore (github.com/ngmoco/falcore).  One of the
core things we built it for was reverse proxying into other applications.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
OmarShariffDontLikeIt  
View profile  
 More options Apr 27 2012, 3:19 am
From: OmarShariffDontLikeIt <omarshariffdontlik...@gmail.com>
Date: Fri, 27 Apr 2012 00:19:49 -0700 (PDT)
Local: Fri, Apr 27 2012 3:19 am
Subject: Re: Reverse Proxies: It's been a while since someone mentioned them :)
> You could have a look at falcore (github.com/ngmoco/falcore).  One of the
> core things we built it for was reverse proxying into other applications.

Having a quick read, I must say I really do like the pipelining of
filters. It reminds me a lot of Apaches filters (without the fiddly
mess!).

After a brief look, I can't find anything relating to actually
proxying a request to a back end. Is this built in, a filter, or would
I have to write this myself?

Thanks for the heads up on Falcore, I takes an approach I am familiar
with (filters/pipeline) so I may end up switching my project over to
this approach as it would result in a much cleaner architecture,
especially as I have several features I would like to add in future
that would be trivial to add as additional filters.

Cheers!
Ben


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
OmarShariffDontLikeIt  
View profile  
 More options Apr 27 2012, 7:43 am
From: OmarShariffDontLikeIt <omarshariffdontlik...@gmail.com>
Date: Fri, 27 Apr 2012 04:43:47 -0700 (PDT)
Local: Fri, Apr 27 2012 7:43 am
Subject: Re: Reverse Proxies: It's been a while since someone mentioned them :)
> After a brief look, I can't find anything relating to actually
> proxying a request to a back end. Is this built in, a filter, or would
> I have to write this myself?

Just found it; the upstream dir contains a upstream/proxy filter.

Cheers,
Ben


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »