best way to get user's IP address in pylons?

[kevin]

hello,

hope all is well.

i'm wondering what the best way to get a user's IP address is?
because the reCaptcha verification wants the remote IP.

I know I can get request.host and socket.gethostbyname can resolve
host to IP.

but i'm looking for something else first, maybe a Pylons/WSGI field
for IP, because resolution is relatively expensive at scale.

thanks,
kevin

[Chris AtLee]

request.environ['REMOTE_ADDR'] does it for me

[Noah Gift]

if you are a javascript guy you can use window.location.hostname to
get the domain name

>
> >
>

--
Noah Gift
http://noahgift.com

[Wichert Akkerman]

Possibly better:

request.environ.get("X_FORWARDED_FOR", request.environ["REMOTE_ADDR"])

that handles requests coming in from a proxy server as well. You may
want add some extra checking to see if REMOTE_ADDR really is a proxy
server before trusting the X-Forwarded-For header.

Wichert.

--
Wichert Akkerman <wic...@wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.

[Jonathan Vanasco]

On Jul 3, 4:42 pm, Wichert Akkerman <wich...@wiggy.net> wrote:
> Possibly better:
>
>   request.environ.get("X_FORWARDED_FOR", request.environ["REMOTE_ADDR"])

maybe someone can make a middleware or pylons patch + config setting
that migrates X_FORWARDED_FOR to REMOTE_ADDR

this is the one i maintain for mod_perl
http://search.cpan.org/dist/Apache2-xForwardedFor/lib/Apache2/xForwardedFor.pm

[Ian Bicking]

I was going to say that PrefixMiddleware already did this, but
apparently it didn't. It does now in Paste Deploy trunk.

--
Ian Bicking : ia...@colorstudy.com : http://blog.ianbicking.org

[Mike Orr]

On Thu, Jul 3, 2008 at 1:42 PM, Wichert Akkerman <wic...@wiggy.net> wrote:
> Possibly better:
>
> request.environ.get("X_FORWARDED_FOR", request.environ["REMOTE_ADDR"])

It's actually HTTP_X_FORWARDED_FOR, not X_FORWARDED_FOR, at least on
my Apache 2.2 server.

--
Mike Orr <slugg...@gmail.com>

[Wichert Akkerman]

Previously Jonathan Vanasco wrote:
>
> On Jul 3, 4:42 pm, Wichert Akkerman <wich...@wiggy.net> wrote:
> > Possibly better:
> >
> >   request.environ.get("X_FORWARDED_FOR", request.environ["REMOTE_ADDR"])
>
> maybe someone can make a middleware or pylons patch + config setting
> that migrates X_FORWARDED_FOR to REMOTE_ADDR

It should not migrate - there is real value in knowing both settings
separately.

[Jonathan Vanasco]

> It should not migrate - there is real value in knowing both settings
> separately.

Agreed that there is real knowledge in knowing both -- however in many
situations having both is confusing and breaks plugins or code

It's up to the developer of an application if they want it to be aware
of an internal proxy or not.

On some projects I may need to know which one of my loadbalancers
something came from. On others I may need to know if there is any
sort of proxying before it gets to me. On most projects though, the
only IP that I care about is the end-user --and I don't care about
information my reverse proxy puts in there. It's much simpler to
write code that only has to deal with REMOTE_ADDR , and only validate
the x-forwarded-for headers -- and possibly stash that elsewhere -- in
something that runs before my application logic begins. in perl i can
do that with specific phases in the apache request process; in pylons
i can do it in middleware.

[Shannon -jj Behrens]

On Thu, Jul 3, 2008 at 2:05 PM, Ian Bicking <ia...@colorstudy.com> wrote:
>
> Jonathan Vanasco wrote:
>> On Jul 3, 4:42 pm, Wichert Akkerman <wich...@wiggy.net> wrote:
>>> Possibly better:
>>>
>>> request.environ.get("X_FORWARDED_FOR", request.environ["REMOTE_ADDR"])
>>
>> maybe someone can make a middleware or pylons patch + config setting
>> that migrates X_FORWARDED_FOR to REMOTE_ADDR
>>
>> this is the one i maintain for mod_perl
>> http://search.cpan.org/dist/Apache2-xForwardedFor/lib/Apache2/xForwardedFor.pm
>
> I was going to say that PrefixMiddleware already did this, but
> apparently it didn't. It does now in Paste Deploy trunk.

I remember submitting a pretty decent patch for this a long time ago.

-jj

--
It's a walled garden, but the flowers sure are lovely!
http://jjinux.blogspot.com/

[Ian Bicking]

Shannon -jj Behrens wrote:
> On Thu, Jul 3, 2008 at 2:05 PM, Ian Bicking <ia...@colorstudy.com> wrote:
>> Jonathan Vanasco wrote:
>>> On Jul 3, 4:42 pm, Wichert Akkerman <wich...@wiggy.net> wrote:
>>>> Possibly better:
>>>>
>>>> request.environ.get("X_FORWARDED_FOR", request.environ["REMOTE_ADDR"])
>>> maybe someone can make a middleware or pylons patch + config setting
>>> that migrates X_FORWARDED_FOR to REMOTE_ADDR
>>>
>>> this is the one i maintain for mod_perl
>>> http://search.cpan.org/dist/Apache2-xForwardedFor/lib/Apache2/xForwardedFor.pm
>> I was going to say that PrefixMiddleware already did this, but
>> apparently it didn't. It does now in Paste Deploy trunk.
>
> I remember submitting a pretty decent patch for this a long time ago.

I think we had philosophical differences about whether to read the
headers in attributes, or rewrite the environment, and I was on the
rewrite side. And... wasn't it X-Forwarded-Server?

[Christopher Weimann]

Jonathan Vanasco wrote:
>
> maybe someone can make a middleware or pylons patch + config setting
> that migrates X_FORWARDED_FOR to REMOTE_ADDR
>

REMOTE_ADDR is typically set by your webserver and can be trusted. X_FORWARDED_FOR is an HTTP header typically set by proxy servers or even the client and I would have to say it can NOT be trusted so should certainly not replace the reliable data in REMOTE_ADDR.

[Ian Bicking]

This is in part the reason why it is handled in middleware. If you *do*
have a proxy in your installation then you can (and should) trust
X-Forwarded-For, and moving it to REMOTE_ADDR signifies the
trustworthiness of the value. If you are not behind a proxy you should
ignore the value. (Arguably, maybe you should even remove the header or
reject the request, but nothing currently does that.)

[Jonathan Vanasco]

On Jul 7, 1:46 pm, Ian Bicking <i...@colorstudy.com> wrote:

> Christopher Weimann wrote:
> > REMOTE_ADDR is typically set by your webserver and can be trusted.  X_FORWARDED_FOR is an HTTP header typically set by proxy servers or even the client and I would have to say it can NOT be trusted so should certainly not replace the reliable data in REMOTE_ADDR.

> This is in part the reason why it is handled in middleware.  If you *do*
> have a proxy in your installation then you can (and should) trust
> X-Forwarded-For, and moving it to REMOTE_ADDR signifies the
> trustworthiness of the value.  If you are not behind a proxy you should
> ignore the value.  (Arguably, maybe you should even remove the header or
> reject the request, but nothing currently does that.)

Correct.

The middleware I use under modperl works like this:

Port 80 proxy-
Sets X-Forwarded-For to REMOTE_ADDR
Sets X-Forwarded-For-Lan to LAN_SECRET
Sets REMOTE_ADDR to the lan id / 127.0.0.1

ModPerl 'middleware'
Requires X-Forwarded-For headers to have:
REMOTE_ADDR of specifc lan ips
X-Forwarded-For-Lan with correct lan secret
If those qualifications are met, then the X-Forwarded-For becomes
REMOTE_ADDR. If they do not, the request is rejected.

This approach ensures that data IS reliable, and GREATLY simplifies
development and clustering across load balancers.

[Shannon -jj Behrens]

Ian wrote:
> I think we had philosophical differences about whether to read the
> headers in attributes, or rewrite the environment, and I was on the rewrite
> side. And... wasn't it X-Forwarded-Server?

Sorry, didn't mean to pick on you, Ian ;)

I'm all for rewriting the environ. If I remember right, my version of the code
required no configuration, but could not handle being "mounted" in a
subdirectory whereas the existing code was the opposite. Either way, I wish it
were fixed by default. This bug has chased me through two companies ;)

I'd be happy even it Routes automatically respected X_FORWARDED_FOR or
something.

Jonathan wrote:
> The middleware I use under modperl works like this:
>
> Port 80 proxy-
> Sets X-Forwarded-For to REMOTE_ADDR
> Sets X-Forwarded-For-Lan to LAN_SECRET
> Sets REMOTE_ADDR to the lan id / 127.0.0.1
>
> ModPerl 'middleware'
> Requires X-Forwarded-For headers to have:
> REMOTE_ADDR of specifc lan ips
> X-Forwarded-For-Lan with correct lan secret
> If those qualifications are met, then the X-Forwarded-For becomes
> REMOTE_ADDR. If they do not, the request is rejected.

Clever ;)

By the way, I'm confused. What *is* the standard answer to this
question for Pylons?

-jj

[Shannon -jj Behrens]

Ugh, looking at request.environ, Varnish is giving me
HTTP_X_FORWARDED_FOR. Looking at my old Aquarium code, it respected
X_FORWARDED_HOST. Looking at Paste's proxy middleware, I see that it
looks for X-Forwarded-Server. Gees, am I confused! Can someone lend
me a clue?

Thanks,
-jj

[Ian Bicking]

There's no standards or even any documentation I've found on these
headers. They are totally willy-nilly. It's a mess.

Incidentally PasteDeploy trunk (and probably more recent releases)
supports X-Forwarded-Host (and I think I might have actually noticed
that header from reading the Aquarium code).

[Shannon -jj Behrens]

It turned out that I just needed to follow the Cookbook recipe a
little bit more closely:
http://wiki.pylonshq.com/display/pylonscookbook/Running+Pylons+with+NGINX

The trick is configuring Nginx with:

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Once you do this, you don't even need to use Paste's proxy middleware.
Nginx itself will setup the Host header correctly.

I'm still not sure why that recipe configures so many buffer sizes.
When I read things like that, it makes me wonder why they're necessary
and whether Nginx's own defaults are actually broken. I also wonder
if those settings are even appropriate for my situation.

Happy Hacking!

[Ben Bangert]

On Jul 9, 2008, at 11:56 AM, Shannon -jj Behrens wrote:

> The trick is configuring Nginx with:
>
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>
> Once you do this, you don't even need to use Paste's proxy middleware.
> Nginx itself will setup the Host header correctly.

Actually, you sort of still need to use PasteDeploy's
PrefixMiddleware. Especially the version that I think is only in the
latest code under the following condition:
* You have a site that is proxied to under http AND https

This is because the wsgi.url_scheme will get set to http by default,
so generating absolute URL's under the https side will result in non-
https URL's. The latest trunk of PasteDeploy's PrefixMiddleware will
look for another Header, X_FORWARDED_PROTO. So my setup for nginx
looks like this (for the HTTPS proxy location):

proxy_set_header Host $host;
proxy_set_header X_FORWARDED_PROTO https;

And I have PrefixMiddleware in my config/middleware.py.
PrefixMiddleware will then properly setup the wsgi.url_scheme which
ensures that everything makes appropriate URL's for where it is.

Cheers,
Ben