Setting the source IP Address for WvHTTPPool

[Patrick Patterson]

Hi all:

Due to several insanties all happening at once, we find ourselves in the position of needing for pathfinder a way to say "Send out all requests from this particular IP Address" (from a multi IP Aliased host). Perhaps a bit of additional explanation:

Say you have a host with the following config:

eth0 192.168.6.1
eth0:1 192.168.6.2
eth0:2 192.168.6.3
eth0:3 192.168.6.4

Since there are no specific "route this IP to this set of addresses" entries in the routing table, any request originating from this server will have the source address of 192.168.6.1. What we need is some way of saying for a given daemon instantiation "bind to this IP Address, so that each request appears to originate from that IP Address". This appears to be possible with calling bind() prior to connect().

Now, WvTCPConn has a way to get the source IP address, but neither Dave nor I have seen a way to set it, aside from using getfd(), and fiddling from there. And while this might let us do that for a straight WvTCPConn, there is no obvious way to set the source IP for connections kicked off from a WvHTTPPool.

The way I see it, the best way to do this is to add an additional constructor to both WvTCPConn and WvHTTPPool that allows for the setting of the source IP. Does anyone have any better ways?

(And no, we can't fiddle with the routing table on the host pathfinder is installed from, nor do we want to do any of the several other tricks that would trick the firewall that we need to cross into thinking that we're from a different IP.)

Thanks.

--
Personal Mail from Patrick Patterson
No company affiliation

[Avery Pennarun]

On Wed, Mar 31, 2010 at 12:24 PM, Patrick Patterson <ppat...@gmail.com> wrote:
> Since there are no specific "route this IP to this set of addresses" entries
> in the routing table, any request originating from this server will have the
> source address of 192.168.6.1. What we need is some way of saying for a
> given daemon instantiation "bind to this IP Address, so that each request
> appears to originate from that IP Address". This appears to be possible with
> calling bind() prior to connect().

As you can imagine, this is a pretty rare requirement, so it's not
currently built into WvHttpPool for the same reason that it's not
built into, say, Firefox.

I don't see any reason to reject a patch that would add this feature
(since the patch would likely be rather simple and non-intrusive) but
of course you'd then have to wait for your favourite distros to pick
up the new WvStreams.

> The way I see it, the best way to do this is to add an additional
> constructor to both WvTCPConn and WvHTTPPool that allows for the setting of
> the source IP. Does anyone have any better ways?

If you want to send a patch to do this, I won't object. For
WvTCPConn, a new constructor makes sense. In the case of WvHttpPool,
I'd rather have a member function to set the new value, since
WvHttpPool doesn't initiate any connections until after the
constructor anyway.

> (And no, we can't fiddle with the routing table on the host pathfinder is
> installed from, nor do we want to do any of the several other tricks that
> would trick the firewall that we need to cross into thinking that we're from
> a different IP.)

Fiddling with the routing table would still be the best way :)

But speaking of things that aren't as good as fiddling with the
routing table, you could do what we did with DoubleVision way back in
the Weaver. Just make an LD_PRELOAD library that captures connect()
and does a bind() on the socket, then forwards to the normal
connect(). If the socket is already bound, the new bind() will fail,
so it's harmless. You could get the address to bind() to from an
environment variable.

The nice thing about this technique is it *does* work with Firefox or
any other program, so you only have to write it once :) The less-nice
thing is that overriding libc functions, while possible on nearly
every OS, is slightly different on each one.

(Double Vision existed before the Linux kernel had support for binding
source addresses via the routing table / iptables, so it was the
*only* simple way to do it at the time.)

Have fun,

Avery

[Avery Pennarun]

On Wed, Mar 31, 2010 at 1:45 PM, Avery Pennarun <apen...@gmail.com> wrote:
> In the case of WvHttpPool,
> I'd rather have a member function to set the new value, since
> WvHttpPool doesn't initiate any connections until after the
> constructor anyway.

I take that back... I member variable with no function would be even better :)

Avery

[Patrick Patterson]

Hi Avery:

On Wed, Mar 31, 2010 at 1:45 PM, Avery Pennarun <apen...@gmail.com> wrote:

On Wed, Mar 31, 2010 at 12:24 PM, Patrick Patterson <ppat...@gmail.com> wrote:
> Since there are no specific "route this IP to this set of addresses" entries
> in the routing table, any request originating from this server will have the
> source address of 192.168.6.1. What we need is some way of saying for a
> given daemon instantiation "bind to this IP Address, so that each request
> appears to originate from that IP Address". This appears to be possible with
> calling bind() prior to connect().

As you can imagine, this is a pretty rare requirement, so it's not
currently built into WvHttpPool for the same reason that it's not
built into, say, Firefox.

Yeah - I know - however, the requirement did come up.
 

I don't see any reason to reject a patch that would add this feature
(since the patch would likely be rather simple and non-intrusive) but
of course you'd then have to wait for your favourite distros to pick
up the new WvStreams.

Well, since the system in question is running a sufficiently old distro that I doubt that they would have ANY new software in it ever, we're rolling our own WvStreams for it. The reason for the query is that I hate to maintain things out of tree, and it seemed like it would be a good thing to have in the swiss-army-knife-of-networking that WvStreams is :)
 

> The way I see it, the best way to do this is to add an additional
> constructor to both WvTCPConn and WvHTTPPool that allows for the setting of
> the source IP. Does anyone have any better ways?

If you want to send a patch to do this, I won't object.  For
WvTCPConn, a new constructor makes sense.  In the case of WvHttpPool,
I'd rather have a member function to set the new value, since
WvHttpPool doesn't initiate any connections until after the
constructor anyway.

Ok - a member variable it is (from your follow on mail :)
 

> (And no, we can't fiddle with the routing table on the host pathfinder is
> installed from, nor do we want to do any of the several other tricks that
> would trick the firewall that we need to cross into thinking that we're from
> a different IP.)

Fiddling with the routing table would still be the best way :)

I know - however this is running in a VERY complicated environment, with a third party operating the OS, and we can't do that.....
 

But speaking of things that aren't as good as fiddling with the
routing table, you could do what we did with DoubleVision way back in
the Weaver.  Just make an LD_PRELOAD library that captures connect()
and does a bind() on the socket, then forwards to the normal
connect().  If the socket is already bound, the new bind() will fail,
so it's harmless.  You could get the address to bind() to from an
environment variable.
 

The nice thing about this technique is it *does* work with Firefox or
any other program, so you only have to write it once :)  The less-nice
thing is that overriding libc functions, while possible on nearly
every OS, is slightly different on each one.

Well, while that is an idea, I'm also working in a very security conscious environment, so I doubt that "the powers that be" would gaze kindly on LD_PRELOAD, given any number of "OMG, LD_PRELOAD is horrible" stories/myths out there... I might be able to get away with it if I luck out and find someone to approve the system change request that doesn't know what it is, but I wouldn't want to count on that.

Thanks for the input.

--

[Avery Pennarun]

On Wed, Mar 31, 2010 at 10:03 PM, Patrick Patterson <ppat...@gmail.com> wrote:
> Well, while that is an idea, I'm also working in a very security conscious
> environment, so I doubt that "the powers that be" would gaze kindly on
> LD_PRELOAD, given any number of "OMG, LD_PRELOAD is horrible" stories/myths
> out there... I might be able to get away with it if I luck out and find
> someone to approve the system change request that doesn't know what it is,
> but I wouldn't want to count on that.

Hmm, now that I think of it, if it's an app you're compiling for
yourself (which it is) you don't have to use LD_PRELOAD at all. Just
provide your own copy of connect() and link it into the app :)

Anyway, if you prefer to patch WvStreams, that works too.

Have fun,

Avery