Listening on multiple IPs at once

[Brian Rak]

I'm currently running into some issues while trying to use a
multihomed box, with a different FTP server on each IP. This is the
code I'm using:

from pyftpdlib import ftpserver
from threading import Thread
import socket

def getIPAddresses():
return [ip for ip in socket.gethostbyname_ex(socket.gethostname())
[2] if not ip.startswith("127.")]

def startFTPServer(address):
handler = ftpserver.FTPHandler
handler.authorizer = ftpserver.DummyAuthorizer()
handler.authorizer.add_user(address,'pass','C:/')

ftpd = ftpserver.FTPServer((address,29), handler)
ftpd.serve_forever()

threads = []
for addr in getIPAddresses():
curthread = Thread(target=startFTPServer, args=(addr,))
curthread.daemon = True
curthread.start()
threads.append(curthread)

while 1==1:
pass

Let's say I have three IP addresses on the box, 10.1.1.1, 10.5.5.5,
and 192.168.1.1. What happens is that when I connect to 10.1.1.1 it
attempts to use the authorizer for 192.168.1.1 (which was the last
server created).

Is threading supported in this way? Is there something I'm doing
wrong that is causing it to use the last authorizer, instead of the
one bound to the specific IP?

[Giampaolo Rodolà]

Hi,

that's not correct (above all, you don't need to spawn threads).

pyftpdlib will automatically listen on on all IP addresses of a given network interface by specifying "0.0.0.0" or "" as host address:

 >>> ftpd = ftpserver.FTPServer(("", 21), ftp_handler) 

If you have multiple network interfaces and you want to listen on all of them this should work (not tested):

 >>>ftpd1 = ftpserver.FTPServer(("10.1.1.1", 21), ftp_handler)

 >>>ftpd2 = ftpserver.FTPServer(("10.5.5.5", 21), ftp_handler)

 >>>ftpd1.serve_forever()

[Brian Rak]

That does work. Minor issue that it displays it's only listening on
the last port, but actually listens on all of them.

However, the issue I was seeing is still present. All of the
FTPHandler instances share the same DummyAuthorizor. I attempt
attempting to have different authentication for each different IP/
port. Looking at the code again, this makes sense. The handler isn't
actually created until a connection is made, so by changing the
authorizer each time I open a connection, it ends up being that only
the last authorizer is actually used.

This is what I'm currently attempting. It should create a user
'10.1.1.1:pass' that will only work when the connection is made to
10.1.1.1:

from pyftpdlib import ftpserver
from threading import Thread
import socket

def getIPAddresses():
return [ip for ip in socket.gethostbyname_ex(socket.gethostname())
[2] if not ip.startswith("127.")]

ftps = []
for address in getIPAddresses():

handler = ftpserver.FTPHandler
handler.authorizer = ftpserver.DummyAuthorizer()
handler.authorizer.add_user(address,'pass','C:/')

ftpd = ftpserver.FTPServer((address,29), handler)

ftps.append(ftpd)

ftps[0].serve_forever()


It's looking like to get the behaviour I want (listening on multiple
IP's, different *instance* of the same authorizor for each IP) I will
need to rewrite the FTPServer class.

[Nick Craig-Wood]

On 16/06/11 18:33, Giampaolo Rodol� wrote:
> that's not correct (above all, you don't need to spawn threads).
> pyftpdlib will automatically listen on on all IP addresses of a given
> network interface by specifying "0.0.0.0" or "" as host address:

When I tried "0.0.0.0" it worked, but it stopped passive mode working as
the ftp server was offering 0,0,0,0 in the control channel which
obviously won't work.

This might be a bug!

I didn't try "" though.

--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick

[Brian Rak]

So, what I ended up coming up with is this:

from pyftpdlib.ftpserver import FTPServer, log, logerror
import asyncore

class VhostFTPServer(FTPServer):
def handle_accept(self):
"""Called when remote client initiates a connection."""
try:
sock, addr = self.accept()
except TypeError:
# sometimes accept() might return None (see issue 91)
return
except socket.error, err:
# ECONNABORTED might be thrown on *BSD (see issue 105)
if err[0] != errno.ECONNABORTED:
logerror(traceback.format_exc())
return
else:
# sometimes addr == None instead of (ip, port) (see issue
104)
if addr == None:
return

hostip = sock.getsockname()
self.handler.authorizer.setHost(hostip[0])

try:
handler = self.handler(sock, self)
except socket.error, err:
# safety measure in case of undiscovered asyncore bugs,
see:
# http://code.google.com/p/pyftpdlib/issues/detail?id=143
logerror(str(err))
return
if not handler.connected:
return
log("[]%s:%s Connected." % addr[:2])
ip = addr[0]
self.ip_map.append(ip)

# For performance and security reasons we should always set a
# limit for the number of file descriptors that socket_map
# should contain. When we're running out of such limit we'll
# use the last available channel for sending a 421 response
# to the client before disconnecting it.
if self.max_cons and (len(asyncore.socket_map) >
self.max_cons):
handler.handle_max_cons()
return

# accept only a limited number of connections from the same
# source address.
if self.max_cons_per_ip and (self.ip_map.count(ip) >
self.max_cons_per_ip):
handler.handle_max_cons_per_ip()
return

handler.handle()
from pyftpdlib import ftpserver

import socket

def getIPAddresses():
return [ip for ip in socket.gethostbyname_ex(socket.gethostname())
[2] if not ip.startswith("127.")]

ftps = []

handler = ftpserver.FTPHandler
handler.authorizer = VhostAuthorizor()

for addr in getIPAddresses():
ftpd = VhostFTPServer((addr,21), handler)
ftps.append(ftpd)

ftps[0].serve_forever()

It requires that the authorizor you use have one extra function,
setHost(ipaddress). This will be called before each connection
attempt with the server's IP address as the parameter, and allows the
authorizor to change it's behavior based on the server's IP. This
isn't a perfect way to do it, as it breaks compatibility with all the
previous authorizors, and requires that you use the same authorization
method for all the server IPs.

[Giampaolo Rodolà]

Instead of overriding FTPServer class you can override FTPHandler
instead, and set authorizer attribute once the client connects, which
is more cleaner/safer:

class CustomFtpHandler(ftpserver.FTPHandler):

def handle(self):
# this is called when client connects

hostip = sock.getsockname()
self.handler.authorizer.setHost(hostip[0])

2011/6/17 Brian Rak <d...@devicenull.org>:

> --
> You received this message because you are subscribed to the "Python FTP server library" project group:
> http://code.google.com/p/pyftpdlib/
> To post to this group, send email to pyft...@googlegroups.com
> To unsubscribe from this group, send email to pyftpdlib-...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/pyftpdlib

[Giampaolo Rodolà]

2011/6/17 Nick Craig-Wood <ni...@craig-wood.com>:

> --
> You received this message because you are subscribed to the "Python FTP server library" project group:
> http://code.google.com/p/pyftpdlib/
> To post to this group, send email to pyft...@googlegroups.com
> To unsubscribe from this group, send email to pyftpdlib-...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/pyftpdlib

It's likely the server was behind a NAT and you didn't specify a
masquerade IP address (FTPHandler.masqueraded_ip attribute).

[Giampaolo Rodolà]

> That does work.  Minor issue that it displays it's only listening on 
> the last port, but actually listens on all of them. 

Note that in r868 I turned serve_forever() into a class method so now instead of doing:

 >>> ftpd1 = ftpserver.FTPServer(address1, ftp_handler)
 >>> ftpd2 = ftpserver.FTPServer(address2, ftp_handler)
 >>> ftpd2.serve_forever()  # start both servers

....you can do:

 >>> ftpd1 = ftpserver.FTPServer(address1, ftp_handler)
 >>> ftpd2 = ftpserver.FTPServer(address2, ftp_handler)
 >>> ftpserver.FTPServer.serve_forever()

This solves the misleading log:port log and is more intuitive.

[Brian Rak]

That does look a lot cleaner, and works nicely. In case anyone else
needs it, this is the exact code I am using

class VhostFTPHandler(ftpserver.FTPHandler):
def handle(self):
hostip = self.socket.getsockname()
self.authorizer.setHost(hostip[0])

super(VhostFTPHandler,self).handle()

> >            #http://code.google.com/p/pyftpdlib/issues/detail?id=143

[Nick Craig-Wood]

On 18/06/11 11:13, Giampaolo Rodol� wrote:
> 2011/6/17 Nick Craig-Wood <ni...@craig-wood.com>:

>> On 16/06/11 18:33, Giampaolo Rodol� wrote:
>>> that's not correct (above all, you don't need to spawn threads).
>>> pyftpdlib will automatically listen on on all IP addresses of a given
>>> network interface by specifying "0.0.0.0" or "" as host address:
>>
>> When I tried "0.0.0.0" it worked, but it stopped passive mode working as
>> the ftp server was offering 0,0,0,0 in the control channel which
>> obviously won't work.
>>
>> This might be a bug!
>>
>> I didn't try "" though.
>

> It's likely the server was behind a NAT and you didn't specify a
> masquerade IP address (FTPHandler.masqueraded_ip attribute).

It turned out to be a bug in our code with setting masqueraded_ip, so
don't worry about it!

Cheers

Nick