I am thinking of using the approach suggested here
http://tgolden.sc.sabren.com/python/win32_how_do_i/watch_directory_for_changes.html
which is:
import os, time
path_to_watch = "."
before = dict ([(f, None) for f in os.listdir (path_to_watch)])
while 1:
time.sleep (10)
after = dict ([(f, None) for f in os.listdir (path_to_watch)])
added = [f for f in after if not f in before]
removed = [f for f in before if not f in after]
if added: print "Added: ", ", ".join (added)
if removed: print "Removed: ", ", ".join (removed)
before = after
My concern with this is that a change may be detected before the ftp
daemon process is done writing the file to disk. I don't want to take
any action until the file is written and closed. I know that I could
pole a new file looping to check to see if it's file size is changing
but the timing of such a loop is subject to I/O buffering and is
otherwise not elegant.
Googling shows other solutions using fcntl
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/217829) but it
appears that this only works on Linux.
While I'm at it I'm going to throw in a grump about the Python
documentation of fcntl. The doco indicates to read the source for
fcntl.py to lookup the constants representing the different types of
events/signals that are avaiable. However fcntl on some platforms
seems to be implemented as a binary leaving no way to look up the
contants for the platform.
Suggestions?
there is no reliable way to do this, unless you can modify the sending
application to use a temporary filename during transfer (or find an FTP
server that does this automatically), or you're willing to write your own
FTP responder.
checking the mtime for new files might be good enough for some use
cases (poll the directory at regular intervals; if a file is newer than X
seconds, assume it's still being updated).
</F>
'pydoc fcntl' works fine on my system, even though the module is a
binary .so file.
Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa
I think you may have missed my point. I don't have fcntl.py on my
Solaris box so how do I know what signals that I can used to monitor a
directory for modification. In other words will the following work?
fcntl.fcntl(self.fd, fcntl.F_NOTIFY,
fcntl.DN_DELETE|fcntl.DN_CREATE|fcntl.DN_MULTISHOT)
Without fcntl source, which the python documentation suggests that I
look at, I don't know if any of these constants apply to the Solaris
platform.
$ man fcntl
</F>
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/
Or use an existing Python FTP server! ;)
Untested:
from zope.interface import implements
from twisted.protocols import ftp
class UploadNotifyingAvatar(ftp.FTPShell):
def uploadCompleted(self, path):
"""
Override me!
"""
# But, for example:
from twisted.mail import smtp
smtp.sendmail(
'127.0.0.1',
'ftp_server@localhost',
['ftp_admin@localhost'],
'Hey! Someone uploaded a file: %r' % (path,))
def openForWriting(self, path):
writer = ftp.FTPShell.openForWriting(path)
return CloseNotifyingWriter(
writer,
lambda: self.uploadCompleted(path))
class CloseNotifyingWriter(object):
implements(ftp.IWriteFile)
def __init__(self, wrappedWriter, onUploadCompletion):
self.wrappedWriter = wrappedWriter
self.onUploadCompletion = onUploadCompletion
def receive(self):
receiver = self.wrappedWriter.receive()
receiver.addCallback(
CloseNotifyingReceiver,
self.onUploadCompletion)
return receiver
class CloseNotifyingReceiver(object):
def __init__(self, wrappedReceiver, onUploadCompletion):
self.wrappedReceiver = wrappedReceiver
self.onUploadCompletion = onUploadCompletion
def registerProducer(self, producer, streaming):
return self.wrappedReceiver.registerProducer(producer, streaming)
def unregisterProducer(self):
# Upload's done!
result = self.wrappedReceiver.unregisterProducer()
self.onUploadCompletion()
return result
def write(self, bytes):
return self.wrappedReceiver.write(bytes)
Hook it up to a Realm and a Portal and override uploadCompleted to taste and you've got an FTP server that should do you (granted, it's a little long - some of this code could be be factored to take advantage of parameterizable factories to remove a bunch of the boilerplate).
Jean-Paul
I assume that this little snippet will handle multiple/concurrent
incoming transfers via threading/sub-process, is scalable, secure, etc?
I could even run it on a non-standard port making it a bit more
(ob)secure.
If you can use it, then you can use the nice python-fam module. Very
simple to work with and generally works very well. I wrote a python
app that monitored a directory for file creation and deletion and sent
emails based on certain events.
http://python-fam.sourceforge.net/
Good luck.
Correct, except for the threading/sub-process part. Events from sockets are handled in a single thread in a single process.
>
>I could even run it on a non-standard port making it a bit more
>(ob)secure.
Indeed, although I mainly suggested it because I thought you were tied to FTP. If you actually have security concerns, you might want to use SFTP instead (which Twisted also supports, but to which the code I sent is only partly applicable).
Jean-Paul
Did some reading about twisted and I now understand that things in
general are single threaded.
I started working my way through the twisted finger tutorial. While it
appears that it may be useful for someone writing a new protocol it
doesn't seem to useful for someone just trying to hook into an existing
one.
While I do appreciate the suggestions but I have to say that if the
twisted folks spent half the time writing documentation as they do code
- twisted would probably get used a lot more Python folks. Didn't get
much encouragement/assistance from the twisted irc channel either.
Perhaps the fella I chatted with hadn't had his coffee yet ;)
> While I do appreciate the suggestions but I have to say that if the
> twisted folks spent half the time writing documentation as they do code
> - twisted would probably get used a lot more Python folks. Didn't get
> much encouragement/assistance from the twisted irc channel either.
> Perhaps the fella I chatted with hadn't had his coffee yet ;)
or maybe it's a medication issue. your experience isn't exactly unique:
http://twistedmatrix.com/pipermail/twisted-python/2005-May/010380.html
"In this e-mail thread, someone wrote that people who say Twisted
has no docs ought to be stabbed in the face. Obviously, this was
intended as humor, but I believe that this social custom, along with
as other related social customs in Twistedland, deter people from
some other cultures from participating."
</F>