Attach Performance
Javier Barreiro
it's my first posting to this group, so I'd like to thank the bitten
developers for making such a great tool available, I use it for
several of my projects and find it very well designed and extremely
useful.
now, on to my topic. there was a posting in october about problems
with <attach> :
1- it seems slow
2- it locks the entire trac project for the duration of the upload
http://groups.google.com/group/bitten/browse_thread/thread/95057dd0a57bf0dd
I can live with number 1, but number 2 affects usability seriously. In
my case, for one of my projects I upload a couple of relatively small
files (15MB and 25MB), and the trac project locks for about 5 minutes,
which has several undesirable side effects :
- nobody can access the wiki
- other bitten slaves fail and quit with a "500 server error" after
they give up waiting to attach their own build artifacts.
- access to the svn repository seems to be locked as well, although I
haven't systematically verified that <attach> is the cause.
with about 5 developers in the project, we could be experiencing this
for most of the day some days, so I'm thinking about dropping the use
of <attach>, but I'd hate to do that since having the artifacts
available for download is so useful.
several questions :
- is there already a ticket and/or plans to address this
- is there a reasonable workaround
- in the october thread Mark Potter mentioned he uses the XmlRpc
plugin to upload the artifacts somewhere else (not on the build
pages). Mark : would you be so kind to post the client code or
pointers on how to make that happen?
thanks in advance,
Javier
potter
> - is there already a ticket and/or plans to address this
> - is there a reasonable workaround
My current work around is the use of the XmlRpc. I am currently
thinking about changing this to simply copying the resulting files to
a network drive were an http or ftp server will automatically make
them available.
> - in the october thread Mark Potter mentioned he uses the XmlRpc
> plugin to upload the artifacts somewhere else (not on the build
> pages). Mark : would you be so kind to post the client code or
> pointers on how to make that happen?
Notes:
* This was not written for general consumption, I got it working for
our installation and stopped.
* Oh dear, I didn't even comment the code. Bad Potter, bad Potter.
* We use the AccountManagerPlugin for authentication, and therefore
this script might not work without change.
* To use XmlRpcPlugin with AccountManagerPlugin one also needs
HttpAuthPlugin.
* I just quickly removed some site specific strings from this script
which one may need to fix or replace. (I remove our Trac server name
and replaced it with localhost).
{{{
import getopt
import sys
import xmlrpclib
USAGE="""USAGE:
Attach [-u username] [-p password] [-s server] page file [comment]
WHERE:
username Gives the name of the user to login as. Default: none.
password Gives the password to use to login. Default: none.
server Gives the path the the trac server to connect to.
Default:
localhost/trac.
page Gives the Wiki page to attach to.
file Gives the file to attach.
comment Gives the comment to associated with the attachment."""
OPTS="u:p:s:"
USERNAME=""
PASSWORD=""
SERVER="localhost/trac"
COMMENT=""
def usage():
print USAGE
sys.exit(1)
def getServer( server, username, password ):
url = "http://"
if len(username):
url = url + username
if len(password):
url = url + ':' + password
url = url + '@'
url = url + server + "/login/xmlrpc"
return xmlrpclib.ServerProxy(url)
def main():
global USERNAME
global PASSWORD
global SERVER
try:
options, arguments = getopt.getopt(sys.argv[1:], OPTS)
except getopt.error:
usage()
for o, a in options:
if o == '-u':
USERNAME=a
if o == '-p':
PASSWORD=a
if o == '-s':
SERVER=a
if len(arguments) < 2:
print "ERROR: Too few arguments"
usage()
if len(arguments) > 3:
print "ERROR: Too many arguemnts"
usage()
page = arguments[0]
file = arguments[1]
comment = ""
if len(arguments) == 3:
comment = arguments[2]
try:
server = getServer( SERVER, USERNAME, PASSWORD )
except:
print "Unable to connect to server: ", sys.exc_info()[0]
sys.exit(1)
try:
data = open(file,"rb").read()
except IOError, err:
print "Unable to read file", file, ":", err;
sys.exit(1)
except:
print( "Unable to read file", file, ":", sys.exc_info()[0] )
sys.exit(1)
print "Attaching", file, "to", page, "on", SERVER, "as", USERNAME
try:
server.wiki.putAttachmentEx( page, file, comment, xmlrpclib.Binary
(data), True)
except xmlrpclib.Fault, err:
print "Unable to attach:", err;
sys.exit(1)
except:
print "Unable to attach: ", sys.exc_info()[0]
sys.exit(1)
if __name__ == '__main__':
main()
}}}
potter
by Google Groups and will need to be corrected when copying.
In the USAGE string, circa line 10.
> server Gives the path the the trac server to connect to.
> Default:
In the last try block of the main() function, circa line 80:
> server.wiki.putAttachmentEx( page, file, comment, xmlrpclib.Binary
> (data), True)
Oh, I also did not mention that I call this file "Attach.py".