Creating a new ticket from a shell script (SVN commit hook)

[Alan]

Hi. I am a new time member to this list. We are trying to put
together a simple code review infrastructure. I have a shell script
going that can generate the following text:

Change set [4813]

This is an example SVN commit message of a change I just made.
commit-review:joe

$COMMIT_REVIEW_DIFF $SVN/trunk/README.txt@{4812,4813}

Review allocated to: nobody

I can change around the text. The idea is I want to create a new
ticket that has at least the change set and a list of commands a
reviewer can copy/paste into a shell to review the changes. The change
set would link up to the Trac built in diff facility. The other
commands is if someone wants to use say tkdiff instead. Anyway, I can
produce the text I want pretty easily just in the shell script and
work out the various properties for the new ticket.

I was going to create a new "type" in my Trac installation for "commit-
review" and so wanted on every SVN commit (or possibly only if the
commit message contained the magic string "commit-review:<user>") to
automatically create a new Trac ticket which was for the specified
user to review the commit. I have a commit hook shell script working
that collects all the information I need to create a new ticket. I
just don't know how to create a ticket from a shell script.

Sorry if there is a standard way to do this. I just cannot find it.
'trac-admin' for example can remove tickets, but does not seem to be
able to create tickets.

Any help greatly appreciated!!!

Alan


My problem is I don't know Python or the Trac internals. Does anyone
have or can anyone point me at some sample Python code that can say
read from stdin the description with command line arguments to
populate the short summary, type, priority, assign-to, etc settings
(just using argv[1], argv[2] etc is fine) of a new ticket? I was
hoping this would be a few seconds for someone who knew the internals
- but would be hours or days for me to learn it all from scratch.

Any help greatly appreciated!

[Lars Stavholm]

http://trac-hacks.org/wiki/EmailtoTracScript
/L

[Tim Alsop]

Alan,

I suggest you take a look at :

http://trac-hacks.org/wiki/PeerReviewPlugin

In particular, check out http://www.blisted.org/wiki/projects/JianCha

Thanks,
Tim

[Alan Kent]

Lars Stavholm wrote:
> http://trac-hacks.org/wiki/EmailtoTracScript
> /L
>

Thanks - I had looked at the code already. My problem is I don't want
an email submission process, I want to create a new ticket from a
command line. I tried to look into the code, but I am not a Python
programmer (yes, I can learn it) and am not familiar with the internal
Trac APIs (which I assume are documented somewhere). I was hoping to
avoid the days of learning a new programming language and new APIs if at
all possible. I was hoping it would only be a few minutes for someone
to do (but days for me) - or if someone knew of such an existing script
in one of the other Trac hacks.

I have browsed through the list several times, but the length of the
list of hacks available does making finding something specific
non-trivial - hence this plea for help!

Thanks for your help,
Alan

[Alan Kent]

Tim Alsop wrote:
> Alan,
>
> I suggest you take a look at :
>
> http://trac-hacks.org/wiki/PeerReviewPlugin
>
> In particular, check out http://www.blisted.org/wiki/projects/JianCha
>
> Thanks,
> Tim
>

Thanks. I had seen the peer review plugin. The comments I had read on
it however did not make it look appealing. (As others have said), I
want something that hooks into the SVN commit process, so the changes
are automatically identified.

I had a look an JianCha (I had not seen that before). It was not clear
to me from the above whether there was an actual implementation yet, or
just specs on how it is going to be done. Looking around I did not get
the impression that it is backed by an active and live community. I
would like something either trivial so we can support it in house, or if
more complex backed by a live community.

I have most of the infrastructure in place already for a simple solution
(borrowed from a mature C++ project in SVN that does not use Trac) - I
am trying to port the same practice to a Trac based Java process. All I
need is to create a ticket from a shell script and its ready for
internal trial. Not knowing Python or the Trac API means writing this is
not a trivial undertaking.

Thanks for the links though - it is appreciated.
Alan

[Emmanuel Blot]

> Thanks - I had looked at the code already. My problem is I don't want
> an email submission process, I want to create a new ticket from a
> command line. I tried to look into the code, but I am not a Python
> programmer (yes, I can learn it) and am not familiar with the internal
> Trac APIs (which I assume are documented somewhere).

If you know a language that can execute XMLRPC remote command (Perl,
for example), you could use the XmlRpcPlugin to create a ticket with a
very simple API.

Cheers,
Manu

[Dimitri Maziuk]

Errm... why bother, can't you just post newticket with e.g. curl?

Alan, if you use Postgres for backend, you can inject your tickets directly
into DB using anything from Java to psql shell. With the default sqlite
engine, you're limited (AFAIK) to C API or sqlite3 shell.

It looks like to create new ticket you only really need to insert a row into
ticket table, so something like
sqlite3 'mytrac.db' 'insert into ticket ...'
should do the trick. (Disclaimer: ICBW, I haven't tried this myself.)

Dima
--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu

[Noah Kantrowitz]

This is very bad advice. Never go behind the back of software unless you
have no option. We provide a very rich Python API for local use, and an
XML-RPC system for non-Python or non-local scripting. All the common
languages will have at least one library for XML-RPC (Perl looks to have a
few dozen), so you have no excuse to not use it.

--Noah

[Alan Kent]

Dimitri Maziuk wrote:
> Errm... why bother, can't you just post newticket with e.g. curl?
>

Posting via URLs is a bit of a problem as the site has HTTPS encrypted
username/passwords. But is an option I guess. I just have to hard code
a password into the commit script.

> Alan, if you use Postgres for backend, you can inject your tickets directly
> into DB using anything from Java to psql shell. With the default sqlite
> engine, you're limited (AFAIK) to C API or sqlite3 shell.
>
> It looks like to create new ticket you only really need to insert a row into
> ticket table, so something like
> sqlite3 'mytrac.db' 'insert into ticket ...'
> should do the trick. (Disclaimer: ICBW, I haven't tried this myself.)
>

Thanks. This is sounding an interesting approach - I don't have to do
web authentication this way (from the shell script). Thanks for the
lead. I am using sqlite3 and I can see the table structures - there
are actually 'ticket', 'ticket_change', and 'ticket_custom'. However if
I do this (and its safe), then the rest of Trac is not going to mail out
notifications of new tickets etc is it?

However someone else suggested the XmlRpcPlugin. Again, HTTPS posts are
a pain due to authentication, but looking at the XmlRpcPlugin source
code I found a 'create()' method which seems to be along the lines of
what I need - so I am going to try a bit of copy/paste and see if I can
create my first Python program.....

Well, here it is in case anyone is interested for future reference. It
just creates a new ticket (for my specific purpose - its not general
purpose - lots of hard coded strings). So it all seems to be going
OK... so far!

Thanks everyone for their help.
Alan

#!/usr/bin/env python

import trac.ticket.model as model
from trac.ticket.notification import TicketNotifyEmail
from sys import stdin
from sys import argv
from trac.env import open_environment

if len(argv) == 5:

project = argv[1]
summary = argv[2]
description = stdin.read()
committer = argv[3]
reviewer = argv[4]

env = open_environment(project)

t = model.Ticket(env)
t['status'] = 'new'
t['summary'] = summary
t['description'] = description
t['reporter'] = committer
t['owner'] = reviewer
t['type'] = 'commit-review'
t['priority'] = '5 - important'
t['component'] = 'general'
t.insert()

try:
tn = TicketNotifyEmail(env)
tn.notify(t, newticket=True)
except Exception, e:
print "Failure sending notification on creation of ticket #%s:
%s" % (t.
id, e))

else:
print "Usage: %s project summary submitter reviewer" % (argv[0])
print "Full description is read from stdin"

[Jeroen Ruigrok van der Werven]

-On [20080110 05:06], Alan Kent (ALAN....@saic.com) wrote:
>> It looks like to create new ticket you only really need to insert a row into
>> ticket table, so something like
>> sqlite3 'mytrac.db' 'insert into ticket ...'
>> should do the trick. (Disclaimer: ICBW, I haven't tried this myself.)
>
>Thanks. This is sounding an interesting approach - I don't have to do
>web authentication this way (from the shell script). Thanks for the
>lead. I am using sqlite3 and I can see the table structures - there
>are actually 'ticket', 'ticket_change', and 'ticket_custom'. However if
>I do this (and its safe), then the rest of Trac is not going to mail out
>notifications of new tickets etc is it?

Like Noah before me, I also would seriously advise about going this route. It
will effectively mean you are on your own and there are no guarantees your
data will remain intact, that your cat will keep all its hair, that your
girlfriend will not walk out on you and your house burn down. So you've been
warned. :)

You're better off looking at `pydoc trac.ticket.api` and using that API.

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
Agnosco veteris vestigia flammae...

[Lars Stavholm]

Jeroen Ruigrok van der Werven wrote:
> -On [20080110 05:06], Alan Kent (ALAN....@saic.com) wrote:
>>> It looks like to create new ticket you only really need to insert a row into
>>> ticket table, so something like
>>> sqlite3 'mytrac.db' 'insert into ticket ...'
>>> should do the trick. (Disclaimer: ICBW, I haven't tried this myself.)
>> Thanks. This is sounding an interesting approach - I don't have to do
>> web authentication this way (from the shell script). Thanks for the
>> lead. I am using sqlite3 and I can see the table structures - there
>> are actually 'ticket', 'ticket_change', and 'ticket_custom'. However if
>> I do this (and its safe), then the rest of Trac is not going to mail out
>> notifications of new tickets etc is it?
>
> Like Noah before me, I also would seriously advise about going this route. It
> will effectively mean you are on your own and there are no guarantees your
> data will remain intact, that your cat will keep all its hair, that your
> girlfriend will not walk out on you and your house burn down. So you've been
> warned. :)
>
> You're better off looking at `pydoc trac.ticket.api` and using that API.

Right so, good advice. While we're on the subject:
I'm using 0.10.4 and I get:

% pydoc trac.ticket.api
Traceback (most recent call last):
File "/usr/bin/pydoc", line 5, in ?
pydoc.cli()
File "/usr/lib/python2.4/pydoc.py", line 2235, in cli
help.help(arg)
File "/usr/lib/python2.4/pydoc.py", line 1689, in help
elif request: doc(request, 'Help on %s:')
File "/usr/lib/python2.4/pydoc.py", line 1475, in doc
pager(title % desc + '\n\n' + text.document(object, name))
File "/usr/lib/python2.4/pydoc.py", line 295, in document
if inspect.ismodule(object): return self.docmodule(*args)
File "/usr/lib/python2.4/pydoc.py", line 1066, in docmodule
contents = [self.formattree(
File "/usr/lib/python2.4/inspect.py", line 590, in getclasstree
for parent in c.__bases__:
TypeError: iteration over non-sequence

Anyone know why?
/L

[Jeroen Ruigrok van der Werven]

-On [20080110 08:22], Lars Stavholm (st...@telcotec.se) wrote:
>Right so, good advice. While we're on the subject:
>I'm using 0.10.4 and I get:
>
>% pydoc trac.ticket.api

pydoc unittest.TestSuite

works as expected? (Just want to make sure whether it is pydoc or trac
failing.)

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/

Courage is not the absence of fear, but rather the judgment that
something else is more important than fear.

[Emmanuel Blot]

> Errm... why bother, can't you just post newticket with e.g. curl?

Maintenance, robustness, and many other reasons that make APIs exist ;-)

Cheers,
Manu

[Lars Stavholm]

Jeroen Ruigrok van der Werven wrote:
> -On [20080110 08:22], Lars Stavholm (st...@telcotec.se) wrote:
>> Right so, good advice. While we're on the subject:
>> I'm using 0.10.4 and I get:
>>
>> % pydoc trac.ticket.api
>
> pydoc unittest.TestSuite
>
> works as expected? (Just want to make sure whether it is pydoc or trac
> failing.)

Yes indeed, it does, and so does...

% pydoc trac

...but not...

% pydoc trac.ticket
% pydoc trac.ticket.api

I'm not python savvy so I have no clue,
any help appreciated.
/L

[Jeroen Ruigrok van der Werven]

-On [20080110 11:25], Lars Stavholm (st...@telcotec.se) wrote:
>...but not...
>
>% pydoc trac.ticket
>% pydoc trac.ticket.api

That's odd. I just tested it on my FreeBSD box with both 2.4 and 2.5 and I can
pydoc those two module specifications without any hickups.

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/

Blessed are the meek, for they shall inherit the earth...

[Dimitri Maziuk]

Emmanuel Blot wrote:
>> Errm... why bother, can't you just post newticket with e.g. curl?
>
> Maintenance, robustness, and many other reasons that make APIs exist ;-)

OP specifically stated he didn't want to use existing python API. My
comment was about wrapping RPC call in XML and sending it over HTTP --
it's almost always a waste of cycles and bandwidth.

Personally, I'd learn Python -- we use enough Python software nowadays
I'll pretty much have to, sooner or later -- and it looks like OP is
going that way too.

Dima

[Emmanuel Blot]

Sorry I quoted the wrong line, my answer was about raw access to the
DB layer, anyway:

> OP specifically stated he didn't want to use existing python API.

My answer was about using another language, ie. similar API, not
related to Python - I was quoting PERL, for example.
No need to learn Python here.

> it's almost always a waste of cycles and bandwidth.

Is this optmization really important on nowadays machines ?

Cheers,
Manu

[Dimitri Maziuk]

On Thursday 10 January 2008 10:54:26 Emmanuel Blot wrote:
> Sorry I quoted the wrong line, my answer was about raw access to the
> DB layer

Yeah, I kinda figured. I did put a disclaimer on that, plus OP sounded smart
enough to understand about reaching into storage tier directly and
bypassing "business logic" layer.

> Is this optmization really important on nowadays machines ?

Yes. Nobody's figured out how to beat the kiss principle yet, and whether
today's focus is on energy consumption, maintenance costs, or whatever, it
always tends to boil down to that in the end. Optimization aside, if all you
want is to send or receive a chunk of data over http, xml is nothing but
overhead. If you want rpc, you probably need things xml-http-rpc doesn't do.

If you look around, most people who use that stuff are the ones who paid
megabucks to oracle/ibm. The rest of us do json mashups over restful API --
so much so that even w3c got off its xml horse, finally.

[Emmanuel Blot]

> Yes. Nobody's figured out how to beat the kiss principle yet, and whether
> today's focus is on energy consumption, maintenance costs, or whatever, it
> always tends to boil down to that in the end.

I don't share the same opinion, but as this thread as really evolved
OOT, let's close it ;-)

Cheers,
Manu