Re: Dull problem when trying to automate post-commit review with svn

[Christian Hammond]

Hi,

Just a quick sanity check. In your example, for the password, do you have the password quoted? Is there any way that some part of it could be interfering with the rest of the shell command?

Can you add --debug to that and output it some place that you can check?

Christian

--
Christian Hammond - chi...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com

On Tue, Sep 25, 2012 at 6:23 AM, Borislav Sabev <borisl...@gmail.com> wrote:

Hello all.

I have a dull problem that I can't seem to solve.

What am I trying to do?

I am trying to do:

post-review --repository-url=http://xxx.xxx.xxx.xxx/svn/testRepo2 --revision-range=6:7 --server=http://reviews.example.test/ --username=reviewposter --password=mydullpass --submit-as=admin -p --target-groups=reviewers

All is alright when I try this from bash logged in as my user. Review is posted, published, etc. ==> All permissions and other miscellaneous settings are OK.

What happens when I try this from SVN's post-commit hook?

The resources are commited but the svn operation hangs - actually the post-commit hook does not finish.

What is the problem?

post-review seems to get the user that is executing it and tries to log it in. The user that's executing it is www-data (apache user) I.e. when I run the command as www-data:

sudo -u www-data post-review --repository-url=http://xxx.xxx.xxx.xxx/svn/testRepo2 --revision-range=6:7 --server=http://reviews.example.test/ --username=reviewposter --password=mydullpass! --submit-as=admin -p --target-groups=reviewers -d

I get (notice the -d parameter in the post-review command - DEBUG):

>>> RBTools 0.4.1

>>> Home = /home/borislav

Password for 'www-data':

This is where it hangs waiting for a password to be entered. The commit operation cannot finish and it just stays there. On the other hand when I do the same command with debug output but as myuser I get:

>>> RBTools 0.4.1

>>> Home = /home/borislav

>>> HTTP GETting api/

>>> HTTP GETting http://reviews.example.test/api/info/

>>> Using the new web API

>>> HTTP GETting http://reviews.example.test/api/repositories/

>>> HTTP GETting http://reviews.example.test/api/repositories/1/

>>> HTTP GETting http://reviews.example.test/api/repositories/1/info/

>>> HTTP GETting http://reviews.example.test/api/repositories/2/

>>> HTTP GETting http://reviews.example.test/api/repositories/2/info/

>>> HTTP GETting http://reviews.example.test/api/repositories/3/

>>> HTTP GETting http://reviews.example.test/api/repositories/3/info/

>>> HTTP GETting http://reviews.example.test/api/repositories/4/

>>> HTTP GETting http://reviews.example.test/api/repositories/4/info/

>>> Attempting to create review request on http://xxx.xxx.xxx.xxx/svn/testRepo2 for None

>>> Submitting the review request as admin

>>> HTTP POSTing to http://reviews.example.test/api/review-requests/: {'submit_as': 'admin', 'repository': 'http://xxx.xxx.xxx.xxx/svn/testRepo2'}

>>> Review request created

>>> Attempting to set field 'target_groups' to 'reviewers' for review request '22'

>>> HTTP PUTting to http://reviews.example.test/api/review-requests/22/draft/: {'target_groups': 'reviewers'}

>>> Uploading diff, size: 2316

>>> HTTP POSTing to http://reviews.example.test/api/review-requests/22/diffs/: {'basedir': '/'}

>>> Publishing

>>> HTTP PUTting to http://reviews.example.test/api/review-requests/22/draft/: {'public': 1}

Review request #22 posted.

http://reviews.example.test/r/22/

So actually everything with the posting/publishing etc. works.

One solution I thought of is to execute the command as my user from the post-commit hook but that seems out-stretched and ugly to me.

Are there any other solutions?

Thanks, 

    Borislav.

--
Want to help the Review Board project? Donate today at http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~----------~----~----~----~------~----~------~--~---
To unsubscribe from this group, send email to reviewboard...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/reviewboard?hl=en

[Borislav Sabev]

Hello, Christian.

Debug is already there. :) Notice the -d which is the same as --debug. 

Indeed I had missed quoting the password but it works without quoting it when I execute it as my user. Thus this seems not to be the thing stopping the script.

As I previously stated when I try to execute the same command but as www-data for example, I get a prompt for a password for www-data!

www-data does not need to enter a password as that user is merely executing the script - all other user/password info is stated in the arguments passed.

Any suggestions?

Borislav.

[Christian Hammond]

Hi,

Unless I'm mistaken, it looks like the debug was for when you executed it yourself, rather than from the original call site.

By running it from sudo, it's using your home directory, meaning it may use your cookies file. I'm interesting in how things look when run from the environment post-review is running in when not run manually.

Christian

--
Christian Hammond - chi...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com

[Borislav Sabev]

Hey, Christian.

I think I posted the debug output when the script is run as www-data, but it turns out I cannot get ANY OUTPUT when executing from the post-commit script.

Here is the post-commit:

REPOS="$1"

REV=$(($2))

OLDREV=$(($REV-1))

RBSRV="http://reviews.example.test/"

SVNLOOK=/usr/bin/svnlook

AUTHOR="$($SVNLOOK author $REPOS -r $REV)"

LogDir="/var/log/svn/"

#Get only the name of the REPOSITORY as REPOS is a filesystem path:

RepoName=${REPOS##*/}

RepoURL="http://xxx.xxx.xxx.xxx/svn/${RepoName}"

#Some debugging info

echo "=====================================" >> /srv/logs/test

whoami >> /srv/logs/test

echo "svnlook: ${SVNLOOK}" >> /srv/logs/test

echo "authhor: ${AUTHOR}" >> /srv/logs/test

echo "Commited rev: ${REV}" >> /srv/logs/test

echo "Old rev: ${OLDREV}" >> /srv/logs/test

echo "RepoName-REPOS: ${RepoName}" >> /srv/logs/test

echo "Repository URL: ${RepoURL}" >> /srv/logs/test

post-review --repository-url="${RepoURL}" --revision-range="${OLDREV}":"${REV}" --server="${RBSRV}" --username=reviewposter --password="password" --submit-as="${AUTHOR}" -p --target-groups=reviewers --debug >> /srv/logs/post-rev 2>&1

exit 0

/srv/logs/post-rev is writable by www-data. Both stdout and stderr are redirected correctly.

root@borislav-desktop:/srv/svn/testRepo2/hooks# ll -aFl /srv/logs/

drwxr-xr-x 2 www-data www-data 4096 2012-09-26 11:42 ./

drwxr-xr-x 7 root     root     4096 2012-09-25 13:27 ../

-rw-r--r-- 1 www-data www-data    0 2012-09-26 11:44 post-rev

-rw-r--r-- 1 www-data www-data    0 2012-09-25 17:30 superusr

-rw-r--r-- 1 www-data www-data 4007 2012-09-26 11:44 test

Any suggestions?

Thanks, Borislav.

[Christian Hammond]

Hi,

Nothing stands out as being wrong... Are you 100% sure that the username and password being sent are correct?

Something that occurred to me. We don't display the string "Password for '<username>'". So that's coming from something else entirely.

Christian

--
Christian Hammond - chi...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com

[Borislav Sabev]

Hi.

I believe this to be an OS issue where the www-data user does not have rights over /usr/local/bin/post-review I tried adding a right for www-data in the sudoers file, but that does 

not help. I am kinda out of options... The only thing that remains is to try on a different OS (which is not an option at the moment since the SVN server runs on a Ubuntu Server box)

or to set the script up so it runs this specific command as another user.

Given that "you do not display the string "Password for '<username>'"" I could fairly conclude that this is an OS-related issue and seek other ways to deal with it.

Thank you for your fast replies and readiness to help. If there is anything else, please keep me posted.

Cheers,

    Borislav.

[Borislav Sabev]

I have also created a question over at superuser so if anyone has this issue also take a look there:

http://superuser.com/questions/479803/permissions-issue-when-trying-to-execute-command-in-post-commit-hook-in-svn

Cheers, 

    Borislav.

[Christian Hammond]

I didn't see it in your script, but are you running the script as sudo in the hook somewhere?

What distro are you running? Is SELinux turned on?

Christian

--
Christian Hammond - chi...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com

--

[Borislav Sabev]

Hello Christian.

I am not running sudo anywhere in the script.

This is the full hook script I already posted:

=======================================repo/hooks/post-commit script paste start===============================================

REPOS="$1"

REV=$(($2))

OLDREV=$(($REV-1))

RBSRV="http://reviews.example.test/"

SVNLOOK=/usr/bin/svnlook

AUTHOR="$($SVNLOOK author $REPOS -r $REV)"

LogDir="/var/log/svn/"

#Get only the name of the REPOSITORY as REPOS is a filesystem path:

RepoName=${REPOS##*/}

RepoURL="http://xxx.xxx.xxx.xxx/svn/${RepoName}"

#Some debugging info

echo "=====================================" >> /srv/logs/test

whoami >> /srv/logs/test

echo "svnlook: ${SVNLOOK}" >> /srv/logs/test

echo "authhor: ${AUTHOR}" >> /srv/logs/test

echo "Commited rev: ${REV}" >> /srv/logs/test

echo "Old rev: ${OLDREV}" >> /srv/logs/test

echo "RepoName-REPOS: ${RepoName}" >> /srv/logs/test

echo "Repository URL: ${RepoURL}" >> /srv/logs/test

post-review --repository-url="${RepoURL}" --revision-range="${OLDREV}":"${REV}" --server="${RBSRV}" --username=reviewposter --password="password" --submit-as="${AUTHOR}" -p --target-groups=reviewers --debug >> /srv/logs/post-rev 2>&1

exit 0

=================================================repo/hooks/post-commit script paste start===========================================

SELinux is not installed.

Test Platform Distribution is (Ubuntu 10.10): <== This is the platform on which I am experiencing the problem. This has not been tested on the live server.

Linux xxxxxx 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:45:36 UTC 2010 x86_64 GNU/Linux

SVN version is: 1.6.12 (r955767)

Future Server Distribution is (Ubuntu 10.10):

Linux xxxxxxx 2.6.32-41-server #94-Ubuntu SMP Fri Jul 6 18:15:07 UTC 2012 x86_64 GNU/Linux

SVN version is: 1.6.6 (r40053)

Regards, 

    Borislav.

[Borislav Sabev]

Hello again.

I found what command is causing the problem. Here is some ps (BSD syntax) output:

www-data 10259  2.5  0.2  40540  9864 pts/1    S+   09:56   0:00 /usr/bin/python /usr/local/bin/post-review --repository-url=http://xxx.xxx.xxx.xxx/svn/testRepo2 --revision-range=1:2 --server=http://reviews.innovity.test/ --username=reviewposter --password=xxxxx --submit-as=borislav -p --target-groups=reviewers --debug

www-data 10271  0.4  0.0      0     0 pts/1    Z+   09:56   0:00 [svn] <defunct>

www-data 10272  0.4  0.0 113280  3884 pts/1    S+   09:56   0:00 svn info http://xxx.xxx.xxx.xxx/svn/testRepo2

www-data 10273  0.0  0.1 270844  6476 ?        S    09:56   0:00 /usr/sbin/apache2 -k start

root     10279  0.0  0.0  19396  1184 pts/0    R+   09:56   0:00 ps aux

Since the repo hast mod-authz-svn, mod-dav-svn enabled and authentication running it asks for a password for the www-data user:

>>> RBTools 0.4.1

>>> Home = /home/borislav

Password for 'www-data': <== THIS IS ACTUALLY CALLED BY `svn info`

This still happens if I run the post-review command with --http-username= --http-password= options set.

So should this now be considered a bug? That there is no option to set authentication parameters for a remote SVN repo so that handling could be automatic?

Regards, 

[Borislav Sabev]

Also if there could be some kind of flag to tell the post-review command that this repository is local to run the svn info command with:

svn info file:///path/to/repo ?

Then svn info would not ask for a password since authentication is set only on the web access part?

[Borislav Sabev]

Hello, Aaron.

Yes the solution provided over at superuser.com feasible in my opinion. 
I appeal the the RBTools developers again. Could there be a workaround added to the tools so that Automatic Post-Commit reviews are possible with "protected"(authentication & authorization) SVN instances?

I see another workaround - to have a svn mirror running with no authentication, but it makes no sense not to have authentication & authorization possibilities.

Regards, 

   Borislav.

On Friday, 12 October 2012 02:39:33 UTC+3, Aaron Bruce wrote:

Chiming in here with the same issue. It's definitely an svn prompt for a svn username/password. I was trying to comment on your superuser post but I'm not a pro enough superuser user :-)

The solution posted there is bad, he suggests logging into the www-data user and caching your svn credentials. That requires making a www-data svn user and then caching credentials for every single repo you want to use post-review with.

Connect with us on twitter, google+, facebook, or linkedin.

Catch our next training in Houston Oct 8 - 12, Berkeley Oct 22 - 26, Seattle Oct 29 - Nov 2, Phoenix Nov 12 - 16 or See All.

This email, including any attachments, is for the sole use of the intended recipient and may contain confidential information. If you are not the intended recipient, please immediately notify us by reply email or by telephone, delete this email and destroy any copies. Thank you.