VREF : remote: error: hook declined to update
Maneesh M P
Earlier I had another thread related to VREF, did some blunders during that time and everything got messed up.I did learn some lessons though. Now I have another issue related to this, hopefully my question is not stupid again :(
we have a tools repo, which no one except @gatkeepers can push to "production" branch or master branch. We have a remote branch called "integration" where all the development happens, also people are allowed to create remote private branches.
Today we have created some additional VREF rules for individual component team to allow modification only in their module [like, for @s-team can modify only files under modules/p_s/ directory]
Below is my gitolite conf file
repo operatons/tools
RW+ = @gatekeepers
R production* master = @operations-dev
- production* master = @operations-dev
RW+ = @operations-dev
RW+ VREF/NAME/modules/p_d/ = @d-team
RW+ VREF/NAME/modules/p_s/ = @s-team
RW+ VREF/NAME/modules/p_k/ = @k-team
RW+ VREF/NAME/modules/p_p/ = @p-team
RW+ VREF/NAME/modules/p_a/ = @a-team
- VREF/NAME/ = @d-eam @s-team
- VREF/NAME/ = @k-team @p-team
- VREF/NAME/ = @a-team
config hooks.run = pgitrepopush
The issue I am facing now is, if a someone from @s-team does below operations,
> git clone g...@abc.com:operations/tools.git
> git branch test_branch
> git checkout test_branch
Modify any files under /modules/p_s/ directory
> git push origin test_branch
This push fails
maneesh@maneesh:~/git/tools$ git push origin test_branch
Counting objects: 30, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 411 bytes | 0 bytes/s, done.
Total 5 (delta 4), reused 0 (delta 0)
remote: FATAL: W VREF/NAME/.gitignore operations/tools man...@abc.com DENIED by VREF/NAME/
remote: error: hook declined to update refs/heads/test_branch
To g...@abc.com:operatons/tools
! [remote rejected] test -> test (hook declined)
error: failed to push some refs to 'g...@abc.com:operatons/tools.git'
Am I doing something wrong here? Or is this working as expected?
Thanks,
Maneesh
Sitaram Chamarty
> Earlier I had another thread related to VREF, did some blunders during
> that time and everything got messed up.I did learn some lessons
> though. Now I have another issue related to this, hopefully my
> question is not stupid again :(
something obvious does happen...
[snipped out everything except the relevant rules]
> RW+ VREF/NAME/modules/p_s/ = @s-team
> Modify any files under /modules/p_s/ directory
>> git push origin test_branch
>
> This push fails
and it's not *within* modules/p_s! Of course the push would fail!
Maneesh M P
Below is the commands tried in sequential order to be specific,
maneesh@maneesh:~/git/tools$ git branch test_branch
maneesh@maneesh:~/git/tools$ git checkout test_branch
maneesh@maneesh:~/git/tools/modules/p_s$ vi README
maneesh@maneesh:~/git/tools/modules/p_s$ git add README
maneesh@maneesh:~/git/tools/modules/p_s$ git commit -m "updating readme file to
test commit"
[test_branch 64ad163] updating readme file
1 file changed, 1 insertion(+), 1 deletion(-)
maneesh@maneesh:~/git/tools/modules/p_s$ git push origin test_branch
Counting objects: 30, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 415 bytes | 0 bytes/s, done.
Total 5 (delta 4), reused 0 (delta 0)
remote: FATAL: W VREF/NAME/.gitignore operations/p_s man...@abc.com DENIED by VREF/NAME/
remote: error: hook declined to update refs/heads/test_branch
To g...@abc.com:operations/tools.git
! [remote rejected] test_branch -> test_branch (hook declined)
error: failed to push some refs to 'g...@abc.com:operations/tools.git'
Thanks,
Maneesh
Sitaram Chamarty
> p_s team members were not modifying the .gitignore file. They were
> basically updating a README file for testing the VREF rules
changes to files:
This says the .gitignore file was changed.
Perhaps you had a previous commit that contained a change to that file,
which was not yet on the server.
You do know that "push" means "ALL commits not already on the server",
not just "the latest commit" right?
Maneesh M P
maneesh@maneesh:~ git push origin test_branch
...........
remote: FATAL: W VREF/NAME/toolsfile operations/tools man...@abc.com DENIED by VREF/NAME/
remote: error: hook declined to update refs/heads/test_branch
To g...@abc.com:operations/tools.git
! [remote rejected] test_branch -> test_branch (hook declined)
error: failed to push some refs to 'g...@abc.com:operations/tools.git'
Now I feel the VREF rule is creating the problem.
The rule =>
RW+ VREF/NAME/modules/p_s/ = @s-team
- VREF/NAME/ = @s-team
The directory structure of repository is,
maneesh@maneesh~:pwd
/home/maneesh/git/tools
maneesh@maneesh~:ls
modules toolsfile
maneesh@maneesh~: ls modules/
p_d p_k p_p p_s
so when the @s-team makes modification to files under directory modules/p_s/
and try to push, the deny rule is coming into picture for "toolsfile"
Thanks,
Maneesh
Sitaram Chamarty
line to better reflect the issue)
On 08/06/2014 10:00 AM, Maneesh M P wrote:
> Yes, I understand that part. To test further we removed git
> .gitignore file and executed the push command, So the error is
> different.
the "old SHA" is basically an empty tree, and the new SHA is your
current tree.
This is a problem that has come up once or twice in the past, but we've
not done anything about it. I'm open to discussion on how to solve it.
This will happen for branch *deletions* also.
Solutions I can think of (there may be others):
1. Have someone who has permissions create that branch from the current
"master" or whatever. Then your @s-team member can push changes to
files in modules/p_s because now the "old SHA" is much more recent
and not an empty tree.
For deletions have the same kind of person delete the ref.
2. Give users a "command" do that so they don't have to call anyone
else. The command will simply check if they're allowed to create
the branch, and create it, setting it to whatever reference branch
they named.
Same for deletions.
Users will first run that command then try the push.
This is essentially the same as the previous method, except you
don't have to call someone else.
3. Modify the VREF/NAME/ feature to something like this:
VREF/NAMENEW/foo
NAMENEW is a VREF that compares "new SHA" against "old SHA" if it is
non-empty, but if it is empty it compares it to some branch that is
contained within new SHA.
Finding that branch will require some coding (probably involving a
grep with the output of 'git show-ref' against a loop of HEAD and
its ancestors, one by one. And I don't know how to do that cleanly
when there is a merge in the ancestry and the other path would have
given you a "ref" that is closer to your recent commits.
Regardless, this will be slightly less efficient than the built-in
NAME VREF code if you have several rules that apply to a user.
For deletions there's no obvious way to use this method.
4. Same as above, but we add option lines like
option branch.base = master for testbranch
and instead of trying to find out that branch we just set the option
and use that instead. No digging through the output of git-show-ref
etc.
Maneesh M P
Regards,
Maneesh(മനീഷ് )
Sitaram Chamarty
> Yes, It is a new branch. Currently we followed the option 1. If we
> could make option 2 work, that will be good thing i think, then
> developers don't need to always go in search of people who have rights
> to create the branch
weigh in.
if no one else does, I'll code that up this weekend.
>
> Thanks,
> Maneesh
>
>
> On Wed, Aug 6, 2014 at 1:45 AM, Sitaram Chamarty <sita...@gmail.com <mailto:sita...@gmail.com>> wrote:
>
> (NOTE: I'd like everyone to read this email; I also changed the subject
> line to better reflect the issue)
>
> On 08/06/2014 10:00 AM, Maneesh M P wrote:
> > Yes, I understand that part. To test further we removed git
> > .gitignore file and executed the push command, So the error is
> > different.
>
milk
miss corner cases.
4 is ok as it gives an explicit mapping, but it isn't general enough
(will need to allow wildcard for branch names to make this scale well)
and will most likely still involve 1 and/or 2 when it doesn't apply.
1 is in place already, but it means users are blocked on other users.
2 looks fine, but additional logic might be needed to determine
"valid" reference branches. I'm not certain if all branches are valid.
In practice, I would usually create VREFs that determine the valid
branches they would apply to. Being able to "configure" VREF/NAME/
makes sense to me, but the gitolite-options method is probably too
limited for general configuration.
-milki
Sitaram Chamarty
> I don't like 3. Adding branch searching logic will most likely mean we
> miss corner cases.
> 4 is ok as it gives an explicit mapping, but it isn't general enough
> (will need to allow wildcard for branch names to make this scale well)
> and will most likely still involve 1 and/or 2 when it doesn't apply.
really predict, so we shouldn't even try to!
> 1 is in place already, but it means users are blocked on other users.
>
> 2 looks fine, but additional logic might be needed to determine
> "valid" reference branches. I'm not certain if all branches are valid.
I think I'll just choose option 1 for now (i.e, do nothing!) until I see
a lot of angst about this :-)
Maneesh M P
Sitaram Chamarty
> Would it be possible to get a patch file for option 2 ? That would
> really help my project.
command, so it's not a "patch" per se.
Put this in "commands" and test it out, then let me know so I can commit
and push it.
Please note that, as usual, the code is basically 2 lines, and a few
more scaffolding; the rest is comments/usage message. And it's just as
easy in shell too by the way.
Maneesh M P
Sitaram Chamarty
On 08/11/2014 11:11 AM, Maneesh M P wrote:
> I saved the attached file with the name "branch" under /home/git/gitolite/src/commands/ in the server.
> Then added the coomand 'branch' inside /home/git/.gitolite.rc file
>
> ENABLE => [
>
> # COMMANDS
>
> # These are the commands enabled by default
> 'help',
> 'desc',
> 'info',
> 'perms',
> 'writable',
>
> # Uncomment or add new commands here.
> # 'create',
> # 'fork',
> # 'mirror',
> # 'sskm',
> # 'D',
>
> Executed the command ssh g...@git.abc.com <mailto:g...@git.abc.com> branch branch-test maneesh_test master, but got the below error
Maneesh M P
Sitaram Chamarty
> Ok file permission was the issue. changed the permission to 775, worked like a charm :) Thank you
why 775?
Are you the system administrator for only this machine or a lot more than just this ;-)
Maneesh M P
Carl Thompson
>
> the "old SHA" is basically an empty tree, and the new SHA is your
>
> current tree.
>
>
> This is a problem that has come up once or twice in the past, but we've
>
> not done anything about it. I'm open to discussion on how to solve it.
>
This problem also exists whenever a tag is being pushed since the old tree is the empty tree. In my company's use-cases, it's extremely rare that any new commits are pushed as part of creating a tag. Typically the branch reference is pushed first, then the commit is tagged. Would it be possible to avoid this VREF entirely for tag creation?
Sitaram Chamarty
Sitaram Chamarty
Sorry I took so long to reply but I was offline the whole weekend (and
during the week it's impossible unless it's urgent).
First, the "facts" as I see them:
1. HOW: The only sure way of detecting such situations is to use
something like
git log --name-only $newSHA --not --all
If people have better ideas please respond. (Recap: need to find
commits that are in this push but only if those commits don't
already exist on the server -- perhaps as part of another branch or
tag).
2. COMPATIBILITY: The behaviour of the existing NAME VREF cannot be
changed, in the interests of backward compatibility.
Next, my thoughts on how we can handle this:
1. The detection method may be slow (I'm not sure how slow but in a
large repo it might add quite a bit. Anyone with more than wild
guess please speak up!!)
One way to solve that is to somehow make it clear to gitolite
*which* specific ref it should be checking against.
That is, instead of '--not --all', if we can get away with '--not
master' or something, it would definitely be faster.
2. That is easiest done if we could have a command like this:
ssh git@host copy-ref <repo> <old ref> <new ref>
People would run this command first, and then do the push.
PROS: no need for a new VREF; existing one works fine after this
command has been run. For tags this command is all you need.
CONS: existing infra that pushes tags may need to be retrained to
run this command instead.
3. The other way to do this would be to create a brand new VREF that
does the "--not --all" thing. Maybe VREF/NEWFILES or something.
This would do things the long way (using '--not --all').
Carl Thompson
> 1. HOW: The only sure way of detecting such situations is to use
> something like
>
> git log --name-only $newSHA --not --all
>
> If people have better ideas please respond. (Recap: need to find
> commits that are in this push but only if those commits don't
> already exist on the server -- perhaps as part of another branch or
> tag).
It wouldn't be as accurate or thorough as your git-log command, but could you use
git branch --contains $newSHA
and
git tag --points-at $newSHA
instead? This would at least be a relatively quick way to determine if the new commit is already contained within another branch or pointed at by an existing tag.
>
> 3. The other way to do this would be to create a brand new VREF that
> does the "--not --all" thing. Maybe VREF/NEWFILES or something.
> This would do things the long way (using '--not --all').
I think a new VREF would be best for my use-cases. This would allow me to set the behavior in the gitolite config files, and the users won't have to learn (or remember) to do anything different than what they're currently doing.
Sitaram Chamarty
>>
>> 1. HOW: The only sure way of detecting such situations is to use
>> something like
>>
>> git log --name-only $newSHA --not --all
>>
>> If people have better ideas please respond. (Recap: need to find
>> commits that are in this push but only if those commits don't
>> already exist on the server -- perhaps as part of another branch or
>> tag).
>
>
> It wouldn't be as accurate or thorough as your git-log command, but
> could you use
>
> git branch --contains $newSHA
> and
> git tag --points-at $newSHA
>
> instead? This would at least be a relatively quick way to determine if
> the new commit is already contained within another branch or pointed
> at by an existing tag.
the most common situation, but I think this:
a --- b --- c --- d (master) --- e (new branch being pushed)
is also pretty common. The diff(d, e) does NOT touch any prohibited
files. Using --contains would say "no there's no branch that contains
this", but then what do you do? You still have to fall back on "--not
--all".
>> 3. The other way to do this would be to create a brand new VREF that
>> does the "--not --all" thing. Maybe VREF/NEWFILES or something.
>> This would do things the long way (using '--not --all').
>
> I think a new VREF would be best for my use-cases. This would allow me
> to set the behavior in the gitolite config files, and the users won't
> have to learn (or remember) to do anything different than what they're
> currently doing.
Carl Thompson
> test.
>
The patch works great. I tested the new command on a repository with around 90k commits, and did not notice a performance hit. I was able to successfully push new tags to the repository as long as they were on previously-pushed commits. Tags on new commits were still rejected, as expected. Thanks!