Pull request tracking and limiting access to merge to master

[Reuben Popp]

Good morning (or afternoon/evening) all, the powers that be have dictated that they want average developers to be able to merge to origin/master, essentially saying if you break $project, you get to fix it.  This puts the onus of ensuring changes work and do not break someone else's changes upstream.  It also relieves some of the burden of merging from the architect team (ours is small, and they also tend to do a good deal of development in addition to design).

All that said, is anyone doing some kind of pull request tracking, or have some kind of scaffolding set up to prevent a merge until such time as a pull request is issued?  Technically, we have code reviews, but those are not tied into the system other than just for a review.  There's no enforcement between the code review system (Atlassian Jira) and the repository.

Ideally, as a CYA measure, I would prefer to have some kind of limitation that restricts a merge to master until a pull request is defined for that merge (change?) set.  Anyone have something like this in place already that they can share, or does anyone have any suggestions to implement this?

Thanks in advance, as always
Reuben

[Stephen Morton]

Warning: going off on a bit of a tangent here. Stop now if you're not up for that...

Regarding gitolite and pull requests...

I love gitolite. I love that it is "bare-metal" and there's so little to go wrong. OpenSSH+perl, period. There's no Java, no Database, no Celery Workers --which may have a place powering a GUI, but have no place, IMO, in the "fastpath" of fetching and pushing. I love that when some end-user asks Sitaram for something that makes little sense, he politely does not implement it. And when somebody gives Sitaram a patch, he re-implements the patch in 1/3 the original number of LOC before deploying. And that's why we deploy gitolite. I serve very large git repos to 1,000 people all across the globe, with various different ACLs, with local read-only mirrors and it just works.

However... to be efficient in today's world, programmers want/need more than just a git that works really well. They need CI/CD; code reviews; pull requests that may trigger code reviews and/or CI/CD; graphical browsing of git history; etc. You can bolt this on to Gitolite via reviewboard, Jenkins, gitweb, etc. But it feels bolted on. We have actually bolted on Atlassian Bitbucket (actually an older version from when it was called "Stash") but it was never meant to be used that way. Stash always assumes that there is no upstream to it. Right now we just use it as a graphical git viewer and a way for users to create git forks (we hack these into gitolite so that they automagically appear as wildcard repos). CI/CD we rely more and more on GitLab; it has a pretty awesome system. 

    In an ideal world, things would follow the linux model of "do one thing and do it well" and we'd combine gitolite with the really good code review tool and the really good CI/CD tool and the really good pull request tool. But to be honest, the ecosystem today does not look like that. (I'm sure you could give me examples of tools that are good and I don't want to argue details, but compared to just installing say GitLab, those tools are probably not as good and the whole thing would be harder to administer. Understand that everything I implement has to be inside a corporate firewall so all the fancy bells and whistles you can just clickety-click put together in the cloud, I have to deploy 100% from scratch myself; and many cloud-based tools are not available as self-hosted options.)

    (This would all probably be slightly easier if you could say "everybody must use pull requests" but if you have a rule that "pull requests are optional, and you can also just push", having git+<other tool that does pull requests but is unaware that it has to merge its changes into an ever-changing upstream> becomes more and more problematic. Yes, this sounds kind of crazy, but that's the way it is.)

    We could keep going with gitolite serving git --because we trust it to be rock-solid, and the other reasons listed above-- and bolting on other tools to perform other functions. But they will become harder and harder to administer, they won't interoperate well, and there will always be weird rules that "Oh, you can't actually use feature X" or "You can use feature Y, but only to do this... not if you want to do that..."

All that to say that we are (very very reluctantly and with heavy hearts) considering moving away from gitolite relating to reasons hinted at by Rueben's question.

I don't mean this to be a rant, or a total off-topic email. I send this hoping that it somehow sparks some useful discussion or insight.

Stephen

--
You received this message because you are subscribed to the Google Groups "gitolite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gitolite+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gitolite/CAJYAvwAPvXQZB6qJRmFCtOL22aaRdqgCaUUNiOY8BckWnh_dqA%40mail.gmail.com.

[Sitaram Chamarty]

Hi

I just responded to you on IRC, without checking my email first :)

Anyway back to this... the "hub" ADC you mentioned on IRC has
not been migrated, but should be easy enough if you need it.
However, reading your message above, another alternative is the
"VOTES" VREF, which *is* in v3, although it's been ages since I
wrote it (2012!) and I'm not sure if anyone is using it so do
test it.

The thing is, if it doesn't have to browser based, a lot can be
done in gitolite. Let me know what you need and we can work on
it.

Another thing I mentioned on IRC: gitolite "commands" and
"triggers" can be in any language; you'll see many of them in
shell for example. So if you want to take a crack at it, go
ahead.

The starting point for most of the relevant documentation is
https://gitolite.com/gitolite/non-core.html. Specifically:

* https://gitolite.com/gitolite/non-core.html#gitolite-commands
* https://gitolite.com/gitolite/list-non-core.html#commands
* https://gitolite.com/gitolite/dev-notes.html

Let me know if you need something else from me.

sitaram

PS: Just as a side note, due to Covid and WFH and mandatory
proprietary tools and my unwillingness to put my personal ssh
keys on my work laptop, I now can look at/respond to gitolite
related stuff only once in the morning and once at night. Sorry
about that!

[Sitaram Chamarty]

[Stephen: you'll get this twice because I forgot to "reply all" the
first time; sorry about that]

Stephen

I've always understood that this will eventually happen to most
projects, especially the corporate ones.

Even on the open source side, KDE has moved away. For Fedora
it's only one of the backends that Pagure supports, albeit an
important one I imagine. The day I heard Xfce was moving was
the first time I even knew they had been using gitolite at all.

Gentoo and a few others are probably still using it, but I can't
be 100% sure. Honestly, I see only kernel.org as a long term
thing, simply because of their development process -- and I hope
I haven't jinxed it by mentioning it ;-)

It's very kind of you to say all this, but please don't feel
bad. I'm very clear that (a) there are far too many candidates
in *that* space already, so I have no wish to go there and (b)
there's literally nothing else in *this* space -- i.e., just
access control plus a few extras but nothing browser based, so
that's yet more reason not to move out of this niche.

But all said and done it has become a niche, and niches, by
their very nature, are not heavily populated :)

regards
sitaram

On Mon, Oct 12, 2020 at 08:30:18PM -0400, Stephen Morton wrote:
> Warning: going off on a bit of a tangent here. Stop now if you're not up
> for that...
>
> Regarding gitolite and pull requests...
>
> I love gitolite. I love that it is "bare-metal" and there's so little to go
> wrong. OpenSSH+perl, period. There's no Java, no Database, no Celery
> Workers --which may have a place powering a GUI, but have no place, IMO, in
> the "fastpath" of fetching and pushing. I love that when some end-user asks
> Sitaram for something that makes little sense, he politely does not
> implement it. And when somebody gives Sitaram a patch, he re-implements the
> patch in 1/3 the original number of LOC before deploying. And that's why we
> deploy gitolite. I serve very large git repos to 1,000 people all across

> the globe, with various different ACLs, with local read-only mirrors and *it
> just works*.

>
> However... to be efficient in today's world, programmers want/need more
> than just a git that works really well. They need CI/CD; code reviews; pull
> requests that may trigger code reviews and/or CI/CD; graphical browsing of

> git history; etc. You *can* bolt this on to Gitolite via reviewboard,

> Jenkins, gitweb, etc. But it feels bolted on. We have actually bolted on
> Atlassian Bitbucket (actually an older version from when it was called
> "Stash") but it was never meant to be used that way. Stash always assumes
> that there is no upstream to it. Right now we just use it as a graphical
> git viewer and a way for users to create git forks (we hack these into
> gitolite so that they automagically appear as wildcard repos). CI/CD we
> rely more and more on GitLab; it has a pretty awesome system.
> In an ideal world, things would follow the linux model of "do one thing
> and do it well" and we'd combine gitolite with the really good code review
> tool and the really good CI/CD tool and the really good pull request tool.
> But to be honest, the ecosystem today does not look like that. (I'm sure
> you could give me examples of tools that are good and I don't want to argue
> details, but compared to just installing say GitLab, those tools are
> probably not as good and the whole thing would be harder to administer.
> Understand that everything I implement has to be inside a corporate
> firewall so all the fancy bells and whistles you can just clickety-click
> put together in the cloud, I have to deploy 100% from scratch myself; and
> many cloud-based tools are not available as self-hosted options.)

> (This would all probably be *slightly* easier if you could say

> > <https://groups.google.com/d/msgid/gitolite/CAJYAvwAPvXQZB6qJRmFCtOL22aaRdqgCaUUNiOY8BckWnh_dqA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> > .

> >
>
> --
> You received this message because you are subscribed to the Google Groups "gitolite" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gitolite+u...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/gitolite/CAH8BJxE%3DMcS%2BHU9Bv_x1T0tQev5pvJgxCqcDj4%2B5Yj0qLEjOBg%40mail.gmail.com.

[Konstantin Ryabitsev]

On Tue, Oct 13, 2020 at 10:41:00PM +0530, Sitaram Chamarty wrote:
> Gentoo and a few others are probably still using it, but I can't
> be 100% sure. Honestly, I see only kernel.org as a long term
> thing, simply because of their development process -- and I hope
> I haven't jinxed it by mentioning it ;-)

Also, codeaurora.org, which, with all forks of Android, is now to 30,000
repositories managed via gitolite. If you ever want to hear a good rant,
buy me a drink and ask me about replicating 30,000 repositories from
North America to Beijing over any kind of encrypted link. :)

Yocto project is also using it for git.yoctoproject.org. That's just
those that I'm directly aware of.

Gitolite provides us maximum transparency and auditability for all
changes, so we are not interested in switching to any other solution at
this time.

Best regards,
-K

[Reuben Popp]

So, my intent wasn't to knock gitolite or Sitaram.  I was merely fishing for a potential solution (or even a partial) solution to the question at hand,  I like the fact that gitolite is stripped down and is flexible enough for me to hook other things into it with just a little work.  After all, it's not like I'm trying to make AD play nicely with OpenLDAP (for any who's had that joy haha).  Where I am now I have deployed gitolite as the backend server and cgit for the web interface and I find that the two work great together.  While I like the other projects available, I find that many try to be too much like Github without offering the ability to disable unneeded features.  All I look for (here) is the ability to manage the repository in a fairly easy way, and a fairly simple frontend I can point devs who are NOT CLI savvy to for a uncomplicated visual reference.

That said, I am also a developer here.. kind of like a dev ops person, but my official title is a developer.  My previous position was doing sysadmin for a community college, so yes, i am familiar with the pain of dealing with lots of fingers, lots of pots and lots of cooks.  I like simple, and I like to think that the rest of the dev/arch team here likes it as well.  As our team moves forward with its goal of CI/cloud deployments, we're looking for how code reviews (or similar) fall into line, especially as it pertains to git.  If the entire team can merge into trunk, then at what point are we doing our due diligence to protect the sanctity of the  build of MASTER so that it doesn't break and we can also ward off any auditors when they ask for peer reviews?

Anyway, that's where I'm coming from.  I have no issue trying to roll my own solution if it fits in well in gitolite's framework; I was just curious if others had done something similar.

Reuben

--
You received this message because you are subscribed to the Google Groups "gitolite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gitolite+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gitolite/20201013200838.wpuc4tab265gaybw%40chatter.i7.local.

[Robert Coup]

Hi Reuben,

Many tools manage reviews through alternative refs (refs/pull/{num}/{head|merge} in Github; refs/for/{branch} in Gerrit, etc) — would a simple approach be for a pre-receive hook to check for a matching PR ref existing before allowing the ref to be updated? For example. does a refs/pull/*/head exist as a parent as the current merge commit that's being pushed to the protected branch (eg: main).

Rob :)

[Sitaram Chamarty]

On Tue, Oct 13, 2020 at 03:59:19PM -0500, Reuben Popp wrote:
> So, my intent wasn't to knock gitolite or Sitaram. I was merely fishing

I didn't take it as knocking *at all* :)

> Anyway, that's where I'm coming from. I have no issue trying to roll my
> own solution if it fits in well in gitolite's framework; I was just curious
> if others had done something similar.

If you wouldn't mind thinking aloud, so to speak, on this list,
that might be very interesting. Even if I am not willing to
move gitolite in the GUI direction (for reasons in my previous
email), I'd be curious to hear about it.

regards
sitaram

[Sitaram Chamarty]

On Tue, Oct 13, 2020 at 04:08:38PM -0400, Konstantin Ryabitsev wrote:
> On Tue, Oct 13, 2020 at 10:41:00PM +0530, Sitaram Chamarty wrote:
> > Gentoo and a few others are probably still using it, but I can't
> > be 100% sure. Honestly, I see only kernel.org as a long term
> > thing, simply because of their development process -- and I hope
> > I haven't jinxed it by mentioning it ;-)
>
> Also, codeaurora.org, which, with all forks of Android, is now to 30,000
> repositories managed via gitolite. If you ever want to hear a good rant,
> buy me a drink and ask me about replicating 30,000 repositories from
> North America to Beijing over any kind of encrypted link. :)

Oh I did not know this; thanks!

> Yocto project is also using it for git.yoctoproject.org. That's just
> those that I'm directly aware of.
>
> Gitolite provides us maximum transparency and auditability for all
> changes, so we are not interested in switching to any other solution at
> this time.

:-)

regards
sitaram

[Sitaram Chamarty]

I mentioned the "VOTES" VREF in an earlier mail. I'm sure that
can be repurposed to what you say.

Basically, if there exists a set of commands on the server that
can determine a yes/no answer, you can put it in a VREF and use
that VREF in the gitolite.conf file.

So one could have, for example, the following flow:

- Alice submits PR

git push origin xyz:submitted-PRs/alice/xyz

The "xyz" etc can be mandated (like "bug-1234") or
arbitrary; gitolite wouldn't care.

- People who have authority to approve PRs can look at the
submitted PRs at regular intervals or we could work out some
sort of alerting system if the backend is able to send mails
or whatever.

- Once approved, Bob would

git push origin xyz:approved-PRs/xyz

We'd also have a trigger that gets invoked on a successful
push, which would look at the ref pushed and *delete* the
corresponding "submitted-PR", checking that the hash is the
same.

- At that point, someone else could take all approved PRs and
merge them.

Or Bob could do that directly if he has authority.

The problem with all this is that channels of communication for
minor back-and-forth on these PRs are out-of-band.

As I understand it from others I have spoken to, that is a big
part of the attraction of things like github/lab/etc. Not just
that it is GUI, which is of course a big thing, but because
having that GUI enables a lot more interaction, commenting, etc.

regards
sitaram