Gitolite newbie who's not understanding how to mirror

[cseliasl...@gmail.com]

Hi there,

So I read the mirroring documentation on the gitolite site, but I'm not fully understanding it.

Here's what I'm trying to do: I have a server with a bunch of gitolite repos, and I would like another server to mirror all of them, as backup (that's all).

Are all the instructions on the site absolutely necessary to accomplish that? Again, all I need to do is to have a backup of the repos, such that when someone pushes their work to the main server, my backup server gets it too. No one will ever directly push to the mirror server.

I'm totally new to gitolite, and I was sort of thrown into this task (which needs to be accomplished right away), and I find the instructions on the site to be kind of vague and cryptic (no offense), probably because it was written for more advanced gitolite folks.

Is there anyone here who will not mind to provide me with an extremely dumb down, less-complex set of instructions for this task (if such a set of instructions exist)?

Thank you.

[Sitaram Chamarty]

You don't have to use gitolite mirroring.

Just create a post-receive hook that deduces the current repo
name (or just uses $GL_REPO, since gitolite conveniently sets
that up), and does a mirror push to the slave.

Maybe as simple as

#!/bin/bash

SLAVEHOST=slave.example.com

git push --mirror $SLAVEHOST:/path/to/$GL_REPO.git

Then use the instructions in [1] to setup this hook.

[1]: http://gitolite.com/gitolite/cookbook/#adding-other-non-update-hooks

Of course, you must make sure any new repos you add to the
gitolite.conf are also (manually) created on the slave. You
must also understand that if you use wild repos, some things may
not work right -- like permissions being saved to the mirror.

There're probably a few more things gitolite mirroring does,
which I am forgetting, but at the very core of it, for the
simplest use case, you can get by without it.

rgds
sitaram

PS: I have not tested that script; please test it in order to
use it!

[Sitaram Chamarty]

On Wed, Jun 20, 2018 at 08:18:58AM -0700, cseliasl...@gmail.com wrote:

> probably because it was written for more
> advanced gitolite folks.

s/folks/needs/

:-)

[Stephen Morton]

The gitolite mirroring also handles throttling the mirror pushes so that if you get a flurry of pushes to your repo, you are not running n simultaneous mirror pushes. For those with highly active repos, this can be critical.

Steve

--
You received this message because you are subscribed to the Google Groups "gitolite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gitolite+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[cseliasl...@gmail.com]

Thank you for the prompt reply! That certainly seems much easier than the other instructions. I have a few other questions..

Would I still need to generate ssh-key pairs for the server with the "server-" prefix? If so, in what directory should they be stored? The instructions say "common area" but I'm not sure what would qualify as that. I'm assuming $HOME.

Also, when I manually create the repos on the mirror server, what permissions should each repo have in the gitolite.conf file on the mirror server? Do I put the ip address of the master server? Something like this "git@<ip-address>"?

Thanks, again.

[cseliasl...@gmail.com]

Okay, so I implemented your suggestion, and this is the output I received after attempting to push to the master server.

Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (1/1), done.
Writing objects: 100% (3/3), 308 bytes | 28.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: hello
remote: git2@aec515ld-08
remote: remote: Empty compile time value given to use lib at hooks/update line 6.
remote: remote: Use of uninitialized value in require at hooks/update line 7.
remote: remote: Can't locate Gitolite/Hooks/Update.pm in @INC (you may need to install the Gitolite::Hooks::Update module) (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at hooks/update line 7.
remote: remote: BEGIN failed--compilation aborted at hooks/update line 7.
remote: remote: error: hook declined to update refs/heads/master
remote: To aec515ld-08:repositories/demo
remote: ! [remote rejected] master -> master (hook declined)
remote: error: failed to push some refs to 'git2@aec515ld-08:repositories/demo'
To aec515ld-08:demo
8a37547..21b1e83 master -> master

I added two echos in the script, so as to make sure it's even reading the hook. These echos outputted these two lines you see above:
hello
git2@aec515ld-08

Any idea what might be wrong?

[Sitaram Chamarty]

In the scheme I suggested to you, the slave is NOT a "gitolite server";
it's just a plain ssh based git server that the post-receive hook script
in the master simply pushes to.

Did I misunderstand your *original* email? I thought you wanted
something extremely simple that would just act as a live backup.

If you want the slave to be a proper gitolite server, please scrap this
scheme and go back to the documentation and do it the "official" way.

rgds
sitaram

--
sitaram

[Sitaram Chamarty]

Excellent point Stephen; thank you for reminding me.

Indeed, an active repo can easily choke the master (and the slave)
trying to make multiple mirror pushes at the same time. In fact, as I
recall, gitolite ensures that if a push is already running, the next
attempt gets queued, but while one is queued, nothing else gets queued
-- because there's no point!

It's implemented in a script called "1plus1".

PS: I'm usually terrible at names but "1plus1" is kinda obvious so I
guess it's not bad :-)

regards
sitaram

On 06/20/2018 10:05 PM, Stephen Morton wrote:
> The gitolite mirroring also handles throttling the mirror pushes so that
> if you get a flurry of pushes to your repo, you are not running

> /n/ simultaneous mirror pushes. For those with highly active repos, this

> can be critical.
>
> Steve
>
>
> On Wed, 20 Jun 2018 at 11:48, Sitaram Chamarty <sita...@gmail.com
> <mailto:sita...@gmail.com>> wrote:
>
> On Wed, Jun 20, 2018 at 08:18:58AM -0700, cseliasl...@gmail.com
> <mailto:cseliasl...@gmail.com> wrote:
>
> > probably because it was written for more
> > advanced gitolite folks.
>
> s/folks/needs/
>
> :-)
>
> --
> You received this message because you are subscribed to the Google
> Groups "gitolite" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to gitolite+u...@googlegroups.com

> <mailto:gitolite%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

--
sitaram

[cseliasl...@gmail.com]

On Wednesday, June 20, 2018 at 8:21:14 PM UTC-4, Sitaram Chamarty wrote:
> In the scheme I suggested to you, the slave is NOT a "gitolite server";
> it's just a plain ssh based git server that the post-receive hook script
> in the master simply pushes to.
>
> Did I misunderstand your *original* email? I thought you wanted
> something extremely simple that would just act as a live backup.
>
> If you want the slave to be a proper gitolite server, please scrap this
> scheme and go back to the documentation and do it the "official" way.
>
> rgds
> sitaram
>

You're absolutely correct. All I need is something extremely simple to act as a backup. I don't know why, but I assumed the slave had to be gitolite server as well. I'll retry again with two new servers, with one as non-gitolite. Thanks!

[cseliasl...@gmail.com]

It worked! Thank you very much for your help! :)

[Ali Reza Kayvan]

On Thursday, June 21, 2018 at 11:50:09 PM UTC+4:30, cseliasl...@gmail.com wrote:
> It worked! Thank you very much for your help! :)

I have a question . Couldnt it be done with a simple scp or rsync . if its just a backup server why not copping the whole thing into another server . Isnt it a right solution ?

[Sitaram Chamarty]

sure that'll work too.

Pros: ancillary gitolite files like gl-perms etc., will get
copied.

Cons: (1) possibly lots of needless traffic if git on the master
did a repack or gc, (2) I have no idea about race conditions if
an rsync is happening *while* a ref is being updated. Probably
safe, but I don't actually know.

Finally, I like to have receive.fsckObjects set on my slaves;
then, using git transport instead of file system transport, I
think provides an extra layer of confidence that the data was
good (for some definition of "good").

sitaram