hostname resolution problem in mirroring

[Ming]

Hi,
i was following instructions on
http://gitolite.com/gitolite/mirroring.html
to set up mirroring for the very first time.

In my ~/.ssh/config file on both master and slave, i've the following:

=============================
host GitMaster
user gitolite
hostname 192.168.1.101
port 22

host GitSlave
user gitolite
hostname 192.168.1.115
port 22
=============================

i also have the following line (uncommented) in ~gitolite/.gitolite.rc:
HOSTNAME => "GitSlave",

i noticed the following in the log:
=============================
....
2014-10-06.01:03:46 11893 system,/home/gitolite/gitolite/src/commands/mirror,push,GitSlave,gitolite-admin
2014-10-06.01:03:46 11970 TID=11893 host=GitSlave repo=gitolite-admin,gitolite mirror push started
2014-10-06.01:03:46 11970 mirror: ssh: Could not resolve hostname gitslave: Name or service not known<<newline>>
2014-10-06.01:03:46 11970 mirror: fatal: Could not read from remote repository.
=============================

As you can see in the log:
host=GitSlave
but the error is about:
Could not resolve hostname gitslave: ...

Any idea where i screwed up?

Thanks for reading.

[Sitaram Chamarty]

On 10/06/2014 11:07 AM, Ming wrote:
> Hi,
> i was following instructions on
> http://gitolite.com/gitolite/mirroring.html
> to set up mirroring for the very first time.
>
> In my ~/.ssh/config file on both master and slave, i've the following:
>
> =============================
> host GitMaster
> user gitolite
> hostname 192.168.1.101
> port 22
>
> host GitSlave
> user gitolite
> hostname 192.168.1.115
> port 22
> =============================

Did you then test this, as per bullet 6 in section 3.1? I.e., run
'ssh GitMaster info' on GitSlave, and vice versa.

If so, what was the result?

> i also have the following line (uncommented) in ~gitolite/.gitolite.rc:
> HOSTNAME => "GitSlave",
>
> i noticed the following in the log:
> =============================
> ....
> 2014-10-06.01:03:46 11893 system,/home/gitolite/gitolite/src/commands/mirror,push,GitSlave,gitolite-admin
> 2014-10-06.01:03:46 11970 TID=11893 host=GitSlave repo=gitolite-admin,gitolite mirror push started
> 2014-10-06.01:03:46 11970 mirror: ssh: Could not resolve hostname gitslave: Name or service not known<<newline>>
> 2014-10-06.01:03:46 11970 mirror: fatal: Could not read from remote repository.
> =============================
>
> As you can see in the log:
> host=GitSlave
> but the error is about:
> Could not resolve hostname gitslave: ...
>
> Any idea where i screwed up?

The variable that holds the string "GitSlave" when gitolite prints
"mirror push started" is not updated at all in the rest of that
function. So it could not have sent a lowercase version of it to ssh
via the "git push --mirror" command or otherwise.

What OS is this?

regards
sitaram

[Hseu-Ming Chen]

Thanks for the prompt response. 

> Did you then test this, as per bullet 6 in section 3.1?  I.e., run
> 'ssh GitMaster info' on GitSlave, and vice versa.    If so, what was the result?

i thought i did but i just found out that i did that as myself (with my userid) instead of as the user gitolite (or the user git in your docs) - my apology

No, it does not work on either GitMaster or GitSlave as the user gitolite

> What OS is this?

i'm using the latest Gitolite (gitolite3 v3.6.1-6-gdc8b590 on git 1.9.1) on "Ubuntu 14.04.1 LTS".

On GitMaster, when i did
     ssh GitSlave info
i was prompted for password
     gito...@192.168.1.115's password:

After i provided the password, the output of "tail -f /var/log/auth.log" on GitServer is
    Oct  6 23:31:24 hseuming-desktop sshd[5078]: Accepted password for gitolite from 192.168.1.101 port 56383 ssh2
    Oct  6 23:31:24 hseuming-desktop sshd[5078]: pam_unix(sshd:session): session opened for user gitolite by (uid=0)
    Oct  6 23:31:24 hseuming-desktop systemd-logind[726]: New session 7 of user gitolite.
    Oct  6 23:31:24 hseuming-desktop sshd[5116]: Received disconnect from 192.168.1.101: 11: disconnected by user

On GitMaster, i got the GNU info output.

My local computing env is as follows:

                 hostname                   HOSTNAME in rc       IP
--------------------------------------------------------------------------------------------------------
master       ubuntu2                      GitMaster                  192.168.1.101
slave         hseuming-desktop       GitSlave                    192.168.1.115

Moreover, in my keydir on both master and slave:

%  ls keydir/
hseuming.pub  server-GitMaster.pub  server-GitSlave.pub

The host keys for master and server were copied from /etc/ssh/ssh_host_rsa_key.pub and renamed according to the docs

i'm puzzled as to what went astray exactly ...

[Sitaram Chamarty]

On 10/07/2014 09:39 AM, Hseu-Ming Chen wrote:
> Thanks for the prompt response.
>
>> Did you then test this, as per bullet 6 in section 3.1? I.e., run
>> 'ssh GitMaster info' on GitSlave, and vice versa. If so, what was the result?
>
> i thought i did but i just found out that i did that as myself (with my userid) instead of as the user gitolite (or the user git in your docs) - my apology
>
> No, it does not work on either GitMaster or GitSlave as the user gitolite
>
>> What OS is this?
>
> i'm using the latest Gitolite (gitolite3 v3.6.1-6-gdc8b590 on git 1.9.1) on "Ubuntu 14.04.1 LTS".
>
> On GitMaster, when i did
> ssh GitSlave info
> i was prompted for password

Sounds like typical ssh issues.

I have put down **absolutely everything I know about ssh** in [1] and
[2]. There's literally nothing more I can say on that!

[1]: http://gitolite.com/gitolite/sts.html
[2]: http://gitolite.com/gitolite/glssh.html

[Hseu-Ming Chen]

My bad - yes, it's some silly SSH issue.   i got over that hurdle but ran into the following error on GitMaster:

%  git pull
----------
WARNING: previous mirror push to host 'GitSlave' failed, status is:
2014-10-08.01:47:39    7928    FATAL: GitSlave: pushing 'ming' to slave 'GitSlave' not allowed
2014-10-08.01:47:39    7928    fatal: Could not read from remote repository.
2014-10-08.01:47:39    7928   
2014-10-08.01:47:39    7928    Please make sure you have the correct access rights
2014-10-08.01:47:39    7928    and the repository exists.
----------
Already up-to-date.

i deleted the gl-slave* file but the problem persist.   How do i find out what the problem is?

Thanks

[Sitaram Chamarty]

On 10/08/2014 11:22 AM, Hseu-Ming Chen wrote:
> My bad - yes, it's some silly SSH issue. i got over that hurdle but
> ran into the following error on GitMaster:
>
> % git pull
> ----------
> WARNING: previous mirror push to host 'GitSlave' failed, status is:
> 2014-10-08.01:47:39 7928 FATAL: GitSlave: pushing 'ming' to slave 'GitSlave' not allowed
> 2014-10-08.01:47:39 7928 fatal: Could not read from remote repository.
> 2014-10-08.01:47:39 7928
> 2014-10-08.01:47:39 7928 Please make sure you have the correct access rights
> 2014-10-08.01:47:39 7928 and the repository exists.
> ----------
> Already up-to-date.
>
> i deleted the gl-slave* file but the problem persist. How do i find
> out what the problem is?

If you mean the stuff between dashed lines:

- did you change a hostname as part of your fix? If so, then
there may be files called gl-slave-*.status left over on the
master server. You'll have to manually delete them.

- if, as you say, you already deleted them, then I don't know what
to say except "please double check it's the right server, the
right userid, the right directory, etc..."

[Hseu-Ming Chen]

> did you change a hostname as part of your fix?

My bad. Since it was late at night, i went to bed (after shut down my Linux boxes) and resumed the journey next day.  However, thanks to the DHCP at home, the IP of GitSlave changed.  i did change the ~/.ssh/config accordingly but the git [origin|upstream] still contains the old IP.

The IP assignment for tonight is:
  GitMaster   192.168.1.107
  GitSlave    192.168.1.104

Early this evening i wiped everything out on both GitMaster and GitSlave and tried to see if i have any luck just mirroring some simple changes in testing.git.   Unfortunately even that simple mirroring does not appear to work for me:

<tailing of the log, ~gitolite/.gitolite/logs/gitolite-2014-10.log, on GitServer>
...
2014-10-10.02:11:52    9510    ssh    ARGV=hseuming    SOC=git-receive-pack 'testing'    FROM=192.168.1.107
2014-10-10.02:11:52    9510    pre_git    testing    hseuming    W    any    refs/.*
2014-10-10.02:11:52    9510        mirror,pre_git,testing,user=hseuming,sender=,mode=slave
2014-10-10.02:11:52    9510    die    GitSlave: pushing 'testing' to slave 'GitSlave' not allowed

The  ~gitolite/.gitolite/conf/gitolite.conf on GitSlave is as follows:
====================================
repo gitolite-admin
   RW+     =   hseuming
   option mirror.master = GitMaster
   option mirror.slaves = GitSlave

repo testing
   RW+     =   @all
   option mirror.master = GitMaster
   option mirror.slaves = GitSlave

repo ming
   RW+     =   @all
   option mirror.master = GitMaster
   option mirror.slaves = GitSlave
====================================

Which is identical to ~gitolite/.gitolite/conf/gitolite.conf on GitMaster.

i tried 3 times but always ran into the same error.  i did remove the gl-slave-*.status left over on the master server before each try.

Any suggestion?

One more question: i could not get the following to work 2 days ago:
   ssh GitSlave  info    ==  from GitMaster
   ssh GitMaster info    ==  from GitSlave
as described at
   http://gitolite.com/gitolite/mirroring.html

However, after i did some tweaking which is not documented on the mirroring page, both ssh commands started working.   This is what i did: assuming i use alice.pub in

     gitolite setup -pk alice.pub
while setting up Gitolite.   After i copied Alice's PRIVATE key over to ~gitolite/.ssh/id_rsa (with the line "identityfile ~/.ssh/id_rsa" in ~gitolite/.ssh/config) on both GitMaster & GitSlave, then the following commands both work fine:
   ssh GitSlave  info    ==  from GitMaster
   ssh GitMaster info    ==  from GitSlave
  

Is the tweaking right?

Thanks.

[Sitaram Chamarty]

On 10/10/2014 12:28 PM, Hseu-Ming Chen wrote:
>> did you change a hostname as part of your fix?
>
> My bad. Since it was late at night, i went to bed (after shut down my
> Linux boxes) and resumed the journey next day. However, thanks to the
> DHCP at home, the IP of GitSlave changed. i did change the
> ~/.ssh/config accordingly but the git [origin|upstream] still contains
> the old IP.
>
> The IP assignment for tonight is:
> GitMaster 192.168.1.107
> GitSlave 192.168.1.104

I think you're asking for trouble if you try to setup mirroring on
DHCP-ed servers. In theory there're probably ways to solve this but in
practice it sucks big time and probably won't work reliably/consistently.

> Early this evening i wiped everything out on both GitMaster and
> GitSlave and tried to see if i have any luck just mirroring some
> simple changes in testing.git. Unfortunately even that simple
> mirroring does not appear to work for me:
>
> <tailing of the log, ~gitolite/.gitolite/logs/gitolite-2014-10.log, on GitServer>
> ...
> 2014-10-10.02:11:52 9510 ssh ARGV=hseuming SOC=git-receive-pack 'testing' FROM=192.168.1.107
> 2014-10-10.02:11:52 9510 pre_git testing hseuming W any refs/.*
> 2014-10-10.02:11:52 9510 mirror,pre_git,testing,user=hseuming,sender=,mode=slave
> 2014-10-10.02:11:52 9510 die GitSlave: pushing 'testing' to slave 'GitSlave' not allowed

You can't push to the slave (normally). You have to push to the master.

> The ~gitolite/.gitolite/conf/gitolite.conf on GitSlave is as follows:
> ====================================
> repo gitolite-admin
> RW+ = hseuming
> option mirror.master = GitMaster
> option mirror.slaves = GitSlave
>
> repo testing
> RW+ = @all
> option mirror.master = GitMaster
> option mirror.slaves = GitSlave
>
> repo ming
> RW+ = @all
> option mirror.master = GitMaster
> option mirror.slaves = GitSlave
> ====================================
>
> Which is identical to ~gitolite/.gitolite/conf/gitolite.conf on GitMaster.
>
> i tried 3 times but always ran into the same error. i did remove the
> gl-slave-*.status left over on the master server before each try.
>
> Any suggestion?
>
> One more question: i could not get the following to work 2 days ago:
> ssh GitSlave info == from GitMaster
> ssh GitMaster info == from GitSlave
> as described at
> http://gitolite.com/gitolite/mirroring.html

I'm going to assume that "==" and what follows is not part of the
command you typed (hint: use "#" instead; much more obvious that what
follows is meant to be a comment or a note).

> However, after i did some tweaking which is not documented on the
> mirroring page, both ssh commands started working. This is what i
> did: assuming i use alice.pub in
> gitolite setup -pk alice.pub
> while setting up Gitolite. After i copied Alice's PRIVATE key over
> to ~gitolite/.ssh/id_rsa (with the line "identityfile ~/.ssh/id_rsa"
> in ~gitolite/.ssh/config) on both GitMaster & GitSlave, then the
> following commands both work fine:
> ssh GitSlave info == from GitMaster
> ssh GitMaster info == from GitSlave

> Is the tweaking right?

Probably not, but I didn't actually understand. Who is "alice"?
"alice.pub" is a normal USER key, what does that have to do with
mirroring? Mirroring is between hosts, whose keys, in the "keydir"
directory of your gitolite-admin repo, must always be "server-<name of
host>.pub". (Without the angle brackets, need I add?)

And anytime you have to copy a PRIVATE key from somewhere to somewhere
else, you're doing something wrong. And if I ever say that kind of
nonsense in the gitolite documentation, please shoot me.

[Dirk Heinrichs]

Am Freitag 10 Oktober 2014, 14:00:24 schrieb Sitaram Chamarty:

> > My bad. Since it was late at night, i went to bed (after shut down my
> > Linux boxes) and resumed the journey next day. However, thanks to the
> > DHCP at home, the IP of GitSlave changed. i did change the
> > ~/.ssh/config accordingly but the git [origin|upstream] still contains
> > the old IP.
> >
> >
> >
> > The IP assignment for tonight is:
> > GitMaster 192.168.1.107
> > GitSlave 192.168.1.104
>
> I think you're asking for trouble if you try to setup mirroring on
> DHCP-ed servers. In theory there're probably ways to solve this but in
> practice it sucks big time and probably won't work reliably/consistently.

I use DHCP too, but I let it assign that same IP address to each machine every
time, based on their MAC addresses.

Bye...

Dirk
--
Dirk Heinrichs <dirk.he...@altum.de>
Tel: +49 (0)2471 209385 | Mobil: +49 (0)176 34473913
GPG Public Key CB614542 | Jabber: dirk.he...@altum.de
Tox: he...@toxme.se
Sichere Internetkommunikation: http://www.retroshare.org
Privacy Handbuch: https://www.privacy-handbuch.de

[Hseu-Ming Chen]

Hi Sitaram,

> I think you're asking for trouble if you try to setup mirroring on DHCP-ed servers.

Sorry for not making it clearer: i did not intend to run Gitolite mirroring on DHCP.  The reasons i was attempting Gitolite mirroring on DHCP at home are:
* the VMs (on VMWare) that we used at work as our Gitolite servers were hung (likely due to NAS storage timeout)
* since i have no prior experience with Gitolite mirroring, i thought it would be good to first try it out with the "testing.git" repo on my home Ubuntu boxes

> use "#" instead; much more obvious

agreed, my bad.  Will do.

> You can't push to the slave (normally).  You have to push to the master.

i have been pushing ONLY to the master, never the slave.   The log output in my last email is from automatic sync from the master. 

> Probably not, but I didn't actually understand.  Who is "alice"?
> "alice.pub" is a normal USER key, what does that have to do with mirroring? 

From
   http://gitolite.com/gitolite/mirroring.html
there is the following:
   3  setting up mirroring
      3.1 the initial setup and the rc file
          On each server:
          *  Install gitolite normally.

So i need to first install Gitolite.  According to
     http://gitolite.com/gitolite/install.html
wherein alice.pub is mentioned.  In my case, i am the Gitolite admin and my public key is hseuming.pub

> Mirroring is between hosts, whose keys, in the "keydir"
> directory of your gitolite-admin repo, must always be
> "server-<name of host>.pub".  (Without the angle brackets, need I add?)

That's what i've on both GitMaster and GitSlave servers as in:

%  ls ~gitolite/.gitolite/keydir/
hseuming.pub  server-GitMaster.pub  server-GitSlave.pub  userfoo.pub   userbar.pub

> And anytime you have to copy a PRIVATE key from somewhere to somewhere
> else, you're doing something wrong.

Yes, i know that's mighty weird.  i was just so desperate to get over the hurdle below on the mirroring web page:

   ======================================
   3.1 the initial setup and the rc file
   The following MUST work for each pair of servers that must talk to each other:
       # on server mars
       ssh phobos info
       # the response MUST start with "hello, server-mars..."
   ======================================

i was stuck on the above and it was not clear to me how that works due to my SSH-fu (or lack thereof) since i am not an SA (i'm just a developer who is constantly amazed by what Git can do and consider Gitolite a nice FLOSS layer on top of it)

As stated earlier, after i copied my private key (i'm the Gitolite admin) to ~gitolite/.ssh/id_rsa, both
    ssh  GitSlave  info    # from GitMaster
    ssh  GitMaster info    # from GitSlave
work without a hitch.

Ha!!!   i THINK i now know what stupid mistake i made.  The ~gitolite/.ssh/config (identical on both GitMaster & GitSlave) looks like:

   ======================================
   host GitMaster
      user gitolite
      hostname  192.168.1.107
      port 22
      identityfile ~/.ssh/id_rsa

   host GitSlave
      user gitolite
      hostname  192.168.1.104
      port 22
      identityfile ~/.ssh/id_rsa
   ======================================

And the two server keys in  ~gitolite/.gitolite/keydir  are based on:
   server-GitMaster.pub  ===  /etc/ssh/ssh_host_rsa_key.pub @ GitMaster
   server-GitSlave.pub   ===  /etc/ssh/ssh_host_rsa_key.pub @ GitSlave
so the "identityfile" line in ~gitolite/.ssh/config should really be:
   identityfile  /etc/ssh/ssh_host_rsa_key

Am i finally getting on the right track?

> And if I ever say that kind of nonsense in the gitolite documentation,
> please shoot me.

No, you did not and i know you never will, :-)

==================

Hi Dirk,

> I let it assign that same IP address to each machine every time, based on their MAC addresses.

Yes, i should have configured my access point as what you have and that will make my life a tad easier.  
Thanks.

[Sitaram Chamarty]

On 10/11/2014 03:16 AM, Hseu-Ming Chen wrote:

> ======================================
> 3.1 the initial setup and the rc file
> The following MUST work for each pair of servers that must talk to each other:
> # on server mars
> ssh phobos info
> # the response MUST start with "hello, server-mars..."
> ======================================
>
> i was stuck on the above and it was not clear to me how that works due
> to my SSH-fu (or lack thereof) since i am not an SA (i'm just a
> developer who is constantly amazed by what Git can do and consider
> Gitolite a nice FLOSS layer on top of it)

SA or not, you would not have been stuck if you had read the 3rd bullet
properly, which says "Run ssh-keygen if needed and get an ssh key
pair...".

> server-GitMaster.pub === /etc/ssh/ssh_host_rsa_key.pub @ GitMaster
> server-GitSlave.pub === /etc/ssh/ssh_host_rsa_key.pub @ GitSlave

Uggh. Those keys have nothing to do with gitolite. They are used
purely by ssh, when the client is trying to determine if the host it is
talking to is the right one or someone is playing man in the middle.
(Have you ever noticed that the **first** time you ssh to a new server
you get something like this:

The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is 50:01:71:7e:45:20:62:f1:83:4c:12:07:be:6e:38:64.
Are you sure you want to continue connecting (yes/no)?

THAT's what those keys are.

I think you need to spend some time cuddling up to an ssh book. To be
completely honest, in 5 years of supporting gitolite and 1+ year of
supporting the late lamented gitosis before that, I have never seen
anyone pick up those keys :-)

[Hseu-Ming Chen]

Everything works for me now.  Thanks a lot. 

Just one minor thing that i did not follow the instructions on mirroring.html exactly:
    If the slaves are to be exact copies of the master, you need to do the complete configuration
    only on the master; the slaves can have just this because on the first push to the master it will
    update all the slaves anyway:

That does not appear to be true.  To wit, after i pushed the following change back to
   conf/gitolite.conf @ GitMaster

===================================

repo gitolite-admin
   RW+  =  hseuming
   option mirror.master = GitMaster
   option mirror.slaves = GitSlave

repo testing
   RW+     =   @all
   option mirror.master = GitMaster
   option mirror.slaves = GitSlave

===================================

nothing happened on GitSlave (by tailing the log or cloning repos @ GitSlave).  Subsequent commits/pushes back to testing.git @ GitMaster do not get propagated to GitSlave.  i guess GitSlave is not aware of the mirroring options thus not ready to receive any automatic sync from GitMaster.  As soon as i manually modified conf/gitolite.conf @ GitSlave as above (adding those mirroring options), all is well.

> They are used purely by ssh, when the client is trying to determine if the host
> it is talking to is the right one or someone is playing man in the middle.

> ...

>   The authenticity of host 'localhost (::1)' can't be established.

Yes, i know the above reasonably well.  i have several Ubuntu and FreeBSD boxes at home on DHCP so those messages pop up every now and then.  i also took cryptography (very little about elliptic curve though) and number theory before so i've some basic understanding of the PKI stuff.    Moreover i also manipulated and managed a lot of X.509 certs (self-signed or not) with openssl, C, C++, or Python in the past.

> ...  which says "Run ssh-keygen if needed and get an ssh key pair...".

For whatever reason, when i saw "pubkey for server ..." i immediately thought of the host PKI key pairs in /etc/ssh - my bad

For the "if needed" part, i just got sucked into a weird thinking that after a Linux server rebuild, the keys are missing in /etc/ssh so we have to manually generate [rsa|dsa] keys "if needed".

Again, thank you very much for all the input.

[Arun Mishra]

unsubscribe

--
You received this message because you are subscribed to the Google Groups "gitolite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gitolite+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dirk Heinrichs]

Am Samstag 11 Oktober 2014, 22:06:45 schrieb Arun Mishra:

> unsubscribe

1) To unsubscribe, see footer of each mail you receive from the list.
2) Please don't start a new thread by replying to an existing one. This is
called "Thread Hijacking" and considered bad habit.

HTH...

[Dirk Heinrichs]

Am Sonntag 12 Oktober 2014, 10:49:24 schrieb Dirk Heinrichs:

> 1) To unsubscribe, see footer of each mail you receive from the list.

Correction: Only those mails which don't provide an own signature, like your
own mail.

Bye...

[Dirk Heinrichs]

Am Sonntag 12 Oktober 2014, 10:52:01 schrieb Dirk Heinrichs:

> > 1) To unsubscribe, see footer of each mail you receive from the list.
>
> Correction: Only those mails which don't provide an own signature, like
> your own mail.

Or look at the "List-unsubscribe:" header field. This is indeed embedded in
each mail. However, your mail program needs to have an option to show all
header fields.