Proper protocol

[Marius Mårnes Mathiesen]

Anyone subscribed to this list will know that installing Gitorious is not for the faint of heart. There are a lot of moving parts, a lot of dependencies, and getting everything right is difficult. I really want to change this.

One of the things that complicates the setup is that Gitorious is not just a web app. It's also:

- a service running on top of SSH, using key infrastructure for authentication and authorization

- a daemon that serves Git repositories over a dedicated protocol (git://)

- a service that provides Git access over HTTP

Historically, the only protocol that offered write access in Git was SSH. The git protocol was mainly used because it's faster than HTTP and doesn't require authentcation over SSH. These two things are the main reasons why you want to run all these daemons on your server:

- SSH for push

- Git for fast, anonymous pull

Git's HTTP support changed a lot in version 1.6.6, introducing "smart" HTTP. It's a fairly simple concept; a modern Git client will (if supported by the server) open a series of HTTP POST requests to the server, basically running regular git traffic (upload-pack and receive-pack) over this connection. This means HTTP got a lot faster, the performance is comparable to Git over SSH. And since HTTP supports authentication (unlike the Git protocol) and encryption (SSL), this means you can support all Git usage scenarios over a single protocol, and stil be safe. Oh, and as a bonus, people stuck behind a firewall can finally join the party too.

Gitorious' current HTTP support is based on the old (aka. dumb) HTTP method (Scott Chacon has a really good write-up on the difference between "smart" and "dumb" HTTP at http://progit.org/2010/03/04/smart-http.html), which is slow, error-prone, read-only and locks up your Apache/nginx workers serving static files. Git itself has a cgi that provides smart HTTP, but making it work in a scenario where the incoming URL is different from the path on the file system involves hacking Git itself.

Christian and I have made a few experiments on implementing smart HTTP for Gitorious, and this week we've been focusing on one of them: Mutt [1]. Mutt is a cross breed (and also a git):

- It's written in Ruby

- It runs under JRuby

- It uses the awesome JGit library, which is an implementation of Git in Java. JGit is used in EGit, the Git support in Eclipse, and in Gerrit, the code review system used by the Android project

Mutt will provide anonymous/authenticated fetch/pull and authenticated push to your Gitorious server. It will be awesome, and it will be installed by a single command. 

The question is: once we have an easily installed, anonymous/authenticated, pull/push solution for Git traffic: is it time to deprecate the other protocol handlers in Gitorious: 

- The SSH handler

- The Git handler (git-daemon or git-proxy)

Would anybody miss them? 

Discuss.

Cheers,

- Marius

[1]: https://gitorious.org/gitorious/mutt Yes, I know there's already a program called Mutt, the name just fits really well. I promise we won't name the binary mutt, and we won't read your .muttrc.

[Ismail Dhorat]

Hi,

I am pretty new to this list. Just joined yesterday, and only just getting a gitorious installation setup behind the firewall. However, my thoughts are if HTTP can offer everything that SSH can with the same reliability, rather go with a single protocol. It would make the code easier to maintain, as well as much simpler to configure.

Just my 2cents ;)

Regards,

Ismail

--
To post to this group, send email to gito...@googlegroups.com
To unsubscribe from this group, send email to
gitorious+...@googlegroups.com

[Benjamin Podszun]

Hi.

On Fri, Jun 24, 2011 at 8:40 AM, Marius Mårnes Mathiesen
<marius.m...@gmail.com> wrote:
> The question is: once we have an easily installed, anonymous/authenticated,
> pull/push solution for Git traffic: is it time to deprecate the other
> protocol handlers in Gitorious:
> - The SSH handler
> - The Git handler (git-daemon or git-proxy)
> Would anybody miss them?

I would. The deployment of ssh is well-known and understood, and
probably already exists as a way to access machines/services in a lot
of shops. I trust ssh, trust the code quality and security audits.

The line between gitorious (access control, visualizations, wikis.. -
mostly 'extras' for my usecase) and git via ssh is very comforting. If
I break gitorious/if there's a vulnerability in gitorious, I can
easily give an alternate way to access the repositories to my users
(direct accounts on the machine, gitosis or something like that).

So while I understand the reasoning from a maintainer's point of view,
for me ssh is
- required on every installation anyway
- potentially already deployed to allow access to service (think:
migrating from svn+ssh to git)
- a nice way to separate authentication (not authorization) from gitorious

So my vote, if I get any, is:
Please keep ssh support as a maintained and stable option. I don't
care if installation instructions focus on http in the future (so - go
ahead and promote http over ssh, make the http experience a lot easier
to set up or something), but please don't remove the ssh handler or
let it rot.

Ben, running a gitorious installation with a couple dozen users for
1.5 years now

[Marius Mårnes Mathiesen]

On Fri, Jun 24, 2011 at 9:51 AM, Benjamin Podszun <benjamin...@gmail.com> wrote:

I would. The deployment of ssh is well-known and understood, and
probably already exists as a way to access machines/services in a lot
of shops. I trust ssh, trust the code quality and security audits.

Thanks for the feedback! In fact, one of the things I didn't mention about SSH vs HTTPS is actually security. The fact that users connecting over SSH need a system account on the server is something I'd love not to have to worry about. The ability to restrict which command a user is allowed to run helps a lot, and the gitorious script is really restrictive, but still I'd prefer not having one. On the gitorious.org servers, the public facing IP address and port run as an unprivileged user (git), but most local installations probably use OpenSSH's defaults, leaving the SSH port open.

We'll investigate what it will take to use SSL client certificates for authentication, since this will:

- save the users from entering their passwords when pushing/putting the credentials in .git/config or elsewhere

- provide better integrity for credentials: you'd need access to the actual client certificate to gain access to someone else's account, not just a password which may be shared between different sites

 

So my vote, if I get any, is:
Please keep ssh support as a maintained and stable option. I don't
care if installation instructions focus on http in the future (so - go
ahead and promote http over ssh, make the http experience a lot easier
to set up or something), but please don't remove the ssh handler or
let it rot.

Got it. 

 

Ben, running a gitorious installation with a couple dozen users for
1.5 years now

:-)

- Marius

[elf Pavlik]

Hello,

> We'll investigate what it will take to use SSL client certificates for
> authentication, since this will:
> - save the users from entering their passwords when pushing/putting the
> credentials in .git/config or elsewhere
> - provide better integrity for credentials: you'd need access to the actual
> client certificate to gain access to someone else's account, not just a
> password which may be shared between different sites

You may like to take a look at WebID - http://www.w3.org/wiki/WebID
It works by authenticating with client certificates within browser and beyond!

This thread may also come handy:
http://lists.w3.org/Archives/Public/public-xg-webid/2011Jun/0039.html

WebID has very active and helpful community where I believe you can find support if you need...

Cheers,
elf Pavlik

(living strictly moneyless already for over 2 years)
http://wwelves.org/perpetual-tripper
http://moneyless-world.info
http://hackers4peace.net

[August Lilleaas]

I have a concern with the practicality of using client certificates
over https instead of SSH.

You can tell git which key and cert to use for client cert https
connections. This is done via git-config, so it can be set both
globally (--global, stored in ~/.gitconfig) and per project (/path/to/
project/.git/config). Say you're using multiple Gitorious
installations on a daily basis, i.e. gitorious.org, git.myserver.com,
git.mycompany.com, etc. Wouldn't each Gitorious installation require a
separate client cert? If so, you'd have to configure the key and cert
on a per-repo basis. With SSH, you can use the same ssh keypair for
all those sites, not requiring any per-repo config, since there's no
ident info in the public key other than that you was logged in to
gitorious.org when you added it.

On Jun 24, 7:40 am, Marius Mårnes Mathiesen

> and "dumb" HTTP athttp://progit.org/2010/03/04/smart-http.html), which is

> slow, error-prone, read-only and locks up your Apache/nginx workers serving
> static files. Git itself has a cgi that provides smart HTTP, but making it
> work in a scenario where the incoming URL is different from the path on the
> file system involves hacking Git itself.
>
> Christian and I have made a few experiments on implementing smart HTTP for
> Gitorious, and this week we've been focusing on one of them: Mutt [1]. Mutt
> is a cross breed (and also a git):
> - It's written in Ruby
> - It runs under JRuby
> - It uses the awesome JGit library, which is an implementation of Git in
> Java. JGit is used in EGit, the Git support in Eclipse, and in Gerrit, the
> code review system used by the Android project
>
> Mutt will provide anonymous/authenticated fetch/pull and authenticated push
> to your Gitorious server. It will be awesome, and it will be installed by a
> single command.
>
> The question is: once we have an easily installed, anonymous/authenticated,
> pull/push solution for Git traffic: is it time to deprecate the other
> protocol handlers in Gitorious:
> - The SSH handler
> - The Git handler (git-daemon or git-proxy)
>
> Would anybody miss them?
>
> Discuss.
>
> Cheers,
> - Marius
>

> [1]:https://gitorious.org/gitorious/muttYes, I know there's already a

[martin]

Well I'm quite faint hearted and I have installed Gitorious a number of
times (4-5) so I don't really think it is that hard...

Please keep the SSH and Git protocols as they are. People like me expect
it to work the same way at Gitorious as it does according to the Git
documentation.

For my own servers I would turn off the HTTP protocol for push/pull
anyway...

I like to have http only for the Gitorious web interface. I can open
only SSH and HTTP ports in the router and require login to the web
interface. I use this setup to for my private data.

Besides... I kind of trust SSH more than anything else in this world...
I will have a hard time deciding to allow any other push protocol in my
own servers...

Thanks a lot for a great product!

[Benjamin Podszun]

Another open question: Do clients actually support this http based
cloning decently? Think Git Extensions, TortoiseGit et al?

[Rodrigo Rosenfeld Rosas]

Hi Marius,

First, I would like you to congratulate you and Johan for the good work and efforts you both put on this feature.

Then, I would like to say that I would be really really sad if SSH support was deprecated. I think that providing your public key from a user's point of view is really simple and I would to keep that behavior. This would also easy the adoption for most people that are already used to Github.

Also, we don't use git over HTTP at all in our company: just git:// and SSH.

Regarding simplifying installation, I would go with an official automated installation recipe using either Opscode Chef or Puppet. As I already created a Chef's recipe for installing Gitorious in Debian in about 15 minutes if internet speed is good enough, it would be a matter of making it official so that others can include support for non Debian/Ubuntu systems too.

Then, comes my main concern. Until about an year ago, when I tried the git plugin for Netbeans, which used JGit if I remember correctly, I had several problems with corrupted git repository when using submodules. Even if they corrected this, I don't trust the idea of re-writing the git tools for Java. I would feel much more comfortable if they had written a Cgit front-end instead, just like Grit does. And even that happens, Grit is used in Gitorious just to read the file system, not to write to it.

As long as SSH support is kept, I don't mind adding HTTPS push support because I won't enable it anyway in our local installation. But I wouldn't like to use gitorious.org for hosting my projects if I share them with someone else and I can't make sure if he/she will use HTTPS for pushing. Probably I wouldn't provide write access to anyone else just to make sure only SSH will be used for pushing...

I really don't trust JGit and I don't trust current Java developers either after lots of Java code I've been reading lately in largely used Java libraries... It is like almost every good Java developers that existed have now migrated to some better language, many of them still running on JVM with languages like Scala, Closure, Groovy, JRuby, etc...

Well, that is my opinion. If the difficult of installing Gitorious is the reason for dropping SSH support, I would reply that just writing an official Chef's recipe would solve this problem.

Thanks for asking this on this list!

Best regards!

Rodrigo.

[Pedro Kiefer]

Hi Marius,

I don't think it is a good idea to drop/deprecate support for ssh or git-proxy. But I do like the approach of having modularized protocols, maybe ssh and git handler should be refactored to its own modules? So one can choose what it wants to install.

Any way, great job!

Cheers

--

[Christian Johansen]

On Fri, Jun 24, 2011 at 11:51, August Lilleaas <augustl...@gmail.com> wrote:

I have a concern with the practicality of using client certificates
over https instead of SSH.

You can tell git which key and cert to use for client cert https
connections. This is done via git-config, so it can be set both
globally (--global, stored in ~/.gitconfig) and per project (/path/to/
project/.git/config). Say you're using multiple Gitorious
installations on a daily basis, i.e. gitorious.org, git.myserver.com,
git.mycompany.com, etc. Wouldn't each Gitorious installation require a
separate client cert? If so, you'd have to configure the key and cert
on a per-repo basis. With SSH, you can use the same ssh keypair for
all those sites, not requiring any per-repo config, since there's no
ident info in the public key other than that you was logged in to
gitorious.org when you added it.

This is definitely a valid concern, and we will investigate accordingly. If it does have to be configured on a per-repo basis, we could ease this process by providing a small cli to help out. But ideally, it would "just work". Basically - if we cannot make the experience secure AND easy to use, we will not drop easier alternatives.

Christian

[Christian Johansen]

For my own servers I would turn off the HTTP protocol for push/pull
anyway...

Why?
 

I like to have http only for the Gitorious web interface. I can open
only SSH and HTTP ports in the router and require login to the web
interface. I use this setup to for my private data.

You can protect HTTP push the same way. The way it's currently implemented (thanks to JGit's fantastic API), you can basically provide a separate security handler for HTTP(S) push - or even accept push through a different host name (which can be protected by a firewall and so on).
 

Besides... I kind of trust SSH more than anything else in this world...
I will have a hard time deciding to allow any other push protocol in my
own servers...

I'd argue that the HTTPS approach actually has better security. It's very restricted, does not require a privileged/dedicated user to log in to the server, and is built for this one purpose only. If you have specific security concerns, please share.

Christian

[Christian Johansen]

On Fri, Jun 24, 2011 at 14:00, Benjamin Podszun <benjamin...@gmail.com> wrote:

Another open question: Do clients actually support this http based
cloning decently? Think Git Extensions, TortoiseGit et al?

That's a good question, and it's something we'll have to investigate properly. Thanks!

Christian

[Christian Johansen]

First, I would like you to congratulate you and Johan for the good work and efforts you both put on this feature.

Thanks, but my name is Christian. Johan stepped down, remember? :)
 

Then, I would like to say that I would be really really sad if SSH support was deprecated. I think that providing your public key from a user's point of view is really simple and I would to keep that behavior. This would also easy the adoption for most people that are already used to Github.

We definitely want to keep the end-user experience nice and simple, no worries.
 

Also, we don't use git over HTTP at all in our company: just git:// and SSH.

If you could have well-performing, secure and easy-to-use HTTP pull/push, then why wouldn't you use it?
 

Regarding simplifying installation, I would go with an official automated installation recipe using either Opscode Chef or Puppet. As I already created a Chef's recipe for installing Gitorious in Debian in about 15 minutes if internet speed is good enough, it would be a matter of making it official so that others can include support for non Debian/Ubuntu systems too.

The recipes are great when you can spare an entire server. However, for more low-end environments, I think it would be awesome to simply `gem install gitorious-light` to get a bare, but working, system up an running (and one that could later be migrated to the full Gitorious solution).
 

Then, comes my main concern. Until about an year ago, when I tried the git plugin for Netbeans, which used JGit if I remember correctly, I had several problems with corrupted git repository when using submodules. Even if they corrected this, I don't trust the idea of re-writing the git tools for Java. I would feel much more comfortable if they had written a Cgit front-end instead, just like Grit does. And even that happens, Grit is used in Gitorious just to read the file system, not to write to it.

JGit is under heavy development, and I don't have any trouble trusting it. Also, we're only using it for a very limited set of functionality, which is fairly easy to test and verify that works correctly. Grit also does stuff on its own (i.e. in Ruby), and is not always right on the mark either.
 

As long as SSH support is kept, I don't mind adding HTTPS push support because I won't enable it anyway in our local installation. But I wouldn't like to use gitorious.org for hosting my projects if I share them with someone else and I can't make sure if he/she will use HTTPS for pushing. Probably I wouldn't provide write access to anyone else just to make sure only SSH will be used for pushing...

As I've already said many times, we will not deprecate a working solution in favor of a less capable one, so this shouldn't be a concern. Apart from that, I don't really get the sceptic attitude against HTTPS push? Why is it important that only SSH is used for pushing? (Given that the usability issue can be solved).
 

I really don't trust JGit and I don't trust current Java developers either after lots of Java code I've been reading lately in largely used Java libraries... It is like almost every good Java developers that existed have now migrated to some better language, many of them still running on JVM with languages like Scala, Closure, Groovy, JRuby, etc...

Take a peak at the JGit code. The parts I've looked at are surprisingly good ;)
 

Well, that is my opinion. If the difficult of installing Gitorious is the reason for dropping SSH support, I would reply that just writing an official Chef's recipe would solve this problem.

Thanks for asking this on this list!

And thanks for responding - your feedback is greatly appreciated.

Christian

[Christian Johansen]

I don't think it is a good idea to drop/deprecate support for ssh or git-proxy. But I do like the approach of having modularized protocols, maybe ssh and git handler should be refactored to its own modules? So one can choose what it wants to install.

We will not drop the code completely - but we may deprecate certain solutions from gitorious.org if we deem it right to do so.

Having said that, I agree with you that modularization is the way ahead. In fact, I think modularization (e.g. splitting stuff out as gems, allowing more pieces of Gitorious to be installed/run in isolation) is the best way of helping make Gitorious more manageable to cofigure and deploy - both for us and for all of you running it locally.
 

Any way, great job!

Thanks!

Cheers,
Christian

[Benjamin Podszun]

Once more:

On Fri, Jun 24, 2011 at 11:13 PM, Christian Johansen
<chri...@cjohansen.no> wrote:
> As I've already said many times, we will not deprecate a working solution in
> favor of a less capable one, so this shouldn't be a concern. Apart from
> that, I don't really get the sceptic attitude against HTTPS push? Why is it
> important that only SSH is used for pushing? (Given that the usability issue
> can be solved).

I start to fear that the audience (me included) is chosen 'wrong' and
heavily biased. You're asking people that either are interested in the
development of gitorious already or have 1-N private instances up and
running.
You want to make a change to reach the people that are _not_
subscribed here, I guess. The people on this list probably care less
for 'alternative things for something that already works now, albeit
in a different way'.

So - although I still think that ssh should stay and that GUI client
support is a valid point: I cannot ignore the thought that I might be
just a lazy bum that wants to keep it as it is, for the sake of it.

[Christian Johansen]

So - although I still think that ssh should stay and that GUI client
support is a valid point: I cannot ignore the thought that I might be
just a lazy bum that wants to keep it as it is, for the sake of it.

:) No worries. You are definitely right in that we want to reach a wider audience, but we also want to collaborate with the existing community. It's not like we're looking for "a new audience", we're just trying to improve Gitorious, and sometimes improvement means change and even stripping things down.

Christian

[Rodrigo Rosenfeld Rosas]

Em 24-06-2011 17:13, Christian Johansen escreveu:

First, I would like you to congratulate you and Johan for the good work and efforts you both put on this feature.

Thanks, but my name is Christian. Johan stepped down, remember? :)

Right, sorry, Christian. It is just the new age's effect :)

Then, I would like to say that I would be really really sad if SSH support was deprecated. I think that providing your public key from a user's point of view is really simple and I would to keep that behavior. This would also easy the adoption for most people that are already used to Github.

We definitely want to keep the end-user experience nice and simple, no worries.
 

Also, we don't use git over HTTP at all in our company: just git:// and SSH.

If you could have well-performing, secure and easy-to-use HTTP pull/push, then why wouldn't you use it?

Because I want all my repository submodules URL to use git:// protocol instead of HTTP and HTTP is enabled by default and I don't want to have to worry about that explaining this to my coworkers. We don't have firewall issues related to git:// protocol in our company. I find the git protocol to be more lightweight than HTTP and usually supported out-of-the-box by git tools and I want they to think that Gitorious is just a web interface and that we could do everything without Gitorious using other tools if we wanted...

It is important for me that they understand the concepts behind usual Git workflow...

Regarding simplifying installation, I would go with an official automated installation recipe using either Opscode Chef or Puppet. As I already created a Chef's recipe for installing Gitorious in Debian in about 15 minutes if internet speed is good enough, it would be a matter of making it official so that others can include support for non Debian/Ubuntu systems too.

The recipes are great when you can spare an entire server. However, for more low-end environments, I think it would be awesome to simply `gem install gitorious-light` to get a bare, but working, system up an running (and one that could later be migrated to the full Gitorious solution).

I'm not sure I understood. Recipes can be fully configured to adapt to each user's need and we could provide simple defaults the same way you would do for gitorious-light.

 

Then, comes my main concern. Until about an year ago, when I tried the git plugin for Netbeans, which used JGit if I remember correctly, I had several problems with corrupted git repository when using submodules. Even if they corrected this, I don't trust the idea of re-writing the git tools for Java. I would feel much more comfortable if they had written a Cgit front-end instead, just like Grit does. And even that happens, Grit is used in Gitorious just to read the file system, not to write to it.

JGit is under heavy development, and I don't have any trouble trusting it. Also, we're only using it for a very limited set of functionality, which is fairly easy to test and verify that works correctly. Grit also does stuff on its own (i.e. in Ruby), and is not always right on the mark either.

This is my concern: I've lost git commits due to JGit destroying the git file-system. I definitely don't want to enable any write support of JGit for my repositories... I think Gitorious currently don't use any write support from Grit, right?

As long as SSH support is kept, I don't mind adding HTTPS push support because I won't enable it anyway in our local installation. But I wouldn't like to use gitorious.org for hosting my projects if I share them with someone else and I can't make sure if he/she will use HTTPS for pushing. Probably I wouldn't provide write access to anyone else just to make sure only SSH will be used for pushing...

As I've already said many times,

All of them about now ;)

we will not deprecate a working solution in favor of a less capable one, so this shouldn't be a concern. Apart from that, I don't really get the sceptic attitude against HTTPS push? Why is it important that only SSH is used for pushing? (Given that the usability issue can be solved).

I don't mind in supporting HTTPS push. I just don't want any JGit write support involved in this process... I'm all for it otherwise. It just doesn't mean I'll use it myself, since I'm fine with SSH for our local installation.

I really don't trust JGit and I don't trust current Java developers either after lots of Java code I've been reading lately in largely used Java libraries... It is like almost every good Java developers that existed have now migrated to some better language, many of them still running on JVM with languages like Scala, Closure, Groovy, JRuby, etc...

Take a peak at the JGit code. The parts I've looked at are surprisingly good ;)

I didn't take a look at it, and as I said, I'm not even sure that NBGit used JGit, but I know for sure that NBGit corrupted some repositories with submodules. I just don't like the idea of reverse engineering git and would prefer a front-end approach instead... 

Well, that is my opinion. If the difficult of installing Gitorious is the reason for dropping SSH support, I would reply that just writing an official Chef's recipe would solve this problem.

Thanks for asking this on this list!

And thanks for responding - your feedback is greatly appreciated.

It is from my best interest too ;)

Changing the subject, just as a brief update, I'm almost finished with OpenID support with Devise. It is already working but I need to fix some other existent problems in Gitorious regarding flash messages that are currently not working correctly in the second generation layout and the authentication view when errors happen. Also I need to make the test pass, even if it means replacing it with an integration test...

But probably I'll only be able to continue on next weekend since I'm leaving to a party now and will be out of town as soon as my wife and me wake up tomorrow, returning on Sunday night probably...

Best regards!

[Christian Johansen]

Thanks, but my name is Christian. Johan stepped down, remember? :)

Right, sorry, Christian. It is just the new age's effect :)

;)
 

If you could have well-performing, secure and easy-to-use HTTP pull/push, then why wouldn't you use it?

Because I want all my repository submodules URL to use git:// protocol instead of HTTP and HTTP is enabled by default and I don't want to have to worry about that explaining this to my coworkers. We don't have firewall issues related to git:// protocol in our company. I find the git protocol to be more lightweight than HTTP and usually supported out-of-the-box by git tools and I want they to think that Gitorious is just a web interface and that we could do everything without Gitorious using other tools if we wanted...

It is important for me that they understand the concepts behind usual Git workflow...

For us too, we're not trying to divert anyone from that. Keep in mind that smart HTTP is part of Git. The reason we cannot use it directly is the path translation from URLs to hashed paths. Johan once tried to suggest path translation (or at least hooks for it) be built in to Git itself, but the request was declined.
 

This is my concern: I've lost git commits due to JGit destroying the git file-system. I definitely don't want to enable any write support of JGit for my repositories... I think Gitorious currently don't use any write support from Grit, right?

Understandable. We will test this very thoroughly before recommending its wide-spread use.
 

As long as SSH support is kept, I don't mind adding HTTPS push support because I won't enable it anyway in our local installation. But I wouldn't like to use gitorious.org for hosting my projects if I share them with someone else and I can't make sure if he/she will use HTTPS for pushing. Probably I wouldn't provide write access to anyone else just to make sure only SSH will be used for pushing...

Don't worry, we won't drop the code :)

Changing the subject, just as a brief update, I'm almost finished with OpenID support with Devise. It is already working but I need to fix some other existent problems in Gitorious regarding flash messages that are currently not working correctly in the second generation layout and the authentication view when errors happen. Also I need to make the test pass, even if it means replacing it with an integration test...

But probably I'll only be able to continue on next weekend since I'm leaving to a party now and will be out of town as soon as my wife and me wake up tomorrow, returning on Sunday night probably...

Probably better to keep it on the other thread ;) But, that is fantastic news, cannot wait to give it a spin. If an integration test is what is required I have no problems with that. Have a great weekend!

Christian

[martin]

On Fri, 2011-06-24 at 22:05 +0200, Christian Johansen wrote:
> For my own servers I would turn off the HTTP protocol for
> push/pull
> anyway...
>
> Why?

Explanation is below here:

>
> I like to have http only for the Gitorious web interface. I
> can open
> only SSH and HTTP ports in the router and require login to the
> web
> interface. I use this setup to for my private data.
>
> You can protect HTTP push the same way. The way it's currently
> implemented (thanks to JGit's fantastic API), you can basically
> provide a separate security handler for HTTP(S) push - or even accept
> push through a different host name (which can be protected by a
> firewall and so on).

I want to protect both push and pull... easier to just turn it off...
For data I need to publish I don't use my own servers...

I strongly believe that most, if not all, software project would benefit
from being published.

But most of the data I have in my private/corp git repositories are not
even software projects. So my use may not be representative for what the
Gitorious project aims for. However I think that lots of private
Gitorious servers contain data that the owners think may be worth
protecting.

>
>
> Besides... I kind of trust SSH more than anything else in this
> world...
> I will have a hard time deciding to allow any other push
> protocol in my
> own servers...
>
> I'd argue that the HTTPS approach actually has better security. It's
> very restricted, does not require a privileged/dedicated user to log
> in to the server, and is built for this one purpose only. If you have
> specific security concerns, please share.

The https solution is not mature in the same way as the ssh solution.
SSH has protected Unix/Linux boxes for ages.
Software built for one purpose only is not exposed to the same range of
threats and is therefore maturing slower... ssh has been THE target of
hacking attempts since the protocol was first specified back in the
90ies.
I don't understand why you are concerned about the dedicated git user
account... just lock it down properly. You have exactly the same
situation on every ssh server on the planet.

Please keep the SSH and Git protocols at least until it is dropped by
the git project (which hopefully will never happen).
And I also saw concerns about JGit and writing to the repos. I think all
writing to the repos should be done using code from the git project.

Martin

>
> Christian

[jarrod....@gmail.com]

On , Marius Mårnes Mathiesen <marius.m...@gmail.com> wrote:
> Anyone subscribed to this list will know that installing Gitorious is not for the faint of heart. There are a lot of moving parts, a lot of dependencies, and getting everything right is difficult. I really want to change this.
>

> The question is: once we have an easily installed, anonymous/authenticated, pull/push solution for Git traffic: is it time to deprecate the other protocol handlers in Gitorious: 
>
> - The SSH handler
> - The Git handler (git-daemon or git-proxy)
>
>
> Would anybody miss them? 

Yes, lots of people disable the HTTP support completely and only use SSH for writes and Git of read only access. This is what we do for our installation.
This is very important for work flows where people should be able to check out read only copies but not be able to write.

All my Java projects, and there and dozens and dozens of Java repos in my Gitorious install, use Maven 3 and its support for the maven-release-plugin and scm-plugin which it uses to update Git automatically when preforming releases.
the SCM plugin likes to have a "developer" url and a "guest" url so it can publish documentation with links that people can checkout the code but not be able to write to it.

We only give write access to people that are responding to merge requests and use the git protocol for adding remotes to private clones to pull from the mainline repository.

This separation of concerns is very important to workflows like this.

If you want to enhance the HTTP, then by all means do that, but don't remove any functionality that is already there and working great.

PS: As to the difficulty in installing Gitorious. It took me most of a day and a half to get it up and running on a CentOS 5.5 server. Most of the time was me dealing with all the very specific versions of Ruby stuff that wasn't available via any one YUM repository. I do just about every language but Ruby and it ecosystem, there were lots of things I just had to guess at until I got it right. In particular setting up Passenger details under Apache 2.2.x entailed lots and lots of Googling and emails to the group and IRC chats, because most of the Passenger documentation refers to config files and things with the assumption you know where they actually are. Gitorious wasn't stable until I got this nailed down.

There is already what looks like a nice Chef recipe for Ubuntu, we don't do Ubuntu because we have standardized on RHEL5/CentOS5 because that is what all our clients use, an RPM package and a custom YUM repository for RHEL5/CentOS5.x that at least consolidated all the very specific Ruby/Rails/Passenger details into a single RPM and have dependencies on all the other stuff like Apache, MySQL, etc. would go a long way to a more painless adoption.

[Rodrigo Rosenfeld Rosas]

Hi Jarrod,

I would be happy to merge any changes to my Gitorious Chef recipe for it
to work on CentOS/Redhat too. I can download a CentOS image for testing
it and I can help you with Chef specific bits as long as I understand it.

It just happens that there is a long time since I last used Redhat (last
version was 9 when Fedora was born and I changed to Suse at that time if
I remember correctly). I use Debian for several years now and I never
worked with something like yum in Redhat, Suse, Mandrake, Conectiva or
Mandriva. The only tool available at that time (that I knew) was "rpm"
which is "dpkg" equivalent.

If you want to help supporting CentOS in the Chef recipe, get in touch
with me and we can help each other.

Cheers, Rodrigo.

Em 26-06-2011 14:42, jarrod....@gmail.com escreveu:

> a long way to a more painless adoption. --

[Marius Mårnes Mathiesen]

On Sun, Jun 26, 2011 at 10:16 AM, martin <mar...@siamect.com> wrote:

The https solution is not mature in the same way as the ssh solution.

SSH has protected Unix/Linux boxes for ages.

One might argue that SSH has exposed Unix/Linux boxes to attacks, not protected them, for ages; just have a quick look at the security logs on your server, and you'll discover that SSH is the preferred choice of anyone targeting your server. SSH's will by default offer a connecting user a shell, the gitorious script bypasses this by restricting which actions a user can do on the server.

 

I don't understand why you are concerned about the dedicated git user
account... just lock it down properly. You have exactly the same
situation on every ssh server on the planet.

As I mentioned above, I suspect most users running their own Gitorious servers have sshd running as the root user, since otherwise they'd need a separate IP address/port in order to do maintenance on their servers. I don't think it's reasonable to assume people looking for a way to collaborate on code have experience in locking down a SSH daemon on their server. 

 

And I also saw concerns about JGit and writing to the repos. I think all
writing to the repos should be done using code from the git project.

I really don't get this. JGit had a bug, and that bug was resolved. JGit is used in Eclipse by thousands of developers, and they trust it to do its job. JGit is also used in Gerrit, which means the Android repositories would be at stake if JGit didn't work. I don't think they'd use that if there was a real risk in doing so. Furthermore, have you looked at the vulnerabilities in Git over the last few years? You'll find plenty of buffer overflow vulnerabilities, command injection tricks etc. that don't exist in JGit. 

Would you be as skeptical to for instance the libgit2 project (http://libgit2.github.com/)? 

Cheers,

- Marius

[Marius Mårnes Mathiesen]

On Sun, Jun 26, 2011 at 7:42 PM, <jarrod....@gmail.com> wrote:

Yes, lots of people disable the HTTP support completely and only use SSH for writes and Git of read only access. This is what we do for our installation.

This is very important for work flows where people should be able to check out read only copies but not be able to write.

I'd argue that requiring different URLs for reading and writing is more cumbersome than using the same one and then using authentication/authorization to handle requests to write to the repository.

 

There is already what looks like a nice Chef recipe for Ubuntu, we don't do Ubuntu because we have standardized on RHEL5/CentOS5 because that is what all our clients use, an RPM package and a custom YUM repository for RHEL5/CentOS5.x that at least consolidated all the very specific Ruby/Rails/Passenger details into a single RPM and have dependencies on all the other stuff like Apache, MySQL, etc. would go a long way to a more painless adoption.

It would be wonderful if someone would step up and take responsibility for .rpm/.deb packages of Gitorious, nothing would be better. Last time I checked there weren't packages for some of the dependencies (activemq, sphinx), they'd need to be maintained as well. As for Passenger, it seems Fedora/RedHat won't include that for some time: https://bugzilla.redhat.com/show_bug.cgi?id=470696

Cheers,

- Marius

[martin]

On Mon, 2011-06-27 at 10:17 +0200, Marius Mårnes Mathiesen wrote:
> On Sun, Jun 26, 2011 at 10:16 AM, martin <mar...@siamect.com> wrote:
> The https solution is not mature in the same way as the ssh
> solution.
> SSH has protected Unix/Linux boxes for ages.
>
>
> One might argue that SSH has exposed Unix/Linux boxes to attacks, not
> protected them, for ages; just have a quick look at the security logs
> on your server, and you'll discover that SSH is the preferred choice
> of anyone targeting your server. SSH's will by default offer a
> connecting user a shell, the gitorious script bypasses this by
> restricting which actions a user can do on the server.

I have in average about 200 logged intrusion attempts on the ssh port
per day. I don't allow password authentication... I don't believe that
being the primary target for such attempts make ssh any weaker...

>
> I don't understand why you are concerned about the dedicated
> git user
> account... just lock it down properly. You have exactly the
> same
> situation on every ssh server on the planet.
>
>
> As I mentioned above, I suspect most users running their own Gitorious
> servers have sshd running as the root user, since otherwise they'd
> need a separate IP address/port in order to do maintenance on their
> servers. I don't think it's reasonable to assume people looking for a
> way to collaborate on code have experience in locking down a SSH
> daemon on their server.

If people are knowledgeable enough to follow the instructions to install
Gitorious, then they should have no problem following a lock-down
instruction for ssh!

>
>
> And I also saw concerns about JGit and writing to the repos. I
> think all
> writing to the repos should be done using code from the git
> project.
>
>
> I really don't get this. JGit had a bug, and that bug was resolved.
> JGit is used in Eclipse by thousands of developers, and they trust it
> to do its job. JGit is also used in Gerrit, which means the Android
> repositories would be at stake if JGit didn't work. I don't think
> they'd use that if there was a real risk in doing so. Furthermore,
> have you looked at the vulnerabilities in Git over the last few years?
> You'll find plenty of buffer overflow vulnerabilities, command
> injection tricks etc. that don't exist in JGit.

I don't by default trust people, software nor politicians. I trust what
has been proven to work for others and yes I follow the Git developers
discussions. I do see a very serious attitude towards problems,
especially if it is about security or keeping the data intact.

I have used Eclipse for a while and I'm not impressed. I also read how
their project management is trying to use hooks to verify that the
committers is on the list of trusted people. This shows clearly that
they have yet to understand the concept of distributed work flow. So
Eclipse using JGit does not making JGit anymore trustworthy, quite the
opposite.
Anroid... well Google just skipped the plans for supporting Git (for
now) and went for Mercurial instead... Not that I care but the
comparisons they published speaks... Don't take me wrong, I like Hg too
but if I have to choose... it's Git.
So really there are others I trust more. The Gitorious team including
you for example. Even if you right now are exercising the thought of
skipping ssh, there is absolutely no doubt in my mind that you will come
to the conclusion that keeping ssh/git is necessary.

You don't know if JGit have buffer overflow vulnerabilities or command
injection tricks and whatever things are referred to as "etc"... no one
does... the information is simply not there.

>
>
> Would you be as skeptical to for instance the libgit2 project
> (http://libgit2.github.com/)?

Yes, but I'm skeptical to all projects. Over time, some gain my trust
and respect. Libgit2 has an odd extension to GPLv2 that should be read
carefully...
I strongly believe that the best programmers are searching to contribute
directly to the git project as long as the git project is aiming in the
right direction.
There may be strong programmers in libgit2 as well as JGit and there may
be competent management too. I don't know. Time has to show...
As far as I'm concerned, right now, I push via ssh, pull via git and I
think we all should.go ahead with the https push but let the users
decide if they trust it. Let the users enable it per project or per
repo. Maybe in a few years, I will use it...

Martin

>
>
> Cheers,
> - Marius

[Benjamin Podszun]

Hi there

On Mon, Jun 27, 2011 at 11:17 AM, Marius Mårnes Mathiesen
<marius.m...@gmail.com> wrote:
> On Sun, Jun 26, 2011 at 10:16 AM, martin <mar...@siamect.com> wrote:
>> I don't understand why you are concerned about the dedicated git user
>> account... just lock it down properly. You have exactly the same
>> situation on every ssh server on the planet.
>
> As I mentioned above, I suspect most users running their own Gitorious
> servers have sshd running as the root user, since otherwise they'd need a
> separate IP address/port in order to do maintenance on their servers. I
> don't think it's reasonable to assume people looking for a way to
> collaborate on code have experience in locking down a SSH daemon on their
> server.

Since this came up several times now: Can you explain that part? I
wonder if you'd consider my environment at risk.

Looking at man sshd_config I think I'm fine:

UsePrivilegeSeparation
Specifies whether sshd(8) separates privileges by
creating an unprivileged child process to deal
with incoming network traffic. After successful
authentication, another process will be created that
has the privilege of the authenticated user. The goal of
privilege separation is to prevent privilege
escalation by containing any corruption within the
unprivileged processes. The default is “yes”.

But maybe I'm not understanding the concern. So I am running ssh as
root (like most users, as you said), but it seems to be the default to
enable privilege separation, which kind of ends up doing what you do
manually: It runs the network facing service unprivileged.

Regards,
Ben

[Rodrigo Rosenfeld Rosas]

Em 27-06-2011 08:33, Benjamin Podszun escreveu:
> Hi there
>

> On Mon, Jun 27, 2011 at 11:17 AM, Marius M�rnes Mathiesen

> <marius.m...@gmail.com> wrote:
>> On Sun, Jun 26, 2011 at 10:16 AM, martin<mar...@siamect.com> wrote:
>>> I don't understand why you are concerned about the dedicated git user
>>> account... just lock it down properly. You have exactly the same
>>> situation on every ssh server on the planet.
>> As I mentioned above, I suspect most users running their own Gitorious
>> servers have sshd running as the root user, since otherwise they'd need a
>> separate IP address/port in order to do maintenance on their servers. I
>> don't think it's reasonable to assume people looking for a way to
>> collaborate on code have experience in locking down a SSH daemon on their
>> server.
> Since this came up several times now: Can you explain that part? I

> wonder if you'd consider my environment at risk...

It is missing a bit of context here...

What Marius is trying to say is that *if you're exposing your Gitorious
installation to the web* you *must* make sure you protect it adequately.

Most Gitorious installations are exposed to intranet only or using VPN,
so that is ok to have the default SSH configuration set up.

If you need to expose Gitorious, than it makes sense to disable SSH and
go with HTTPS if you don't want to expose SSH to the Internet.
Otherwise, you should probably disable password authentication for all
users. It would also be a great idea to disallow SSH login with the root
account and create another one for logging in instead. Not that SSH is
unreliable, but these are best practices if your security concerns are high.

I just think that is not the usual case. Specially because Gitorious
still doesn't support private repositories adequately, so who would like
to expose their private source code to the web?

[Rodrigo Rosenfeld Rosas]

Hi Marius, although there is no ActiveMQ package for Debian, there is the stompserver package that could be used instead. Sphinx exists like sphinxsearch package in Debian (I believe in Ubuntu too). There is also 'libapache2-mod-passenger'.

Not that it would be simple to write such gitorious.deb package, but I don't think the main reasons are the lack of packages, in Debian at least... :)

[Benjamin Podszun]

Hi

On Mon, Jun 27, 2011 at 4:18 PM, Rodrigo Rosenfeld Rosas
<rr.r...@gmail.com> wrote:
> Em 27-06-2011 08:33, Benjamin Podszun escreveu:
>>
>> Hi there
>>

>> On Mon, Jun 27, 2011 at 11:17 AM, Marius Mårnes Mathiesen

>> <marius.m...@gmail.com>  wrote:
>>>
>>> On Sun, Jun 26, 2011 at 10:16 AM, martin<mar...@siamect.com>  wrote:
>>>>
>>>> I don't understand why you are concerned about the dedicated git user
>>>> account... just lock it down properly. You have exactly the same
>>>> situation on every ssh server on the planet.
>>>
>>> As I mentioned above, I suspect most users running their own Gitorious
>>> servers have sshd running as the root user, since otherwise they'd need a
>>> separate IP address/port in order to do maintenance on their servers. I
>>> don't think it's reasonable to assume people looking for a way to
>>> collaborate on code have experience in locking down a SSH daemon on their
>>> server.
>>
>> Since this came up several times now: Can you explain that part? I
>> wonder if you'd consider my environment at risk...

> What Marius is trying to say is that *if you're exposing your Gitorious

> installation to the web* you *must* make sure you protect it adequately.

> If you need to expose Gitorious, than it makes sense to disable SSH and go

> with HTTPS if you don't want to expose SSH to the Internet. Otherwise, you
> should probably disable password authentication for all users. It would also
> be a great idea to disallow SSH login with the root account and create
> another one for logging in instead. Not that SSH is unreliable, but these
> are best practices if your security concerns are high.

Could be. I'm not sure. We're probably all non-native english
speakers. I read 'have sshd running as the root user' as 'running the
process sshd with effective uid 0', which is the default, with some
twists (see the privilege separation thing). I don't know any problems
with this and would love to hear any issues that gitorious.org had
with this in the past. Maybe I adapt my own setup afterwards.

If it the point was purely about securing the box/service itself,
disallowing root logins etc, as you understood it, then my point is
moot and everything's good/I agree totally.

Regards,
Ben

[Rodrigo Rosenfeld Rosas]

Em 27-06-2011 10:25, Benjamin Podszun escreveu:
> Hi
>
> On Mon, Jun 27, 2011 at 4:18 PM, Rodrigo Rosenfeld Rosas
> <rr.r...@gmail.com> wrote:
>> Em 27-06-2011 08:33, Benjamin Podszun escreveu:
>>> Hi there
>>>

>>> On Mon, Jun 27, 2011 at 11:17 AM, Marius M�rnes Mathiesen

I think it is not currently possible to listen on port 22 with effective
uid other than 0 in Unix-like systems, but I may be wrong since I'm not
really a security specialist.

If we set it up to run in another port, than instead of
'g...@some.server/some/repo' we would have 'g...@some.server:2222/some/repo'.

Maybe someone here with better knowledge on security could state
otherwise how to listen on port 22 without running the service with an
unprivileged account.

[Rodrigo Rosenfeld Rosas]

Actually, I hack I usually do when hosting some web application on
Tomcat is usually to run it as the tomcat user on port 8080 and add an
IPTables rule for directing port 80 to 8080... This could be set up for
sshd in a Gitorious server.

[Rodrigo Rosenfeld Rosas]

Em 27-06-2011 10:47, Rodrigo Rosenfeld Rosas escreveu:
> ...

>> I think it is not currently possible to listen on port 22 with
>> effective uid other than 0 in Unix-like systems, but I may be wrong
>> since I'm not really a security specialist.
>>
>> If we set it up to run in another port, than instead of
>> 'g...@some.server/some/repo' we would have
>> 'g...@some.server:2222/some/repo'.
>>
>> Maybe someone here with better knowledge on security could state
>> otherwise how to listen on port 22 without running the service with
>> an unprivileged account.
>
>
> Actually, I hack I usually do when hosting some web application on
> Tomcat is usually to run it as the tomcat user on port 8080 and add an
> IPTables rule for directing port 80 to 8080... This could be set up
> for sshd in a Gitorious server.

According to this article, it is possible to allow an unprivileged user
to bind to privileged ports by using authbind:

http://www.debian-administration.org/articles/386

[Gabriel Mazetto]

HTTPS has one requirement that can cost more to provide a
installation: It requires a unique ip address dedicated to that
service and a valid certificate.

[Marius Mårnes Mathiesen]

On Tue, Jul 12, 2011 at 1:09 AM, Gabriel Mazetto <bro...@gmail.com> wrote:

HTTPS has one requirement that can cost more to provide a
installation: It requires a unique ip address dedicated to that
service and a valid certificate.

While you could get by with a self-signed certificate and configuring your web server to serve different apps based on URL/host names, a dedicated IP address and real SSL cert would be recommended. I'm not sure if this will be a problem for most people running Gitorious themselves though?

- Marius