[gitolite] about gitolite log files

[Christian MICHON]

Hi list,

I recently compared some features of gerrit log files and gitolite log
files. I've a need to perform metrics on my small gitolite server.

One feature I believe could be added to the logs would be to indicate
how long took a git-receive-pack or a git-upload-pack to complete.

Is this easily feasible ?

--
Christian
--
http://detaolb.sourceforge.net/, a linux distribution for Qemu with Git inside !

[Sitaram Chamarty]

[hmm; seems I forgot to reply to the list in the previous message...]

on 20/05/10 13:51 Christian MICHON wrote:
> Hi Sitaram,

>
> On Thu, May 20, 2010 at 10:02 AM, Sitaram Chamarty
<sit...@atc.tcs.com> wrote:
>>> Is this easily feasible ?
>>

>> In theory, replacing the exec at the end of gl-auth-command with a
>> "system", and putting in a high-res timer should do it.
>
> could you please show me the exact command you've in mind ?
> especialler the timer part.

try the attached patch file; against the latest "pu" but should work
against any recent version, really.

The sample output of that patch, when pushing a copy of the linux repo
to the server, is:

2010-05-20.14:13:58 git-receive-pack 'linux' u1 started
2010-05-20.14:13:58 W 00000000000000 2eaa9cfdf33b8d linux
refs/heads/master u1 refs/.*
2010-05-20.14:13:58 git-receive-pack 'linux' u1 completed; took
156.009563 seconds

>>
>> I'm curious how this information helps...
>>
>
> Figure out the load of the gitolite server, especially parallel
> uploads/downloads :-)

I wonder how many people even look at the existing log file; I know I
only created it for audit purposes, not performance. Anyway let me know
if the patch works for you.

Sitaram

[Christian MICHON]

On Thu, May 20, 2010 at 10:55 AM, Sitaram Chamarty <sit...@atc.tcs.com> wrote:
> try the attached patch file; against the latest "pu" but should work
> against any recent version, really.
>
> The sample output of that patch, when pushing a copy of the linux repo
> to the server, is:
>
> 2010-05-20.14:13:58    git-receive-pack 'linux'    u1 started
> 2010-05-20.14:13:58  W    00000000000000    2eaa9cfdf33b8d    linux
> refs/heads/master    u1    refs/.*
> 2010-05-20.14:13:58    git-receive-pack 'linux'    u1 completed; took
> 156.009563 seconds
>

working fine, thanks a lot

>>>
>>> I'm curious how this information helps...
>>>
>>
>> Figure out the load of the gitolite server, especially parallel
>> uploads/downloads :-)
>
> I wonder how many people even look at the existing log file; I know I
> only created it for audit purposes, not performance.  Anyway let me know
> if the patch works for you.
>

I do care :-)

[Christian MICHON]

Extra request (if I may?). Would it be possible too to measure data
size transferred during clone/fetch/push ?

Christian

[Sitaram Chamarty]

on 20/05/10 14:53 Christian MICHON wrote:

> working fine, thanks a lot
>
>>>>
>>>> I'm curious how this information helps...
>>>>
>>>
>>> Figure out the load of the gitolite server, especially parallel
>>> uploads/downloads :-)
>>
>> I wonder how many people even look at the existing log file; I know I
>> only created it for audit purposes, not performance. �Anyway let me know
>> if the patch works for you.
>>
>
> I do care :-)

The original purpose of gitolite is security, and hence auditability of
access is relevant. Performance is not really central to gitolite's goals.

I also feel you should wrap "git-shell" on the server to acquire and log
that info -- that is, it should not be a "gitolite" thing but a "git"
thing, if you see what I mean.

Due to these reasons, I'm not too keen on putting this into mainline.
Would you mind maintaining it as a patch on your side unless we hear
from a few others that they also want it?

My fear is that now you have elapsed time. Soon the perf guys will want
CPU and IO time, and so on and on... sorry but I will not be getting
into all that :) I'd rather nip it in the bud!

I hope you don't mind :)

Regards,

Sitaram

--
Sitaram Chamarty
+91-40-6667-3521 (work)
+91-92462-22927 (cell; please note new number)
440-3521 (TCS India VOIP)

[Christian MICHON]

On Thu, May 20, 2010 at 12:03 PM, Sitaram Chamarty <sit...@atc.tcs.com> wrote:
>
> on 20/05/10 14:53 Christian MICHON wrote:
>
>> working fine, thanks a lot
>>
>>>>>
>>>>> I'm curious how this information helps...
>>>>>
>>>>
>>>> Figure out the load of the gitolite server, especially parallel
>>>> uploads/downloads :-)
>>>
>>> I wonder how many people even look at the existing log file; I know I
>>> only created it for audit purposes, not performance. �Anyway let me know
>>> if the patch works for you.
>>>
>>
>> I do care :-)
>
> The original purpose of gitolite is security, and hence auditability of
> access is relevant.  Performance is not really central to gitolite's goals.

This is not about performance actually. It's more about knowing how
much data securely flows in and out, and the load required from that
machine.

>
> I also feel you should wrap "git-shell" on the server to acquire and log
> that info -- that is, it should not be a "gitolite" thing but a "git"
> thing, if you see what I mean.

Yes, I agree.

>
> Due to these reasons, I'm not too keen on putting this into mainline.
> Would you mind maintaining it as a patch on your side unless we hear
> from a few others that they also want it?

I'm fine with this.

I believe there's more value added in removing the 'system' calls from
your perl scripts to make it more robust or multi platform. I noticed
a lot of these calls when trying to setup a gitolite server on a linux
machine where only /bin/csh is allowed as login shell.

>
> My fear is that now you have elapsed time.  Soon the perf guys will want
> CPU and IO time, and so on and on... sorry but I will not be getting
> into all that :)  I'd rather nip it in the bud!

I'm not a perf guy. I'm concerned about security as you are (corporate
environment) and yet at the same time I want to know if I need to
upgrade or migrate the gitolite server if it is overloaded.

>
> I hope you don't mind :)
>

No problem :-)