Reference: [1] and other places linked from there.
My opinion: very low risk.
My reason: exploiting it requires that the attacker "pre-load the heap
with a useful callback address".
If your gitolite server is a dedicated server and your users do not have
shell access to it (any userid), I don't see how see this can happen.
If they do have shell access, well then upgrade to openssh 6.4 I guess.
[1]:
https://lwn.net/Articles/573355/