Re: New message: Re: svn2git for Boost

[Dave Abrahams]

on Wed Jan 02 2013, Gitorious <no-reply-AT-gitorious.org> wrote:

> Hello dabrahams
>
> uqs has sent you a message on Gitorious:
> ----------------------------------------------------
>
> Hey,
>
> there's really not much infrastructure involved in doing so, you need
> to get direct access to the SVN repo somehow (most likely by using
> svnsync), then write your rules and a cronjob that runs svnsync,
> svn2git, and perhaps a push to github or gitorious, periodically.

Well, OK, we're doing that. I think we're going to push to bitbucket
because it offers a real commit graph... oh, interesting, Gitorious does
too, now. Perhaps we'll try both.

>
> I'm doing just that for the FreeBSD repositories,

Oh, wonderful!

> mail me at uqs-AT-FreeBSD.org if you need more help

Thanks. Well, here are a few other questions we have:

* Could you explain the "recurse" action? We've read through the
explanation on the website several times and still can't understand
it.

* Is there a tool for ensuring every file state committed to SVN is
part of some Git commit in some Git repo produced by the system?

* It seems like there could be more automation for dealing with svn
copies, and probably a few other tasks. Has it just turned out to be
not worth automating?

--
Dave Abrahams
BoostPro Computing Software Development Training
http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

[Dave Abrahams]

on Wed Jan 02 2013, Ulrich Spörlein <uqs-AT-FreeBSD.org> wrote:

> On Wed, 2013-01-02 at 12:53:02 -0500, Dave Abrahams wrote:
>>
>> on Wed Jan 02 2013, Gitorious <no-reply-AT-gitorious.org> wrote:
>>
>> > Hello dabrahams
>> >
>
>> > uqs has sent you a message on Gitorious:
>> > ----------------------------------------------------
>> >
>> > Hey,
>> >
>> > there's really not much infrastructure involved in doing so, you need
>> > to get direct access to the SVN repo somehow (most likely by using
>> > svnsync), then write your rules and a cronjob that runs svnsync,
>> > svn2git, and perhaps a push to github or gitorious, periodically.
>>
>> Well, OK, we're doing that. I think we're going to push to bitbucket
>> because it offers a real commit graph... oh, interesting, Gitorious does
>> too, now. Perhaps we'll try both.
>

> The more the merrier! :)

>
>> > I'm doing just that for the FreeBSD repositories,
>>
>> Oh, wonderful!
>>
>> > mail me at uqs-AT-FreeBSD.org if you need more help
>>
>> Thanks. Well, here are a few other questions we have:
>

> I should probably have mentioned that this is a pretty simple scenario,
> where one SVN repo gets converted to one GIT repo and the location of
> branches is well-formed and known in advance.

>
>> * Could you explain the "recurse" action? We've read through the
>> explanation on the website several times and still can't understand
>> it.
>

> I never used it, never needed to. I trust you've read this?
> http://techbase.kde.org/Projects/MoveToGit/UsingSvn2Git

Yes, that's where the "explanation on the website" that I referred to lives.
>
> In any case, how complex is your SVN repository in terms of tags and
> branches,

It's a royal mess; see
http://ci.boost.org/websvn/listing.php?repname=boost or
http://ci.boost.org/viewvc/boost/ if you dare.

> and do you need to absolutely get this right, because you are
> switching to GIT for good, or is this more of a long-term test into
> the feasibility of the conversion?

The former.

>> * Is there a tool for ensuring every file state committed to SVN is
>> part of some Git commit in some Git repo produced by the system?
>

> Not to my knowledge, sorry. I only sporadically do this for the FreeBSD
> repositories, because we use SVN keywords, and it's a pain in the neck
> to get a checkout w/o them expanded.

>
>> * It seems like there could be more automation for dealing with svn
>> copies, and probably a few other tasks. Has it just turned out to be
>> not worth automating?
>

> What do you mean by svn copies?

I mean "svn cp"s, a.k.a. branches. See the use of the --stop-on-copy
argument to svn log in the webpage you cited above.

[Dave Abrahams]

on Wed Jan 02 2013, Ulrich Spörlein <uqs-AT-FreeBSD.org> wrote:

>> Well, OK, we're doing that. I think we're going to push to bitbucket
>> because it offers a real commit graph... oh, interesting, Gitorious does
>> too, now. Perhaps we'll try both.
>

> The more the merrier! :)

FWIW, Gitorious kinda sucks for viewing merge history because the
commits aren't sorted by date. Compare:

https://bitbucket.org/boostorg/accumulators/commits/all
https://gitorious.org/accumulators/accumulators/graph/master?type=all

[Dave Abrahams]

Hi Thiago,

Thanks for getting back to us...

on Wed Jan 02 2013, Thiago Macieira <thiago-AT-kde.org> wrote:

> On quarta-feira, 2 de janeiro de 2013 15.05.43, Dave Abrahams wrote:
>> >> * Could you explain the "recurse" action? We've read through the
>> >>
>> >> explanation on the website several times and still can't understand
>> >> it.
>> >>
>> > I never used it, never needed to. I trust you've read this?
>> > http://techbase.kde.org/Projects/MoveToGit/UsingSvn2Git
>>
>> Yes, that's where the "explanation on the website" that I referred to lives.
>

> The "recurse" action is used when you svn cp a directory above the root of
> your repositories in order to create the branch.
>
> For example, KDE sources lived in /trunk/KDE and got branched to
> /branches/KDE/3.5. However, the kdelibs repository roots are at
> /trunk/KDE/kdelibs and /branches/KDE/3.5/kdelibs.
>
> The way that svn2git works is that it looks at the list of changed files from
> SVN. In the svn cp that created the 3.5 branch like above, the repository root
> is not listed, so it doesn't match any rules. You add a "recurse" rule to make
> the tool list the repository at the specified revision and then apply the rule
> set again.

Sorry, I'm still not seeing it. Could you:

- define what you mean by "the root of your repositories"
- give the path to what you consider to be the root for KDE
- give the exact svn cp command that corresponds to this KDE example
- show a fragment of ruleset that corresponds to this KDE example

? I think if I had all that, I'd understand.

Many thanks,

[Mathias Gaunard]

On 03/01/13 16:22, Dave Abrahams wrote:
>
> on Wed Jan 02 2013, Ulrich Spörlein <uqs-AT-FreeBSD.org> wrote:
>
>>> Well, OK, we're doing that. I think we're going to push to bitbucket
>>> because it offers a real commit graph... oh, interesting, Gitorious does
>>> too, now. Perhaps we'll try both.
>>
>> The more the merrier! :)
>
> FWIW, Gitorious kinda sucks for viewing merge history because the
> commits aren't sorted by date. Compare:
>
> https://bitbucket.org/boostorg/accumulators/commits/all
> https://gitorious.org/accumulators/accumulators/graph/master?type=all
>

I personally cannot access the first link.
It asks for a login.

[Dave Abrahams]

I made it public; you should be able to see it now.

[Mathias Gaunard]

On 03/01/13 17:05, Dave Abrahams wrote:
>
> on Thu Jan 03 2013, Mathias Gaunard <loufoque-AT-gmail.com> wrote:
>
>> On 03/01/13 16:22, Dave Abrahams wrote:
>>>
>>> on Wed Jan 02 2013, Ulrich Spörlein <uqs-AT-FreeBSD.org> wrote:
>>>
>>>>> Well, OK, we're doing that. I think we're going to push to bitbucket
>>>>> because it offers a real commit graph... oh, interesting, Gitorious does
>>>>> too, now. Perhaps we'll try both.
>>>>
>>>> The more the merrier! :)
>>>
>>> FWIW, Gitorious kinda sucks for viewing merge history because the
>>> commits aren't sorted by date. Compare:
>>>
>>> https://bitbucket.org/boostorg/accumulators/commits/all
>>> https://gitorious.org/accumulators/accumulators/graph/master?type=all
>>>
>>
>> I personally cannot access the first link.
>> It asks for a login.
>
> I made it public; you should be able to see it now.
>

I don't think one of these is clearly better than the other.
I think I prefer classical GUI applications (gitk, gitg) to both of them.

[Dave Abrahams]

on Thu Jan 03 2013, Mathias Gaunard <loufoque-AT-gmail.com> wrote:

> On 03/01/13 17:05, Dave Abrahams wrote:
>>
>> on Thu Jan 03 2013, Mathias Gaunard <loufoque-AT-gmail.com> wrote:
>>
>>> On 03/01/13 16:22, Dave Abrahams wrote:
>>>>
>>>> FWIW, Gitorious kinda sucks for viewing merge history because the
>>>> commits aren't sorted by date. Compare:
>>>>
>>>> https://bitbucket.org/boostorg/accumulators/commits/all
>>>> https://gitorious.org/accumulators/accumulators/graph/master?type=all
>>>>
>>>
>>> I personally cannot access the first link.
>>> It asks for a login.
>>
>> I made it public; you should be able to see it now.
>>
>
> I don't think one of these is clearly better than the other.

Really? Did you look at the dates of the commits in the gitorious
graph? Did you scroll down to find the tip of the "master" branch?

> I think I prefer classical GUI applications (gitk, gitg) to both of them.

If we are asking people—especially non-git-users—to tweak their own
manifests, and are updating the modularization based on a cronjob (or
something), I want them a dead-simple way for them to be able to review
the results in a useful way.

Also, aside from pagination, is gitk's display really substantially
different from bitbucket's? Not to my memory. (I've never used gitg).

[Mathias Gaunard]

On 03/01/13 17:31, Dave Abrahams wrote:

> If we are asking people—especially non-git-users—to tweak their own
> manifests, and are updating the modularization based on a cronjob (or
> something), I want them a dead-simple way for them to be able to review
> the results in a useful way.
>
> Also, aside from pagination, is gitk's display really substantially
> different from bitbucket's? Not to my memory. (I've never used gitg).

I think people should just use whatever tool they're most comfortable with.

[Dave Abrahams]

Thanks, I think I'm starting to get it...

on Thu Jan 03 2013, Thiago Macieira <thiago-AT-kde.org> wrote:

> On quinta-feira, 3 de janeiro de 2013 10.28.58, Dave Abrahams wrote:
>> > The way that svn2git works is that it looks at the list of changed files
>> > from SVN. In the svn cp that created the 3.5 branch like above, the
>> > repository root is not listed, so it doesn't match any rules. You add a
>> > "recurse" rule to make the tool list the repository at the specified
>> > revision and then apply the rule set again.
>>
>> Sorry, I'm still not seeing it. Could you:
>>
>> - define what you mean by "the root of your repositories"
>> - give the path to what you consider to be the root for KDE
>> - give the exact svn cp command that corresponds to this KDE example
>> - show a fragment of ruleset that corresponds to this KDE example
>>
>> ? I think if I had all that, I'd understand.
>

> The repository now called kdelibs.git was imported from:
>
> /trunk/KDE/kdelibs -> master
> /branches/KDE/(\d.\d)/kdelibs -> KDE/\1
>
> The branching done for KDE 4.0, for example, was done:
>
> svn cp $SVNROOT/trunk/KDE $SVNROOT/branches/KDE/4.0
>
> You can see it in the log:
>
> ------------------------------------------------------------------------
> r757128 | mueller | 2008-01-04 07:09:11 -0200 (Fri, 04 Jan 2008) | 2 lines
> Changed paths:
> A /branches/KDE/4.0 (from /trunk/KDE:757127)
>
> Creating KDE 4.0 branch
>
> ------------------------------------------------------------------------
>
> As you can see, there's no file path changing that would match
> /branches/KDE/(\d.\d)/kdelibs.

Surely when there are files in $SVNROOT/trunk/KDE/kdelibs and you do

svn cp $SVNROOT/trunk/KDE $SVNROOT/branches/KDE/4.0

it changes files whose path *begins* with a match to
/branches/KDE/(\d.\d)/kdelibs/. Is it really just about the fact that
the whole file paths are longer than that, so they don't match the
regexp?

Also, am I right to suppose the "recurse" action used to remedy this is:

--8<---------------cut here---------------start------------->8---
match /(branches|tags)/KDE/[^/]+/$
action recurse
end match
--8<---------------cut here---------------end--------------->8---

in converted/KDE/kdelibs-rules from the KDE ruleset?

[Dave Abrahams]

on Thu Jan 03 2013, Mathias Gaunard <loufoque-AT-gmail.com> wrote:

Um, yes. The question is what we will *provide* to lower the barrier to entry.
So far, it makes sense to push the results to bitbucket so people have
an easier time reviewing the resulting history.