Source of "pattern must end in /" rule?

[Dave Abrahams]

Boost's SVN repository contains a bunch of files in the same directory
that need to be sorted into separate repositories. As a result, we need
svn2git to be able to match names that don't end in a forward slash.
I've been looking through the svn2git code for code that malfunctions
when that rule is violated, not finding anything obvious. Could someone
explain where that rule comes from?

That information would be a big help in closing
https://github.com/ryppl/boost-svn2git/issues/6

Thanks,

--
Dave Abrahams

[Dave Abrahams]

Hi Ian,

Thanks for answering, but I'm afraid I don't understand how your
response addresses my question. You're showing me a rule that doesn't
end in a slash, that supposedly worked. Are you saying that the
ends-in-a-slash rule is obsolete? Or are you saying something else?

on Fri Feb 08 2013, Ian Monroe <ian-AT-monroe.nu> wrote:

> From kdelibs-rules (located at git://anongit.kde.org/kde-ruleset)
>
> match /trunk/playground/libs/webkitkde/(kwebpage.*|network/kwebview.*|searchbar.*)
> repository KDE/kdelibs
> prefix kdewebkit/\1
> branch master
> max revision 851084
> end match
>
> Of course that was done a couple years ago.
>
> Ian

>> _______________________________________________
>> Kde-scm-interest mailing list
>> Kde-scm-...@kde.org
>> https://mail.kde.org/mailman/listinfo/kde-scm-interest

--
Dave Abrahams

[Dave Abrahams]

on Sun Feb 10 2013, Sebastian Dörner <sebastian-AT-sebastian-doerner.de> wrote:

> There used to be the problem that you couldn't match files below a
> recursion (i.e. after a recurse rule has been applied). This has
> however been fixed in 
>
> http://gitorious.org/svn2git/svn2git/commit/
> c59389ceed6214d85011a91263d935f46a501632
>
> so files should work fine. We needed this for the kdesdk conversion,
> which was just finished this weekend. These are the rules:
> https://projects.kde.org/projects/playground/sdk/kde-ruleset/
> repository/revisions/master/entry/kdesdk/dolphin-plugins-rules

Thanks for the info.

It turns out that the svn2git rule DSL is insufficient for Boost's
needs; we'd need thousands (or possibly tens of thousands) of rules to
handle everything we're doing. Therefore, our fork of the project is
being refactored and updated to handle a new rule DSL:
https://github.com/ryppl/Boost2Git/blob/develop/repositories.txt

>
>
>
>
> On 8 February 2013 17:53, Ian Monroe <i...@monroe.nu> wrote:
>
> I

>
> On Fri, Feb 8, 2013 at 9:45 AM, Dave Abrahams <da...@boostpro.com>
> wrote:
> >
> > Hi Ian,
> >
> > Thanks for answering, but I'm afraid I don't understand how
> your
> > response addresses my question.  You're showing me a rule that
> doesn't
> > end in a slash, that supposedly worked.  Are you saying that
> the
> > ends-in-a-slash rule is obsolete?  Or are you saying something
> else?
>

> I actually did check the kdelibs repo and it did bring over
> history as
> described in the rule for at least kwebpage.*. So yes, there
> isn't a
> end-in-slash-rule, or at least there wasn't in 2011.
>
> Ian

[Lars Viklund]

On Sun, Feb 10, 2013 at 07:48:16PM -0500, Dave Abrahams wrote:
>
> on Sun Feb 10 2013, Sebastian Dörner <sebastian-AT-sebastian-doerner.de> wrote:
>
> > There used to be the problem that you couldn't match files below a
> > recursion (i.e. after a recurse rule has been applied). This has
> > however been fixed in 
> >
> > http://gitorious.org/svn2git/svn2git/commit/
> > c59389ceed6214d85011a91263d935f46a501632
> >
> > so files should work fine. We needed this for the kdesdk conversion,
> > which was just finished this weekend. These are the rules:
> > https://projects.kde.org/projects/playground/sdk/kde-ruleset/
> > repository/revisions/master/entry/kdesdk/dolphin-plugins-rules
>
> Thanks for the info.
>
> It turns out that the svn2git rule DSL is insufficient for Boost's
> needs; we'd need thousands (or possibly tens of thousands) of rules to
> handle everything we're doing. Therefore, our fork of the project is
> being refactored and updated to handle a new rule DSL:
> https://github.com/ryppl/Boost2Git/blob/develop/repositories.txt

Have you lads and lasses given ESR's reposurgeon a look? The Boost
migration probably pre-dates it, but it seems to be a rather solid
tool for migrating complicated messes more properly than most tools.

[1] http://www.catb.org/~esr/reposurgeon/
[2] http://www.catb.org/~esr/reposurgeon/features.html

--
Lars Viklund | z...@acc.umu.se

[Dave Abrahams]

Wow, no, we didn't find that tool when we were doing our research.
Daniel's already off on his refactoring journey, having basically
understood the svn2git codebase. Even though it's by ESR, I'm not
confident he wouldn't want to refactor reposurgeon as well. Also, all
that stuff in the reposurgeon docs about how it has superior
documentation doesn't inspire a lot of confidence when AFAICT he makes
up the term "fossil-id" and never exactly defines it or describes the
format of the references file. reposurgeon also seems to make choices
for us that we'd like to make ourselves, for example

If "trunk" is eligible, it always becomes the master branch.

So, color me skeptical. But of course I could always be wrong :-)

--
Dave Abrahams

[Nicolás Alvarez]

A rule *that matches a directory* should end in a slash.

Say a SVN change arrives containing changes to /boost/bind/bind.hpp.
If you have a rule matching /boost/bind, svn2git will tell git-fast-import
to create a file called "/bind.hpp" (the trailing part of the SVN path that
wasn't matched by the regex). git-fast-import will then crash and burn
because paths in the fast-import stream must not have leading or trailing
slashes.

In addition, this would match (say) /boost/bind.hpp and put it into a file
called ".hpp" which you probably don't want. Or anything else that happens
to start with 'bind'.

If your rule has a trailing slash, then "/boost/bind/bind.hpp" with the rule
regex "/boost/bind/" leaves "bind.hpp" to use as the path in the git
repository, and everything goes well.


Now, on exporting individual files. Let's say you want to put
/boost/bind.hpp into the repository as well. If you just make a rule
for "/boost/bind.hpp$" (the $ is usually a good idea when matching files!),
it would match the entire SVN path, and the unmatched trailing part to use
as git path would be empty. To avoid this, you have to use the 'prefix'
statement inside the rule to give it a filename (svn2git will append the
path it got, but as I said, it's empty):

match /boost/bind.hpp$
repository bind
branch master
prefix bind.hpp
end match

Usually you would use captured subpatterns to your advantage here, by
matching /boost/(bind.hpp) and using \1 as the prefix.

--
Nicolas