[boost] Git Modularization Ready for Review

65 views
Skip to first unread message

Dave Abrahams

unread,
May 4, 2013, 3:08:22 PM5/4/13
to bo...@lists.boost.org

Hi All,

After substantial work, including massive changes by me and Daniel
Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
SVN commit in a Git repository. You can view the results at

http://github.com/boostorg
http://bitbucket.org/boostorg

or you can pull from these repositories and view them in your local
browser.

The conversion process is automated by
http://jenkins.boost.org/job/Boost2Git/ using the tool at
http://github.com/ryppl/Boost2Git

The rules that describe how commits are distributed are
in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt

To understand how to edit that file, please read
https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt

Daniel and I are ready to accept your feedback on the results of
modularization, and especially your pull requests containing edits to
the ruleset. I will the steering committee to establish a formal review
period, though.

--
Dave Abrahams



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Beman Dawes

unread,
May 5, 2013, 10:01:24 AM5/5/13
to Boost Developers List
On Sat, May 4, 2013 at 3:08 PM, Dave Abrahams <da...@boostpro.com> wrote:
>
> Hi All,
>
> After substantial work, including massive changes by me and Daniel

See https://github.com/ryppl/Boost2Git/commits to get an idea of how
big a job this was. The commit list is endless.

Dave and Daniel deserve special thanks for carrying this project through!

> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
> SVN commit in a Git repository. You can view the results at
>
> http://github.com/boostorg
> http://bitbucket.org/boostorg
>
> or you can pull from these repositories and view them in your local
> browser.

What is the story on branches and branch names? The Boost.System
release branch is there, but https://github.com/boostorg/system shows
34 branches, and it looks like most of those are branches for other
libraries that happened to include a Boost.System component. I suppose
Git has a way to blow the unwanted branches away from the system repo?

How close are we to being able to experiment with modular boost? Do we
know what the updated
https://svn.boost.org/trac/boost/wiki/TryModBoost instructions are?

Many thanks,

--Beman

Klaim - Joël Lamotte

unread,
May 5, 2013, 10:29:49 AM5/5/13
to Boost Developers List
On Sun, May 5, 2013 at 4:01 PM, Beman Dawes <bda...@acm.org> wrote:

> Dave and Daniel deserve special thanks for carrying this project through!


Indeed, special thanks!
I'm sure this work will be insanely useful.

Joel Lamotte

Saurav

unread,
May 5, 2013, 10:40:12 AM5/5/13
to bo...@lists.boost.org
>
> Hi All,
>
> After substantial work, including massive changes by me and Daniel
> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
> SVN commit in a Git repository. You can view the results at
>
> http://github.com/boostorg
> http://bitbucket.org/boostorg
>
> or you can pull from these repositories and view them in your local
> browser.
>

This is great news!
Please allow me to suggest http://windows.github.com/ here, a very easy way
to use git on windows.


Best regards,
Saurav

Vicente J. Botet Escriba

unread,
May 5, 2013, 4:32:25 PM5/5/13
to bo...@lists.boost.org
Le 04/05/13 21:08, Dave Abrahams a écrit :
> Hi All,
>
> After substantial work, including massive changes by me and Daniel
> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
> SVN commit in a Git repository. You can view the results at
>
> http://github.com/boostorg
> http://bitbucket.org/boostorg
>
> or you can pull from these repositories and view them in your local
> browser.
>
> The conversion process is automated by
> http://jenkins.boost.org/job/Boost2Git/ using the tool at
> http://github.com/ryppl/Boost2Git
>
> The rules that describe how commits are distributed are
> in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt
>
> To understand how to edit that file, please read
> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt
>
> Daniel and I are ready to accept your feedback on the results of
> modularization, and especially your pull requests containing edits to
> the ruleset. I will the steering committee to establish a formal review
> period, though.
>

Now that the it is ready for review, could you point where is the
documentation to review?

Could I move the following from repository thread to core?

"boost/detail/atomic_redef_macros.hpp" :
"include/boost/detail/atomic_redef_macros.hpp";
"boost/detail/atomic_undef_macros.hpp" :
"include/boost/detail/atomic_undef_macros.hpp"; I added these files.
They are used now by Boost.Thread, but it should be used by SmartPtr
(See https://svn.boost.org/trac/boost/ticket/6842 and
https://svn.boost.org/trac/boost/ticket/6843). Best, Vicente

Best,
Vicente

Dave Abrahams

unread,
May 6, 2013, 1:31:05 AM5/6/13
to bo...@lists.boost.org, Daniel Pfeifer

on Sun May 05 2013, Beman Dawes <bdawes-AT-acm.org> wrote:

>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>> SVN commit in a Git repository. You can view the results at
>>
>> http://github.com/boostorg
>> http://bitbucket.org/boostorg
>>
>> or you can pull from these repositories and view them in your local
>> browser.
>
> What is the story on branches and branch names?

I think I need a more precise question in order to give a good answer,
but I'll do my best. You can see the mapping of SVN paths to branch
names in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt
As mentioned in my previous posting, to understand the mapping, read
https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt

It's important to note that in SVN, branches persist basically
unobtrusively in history. If you don't explore the tree above /trunk/
and /branches/release/, you never see them. And even if you delete them
in some later revision, you can still get to them by rewinding history.
For example, there's nothing here:

http://ci.boost.org/svn-trac/browser/branches/unordered

but you can look back into history and there it is:

http://ci.boost.org/svn-trac/browser/branches/unordered?rev=42181

By contrast, you may find Git branches somewhat obtrusive. However, the
usual culture with Git is to delete feature branch references as soon as
they are merged to some other branch. After all, the merged-to branch
keeps the commit history alive and there's no longer any need to keep
the old label around. If you delete a branch without merging it, of
course, any commit history exclusively referenced by that branch is
lost.

Therefore we are conservative, preserving all SVN branches and tags as
Git refs. If branche or tags were deleted in some SVN revision, they'll
be converted to Git tags with the prefix "backups/" (see
https://github.com/boostorg/system/tags for examples). Otherwise, as a
rule, SVN paths below branches/ are converted to Git branches and SVN
paths below tags/ are converted to Git tags, but of course you can
control all that via the specific mappings in repositories.txt

There are quite a few paths that are being mapped to a branch prefixed
with "/old-branches/". I believe that pattern is a relic from earlier
modularization efforts, and is not applied consistently. However, it is
one good way of making some branches less obtrusive. We could decide to
automatically put any SVN branch that hasn't been touched since revision XXXX under
/old-branches/, for example.

> The Boost.System release branch is there, but
> https://github.com/boostorg/system shows 34 branches, and it looks
> like most of those are branches for other libraries that happened to
> include a Boost.System component.

Yeah, quite a few branches seem to be empty in Boost.System, for example
https://github.com/boostorg/system/commit/b59e42803ca7b37e2715657ba60532f988ab4199
We've made some effort to prevent the creation of empty branches,
although it's difficult; clearly what we have isn't working in all
cases. I'll see what I can do about it.

> I suppose Git has a way to blow the unwanted branches away from the
> system repo?

Blowing away branches in Git is trivial, if a little obscure:

git push <remote-name> :<branch-name>

> How close are we to being able to experiment with modular boost? Do we
> know what the updated
> https://svn.boost.org/trac/boost/wiki/TryModBoost instructions are?

As long as the goal is to do it with the current build system, we still
need to create the submodule references in
https://github.com/boostorg/boost that point to all the libraries and
tools. I'm working on that now. I think the instructions "should" be
pretty close to correct once we've done that, but am not certain. Daniel?

--
Dave Abrahams

Dave Abrahams

unread,
May 6, 2013, 1:34:37 AM5/6/13
to bo...@lists.boost.org

on Sun May 05 2013, "Vicente J. Botet Escriba" <vicente.botet-AT-wanadoo.fr> wrote:

> Le 04/05/13 21:08, Dave Abrahams a écrit :
>> Hi All,
>>
>> After substantial work, including massive changes by me and Daniel
>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>> SVN commit in a Git repository. You can view the results at
>
>>
>> http://github.com/boostorg
>> http://bitbucket.org/boostorg
>>
>> or you can pull from these repositories and view them in your local
>> browser.
>>
>> The conversion process is automated by
>> http://jenkins.boost.org/job/Boost2Git/ using the tool at
>> http://github.com/ryppl/Boost2Git
>>
>> The rules that describe how commits are distributed are
>> in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt
>>
>> To understand how to edit that file, please read
>> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt
>>
>> Daniel and I are ready to accept your feedback on the results of
>> modularization, and especially your pull requests containing edits to
>> the ruleset. I will the steering committee to establish a formal review
>> period, though.
>>
>
> Now that the it is ready for review, could you point where is the
> documentation to review?

Sorry, what documentation?

> Could I move the following from repository thread to core?
>
> "boost/detail/atomic_redef_macros.hpp" :
> "include/boost/detail/atomic_redef_macros.hpp";
> "boost/detail/atomic_undef_macros.hpp" :
> "include/boost/detail/atomic_undef_macros.hpp"; I added these
> files. They are used now by Boost.Thread, but it should be used by
> SmartPtr (See https://svn.boost.org/trac/boost/ticket/6842 and
> https://svn.boost.org/trac/boost/ticket/6843). Best, Vicente

First, are you sure they don't belong in their own module?
It might be a good idea to minimize the size of the core "blob."

Second, sure, we can do that. Please try editing the ruleset yourself
and submitting a pull request as described above. We need to find out
if our instructions are adequate.

Thanks,

--
Dave Abrahams

Jonathan Wakely

unread,
May 6, 2013, 3:39:23 AM5/6/13
to bo...@lists.boost.org, Daniel Pfeifer
On 6 May 2013 06:31, Dave Abrahams wrote:
>
>> I suppose Git has a way to blow the unwanted branches away from the
>> system repo?
>
> Blowing away branches in Git is trivial, if a little obscure:
>
> git push <remote-name> :<branch-name>

Modern versions of Git also support this less obscure form:

git push --delete <remote-name> <branch-name>

Beman Dawes

unread,
May 6, 2013, 10:58:53 AM5/6/13
to Boost Developers List
On Mon, May 6, 2013 at 1:34 AM, Dave Abrahams <da...@boostpro.com> wrote:
>
> on Sun May 05 2013, "Vicente J. Botet Escriba" <vicente.botet-AT-wanadoo.fr> wrote:
> ...
>
>> Now that the it is ready for review, could you point where is the
>> documentation to review?
>
> Sorry, what documentation?
>

Perhaps Vicente is referring to the documentation in the "Git and
Modular Boost" section of https://svn.boost.org/trac/boost/

If so, that's mostly my responsibility and I'll start updating it as
needed to reflect the full history now available.

--Beman

Beman Dawes

unread,
May 6, 2013, 11:32:10 AM5/6/13
to Boost Developers List, Daniel Pfeifer
On Mon, May 6, 2013 at 1:31 AM, Dave Abrahams <da...@boostpro.com> wrote:
>
> on Sun May 05 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
>
>>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>>> SVN commit in a Git repository. You can view the results at
>>>
>>> http://github.com/boostorg
>>> http://bitbucket.org/boostorg
>>>
>>> or you can pull from these repositories and view them in your local
>>> browser.
>>
>> What is the story on branches and branch names?
>
> I think I need a more precise question in order to give a good answer,
> but I'll do my best. You can see the mapping of SVN paths to branch
> names in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt
> As mentioned in my previous posting, to understand the mapping, read
> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt

One more precise question: Are we still planning to map trunk->develop
and branches/release->master?

> It's important to note that in SVN, branches persist basically
> unobtrusively in history. If you don't explore the tree above /trunk/
> and /branches/release/, you never see them. And even if you delete them
> in some later revision, you can still get to them by rewinding history.
> For example, there's nothing here:
>
> http://ci.boost.org/svn-trac/browser/branches/unordered
>
> but you can look back into history and there it is:
>
> http://ci.boost.org/svn-trac/browser/branches/unordered?rev=42181
>
> By contrast, you may find Git branches somewhat obtrusive. However, the
> usual culture with Git is to delete feature branch references as soon as
> they are merged to some other branch. After all, the merged-to branch
> keeps the commit history alive and there's no longer any need to keep
> the old label around. If you delete a branch without merging it, of
> course, any commit history exclusively referenced by that branch is
> lost.

Thanks - that's helpful. I've added the last paragraph to
https://svn.boost.org/trac/boost/wiki/StartModWorkflow

> Therefore we are conservative, preserving all SVN branches and tags as
> Git refs. If branche or tags were deleted in some SVN revision, they'll
> be converted to Git tags with the prefix "backups/" (see
> https://github.com/boostorg/system/tags for examples). Otherwise, as a
> rule, SVN paths below branches/ are converted to Git branches and SVN
> paths below tags/ are converted to Git tags, but of course you can
> control all that via the specific mappings in repositories.txt
>
> There are quite a few paths that are being mapped to a branch prefixed
> with "/old-branches/". I believe that pattern is a relic from earlier
> modularization efforts, and is not applied consistently. However, it is
> one good way of making some branches less obtrusive. We could decide to
> automatically put any SVN branch that hasn't been touched since revision XXXX under
> /old-branches/, for example.

That might be worth doing if it isn't too much work. Maybe XXXX could
be the revision that corresponds to say one year ago, or maybe 18
months ago to be a bit more conservative.

>> The Boost.System release branch is there, but
>> https://github.com/boostorg/system shows 34 branches, and it looks
>> like most of those are branches for other libraries that happened to
>> include a Boost.System component.
>
> Yeah, quite a few branches seem to be empty in Boost.System, for example
> https://github.com/boostorg/system/commit/b59e42803ca7b37e2715657ba60532f988ab4199
> We've made some effort to prevent the creation of empty branches,
> although it's difficult; clearly what we have isn't working in all
> cases. I'll see what I can do about it.

Seems like it would be helpful.

>> I suppose Git has a way to blow the unwanted branches away from the
>> system repo?
>
> Blowing away branches in Git is trivial, if a little obscure:
>
> git push <remote-name> :<branch-name>

Good, and even easier with Jonathan's later suggestion.

For a simple library like system, I'm only really interested in
develop and master branches.

>> How close are we to being able to experiment with modular boost? Do we
>> know what the updated
>> https://svn.boost.org/trac/boost/wiki/TryModBoost instructions are?
>
> As long as the goal is to do it with the current build system, we still
> need to create the submodule references in
> https://github.com/boostorg/boost that point to all the libraries and
> tools. I'm working on that now. I think the instructions "should" be
> pretty close to correct once we've done that, but am not certain. Daniel?

There was a tentative change to the current build system that would
allow b2/bjam to handle the "cmake -P forward_headers.cmake" step. I
don't know if that change was ever finalized. If not we will have to
pester the Boost.Build folks to get it working.

--Beman

Vicente J. Botet Escriba

unread,
May 6, 2013, 11:42:57 AM5/6/13
to bo...@lists.boost.org
Le 06/05/13 07:34, Dave Abrahams a écrit :

> on Sun May 05 2013, "Vicente J. Botet Escriba" <vicente.botet-AT-wanadoo.fr> wrote:
>
>> Le 04/05/13 21:08, Dave Abrahams a écrit :
>>> Hi All,
>>>
>>> After substantial work, including massive changes by me and Daniel
>>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>>> SVN commit in a Git repository. You can view the results at
>>> http://github.com/boostorg
>>> http://bitbucket.org/boostorg
>>>
>>> or you can pull from these repositories and view them in your local
>>> browser.
>>>
>>> The conversion process is automated by
>>> http://jenkins.boost.org/job/Boost2Git/ using the tool at
>>> http://github.com/ryppl/Boost2Git
>>>
>>> The rules that describe how commits are distributed are
>>> in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt
>>>
>>> To understand how to edit that file, please read
>>> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt
>>>
>>> Daniel and I are ready to accept your feedback on the results of
>>> modularization, and especially your pull requests containing edits to
>>> the ruleset. I will the steering committee to establish a formal review
>>> period, though.
>>>
>> Now that the it is ready for review, could you point where is the
>> documentation to review?
> Sorry, what documentation?
What is ready for review then, the split on modules?

>
>> Could I move the following from repository thread to core?
>>
>> "boost/detail/atomic_redef_macros.hpp" :
>> "include/boost/detail/atomic_redef_macros.hpp";
>> "boost/detail/atomic_undef_macros.hpp" :
>> "include/boost/detail/atomic_undef_macros.hpp"; I added these
>> files. They are used now by Boost.Thread, but it should be used by
>> SmartPtr (See https://svn.boost.org/trac/boost/ticket/6842 and
>> https://svn.boost.org/trac/boost/ticket/6843). Best, Vicente
> First, are you sure they don't belong in their own module?
> It might be a good idea to minimize the size of the core "blob."
Where this kind of workarounds should go, Boost.Config?

>
> Second, sure, we can do that. Please try editing the ruleset yourself
> and submitting a pull request as described above. We need to find out
> if our instructions are adequate.
>
I'm not sure I know how this must be done :(
Anyway, I will try.

Vicente

Niall Douglas

unread,
May 6, 2013, 12:12:13 PM5/6/13
to bo...@lists.boost.org
> After substantial work, including massive changes by me and Daniel Pfeifer
to
> KDE's svn->Git conversion tool, we have captured every Boost SVN commit in
a
> Git repository. You can view the results at
>
> http://github.com/boostorg
> http://bitbucket.org/boostorg
>
> Daniel and I are ready to accept your feedback on the results of
modularization,
> and especially your pull requests containing edits to the ruleset. I will
the
> steering committee to establish a formal review period, though.

I'm a bit confused. This looks like a hand written map of parts of Boost SVN
onto multiple GIT repos. Am I understanding correctly?

I'm far more confused as to why? Surely the conventional approach is to
convert a SVN repo to GIT. Then people hive off their particular bits into
submodules as and when they themselves decide and they are ready. It looks
here that Boost has decided on skipping the intermediate stage and going
straight to submodules. That sounds awfully fraught: it also demands a lot
from those library maintainers not familiar with Git. Submodules are
extremely different to SVN externals - they're a very light link, and in
many occasions Git does not auto-update submodules even when you tell it
(i.e. it'll fetch, but silently not checkout head if it thinks you might
lose data).

Also, how is Boost going to manage dependency breakage across multiple
submodules? Are you planning to keep headers in one monolithic repo but keep
anything resulting in a DLL/SO in a submodule? Or are you planning to break
out the headers into submodules too? I can see it becoming quite easy for
submodules or branches or forks thereof to get out of step with one another.
I can see the present submodule heavy approach introducing a lot of anti-Git
workflow because those submodules inside the boostorg organization are
treated as special compared to other submodules.

I'm confused. Sorry. I'm probably not making any sense. Is there a design
rationale document somewhere? I can't see one up to date on Google.

Niall

---
Opinions expressed here are my own and do not necessarily represent those of
BlackBerry Inc.

Dave Abrahams

unread,
May 7, 2013, 12:11:02 AM5/7/13
to bo...@lists.boost.org

The way the modules are being split up, and the resulting Git histories.

>>> Could I move the following from repository thread to core?
>>>
>>> "boost/detail/atomic_redef_macros.hpp" :
>>> "include/boost/detail/atomic_redef_macros.hpp";
>>> "boost/detail/atomic_undef_macros.hpp" :
>>> "include/boost/detail/atomic_undef_macros.hpp"; I added these
>>> files. They are used now by Boost.Thread, but it should be used by
>>> SmartPtr (See https://svn.boost.org/trac/boost/ticket/6842 and
>>> https://svn.boost.org/trac/boost/ticket/6843). Best, Vicente
>> First, are you sure they don't belong in their own module?
>> It might be a good idea to minimize the size of the core "blob."
> Where this kind of workarounds should go, Boost.Config?

Oh, I didn't realize they were just workarounds. Perhaps core would be
best.

>> Second, sure, we can do that. Please try editing the ruleset yourself
>> and submitting a pull request as described above. We need to find out
>> if our instructions are adequate.
>>
> I'm not sure I know how this must be done :(

There's only one way to find out!

> Anyway, I will try.

Thanks!

--
Dave Abrahams

Dave Abrahams

unread,
May 7, 2013, 12:28:16 AM5/7/13
to bo...@lists.boost.org

on Mon May 06 2013, Niall Douglas <ndouglas-AT-blackberry.com> wrote:

>> After substantial work, including massive changes by me and Daniel Pfeifer
> to
>> KDE's svn->Git conversion tool, we have captured every Boost SVN commit in
> a
>> Git repository. You can view the results at
>>
>> http://github.com/boostorg
>> http://bitbucket.org/boostorg
>>
>> Daniel and I are ready to accept your feedback on the results of
> modularization,
>> and especially your pull requests containing edits to the ruleset. I will
> the
>> steering committee to establish a formal review period, though.
>
> I'm a bit confused. This looks like a hand written map of parts of Boost SVN
> onto multiple GIT repos. Am I understanding correctly?

Depends what you mean by hand written (there's a lot of automation
involved), but yes.

> I'm far more confused as to why? Surely the conventional approach is to
> convert a SVN repo to GIT.

Good luck with that, given the way SVN "branches" (really directories)
have been used in Boost's history. No automated tool is able to
determine how to map those things properly onto Git branches. You can
see evidence of that in repositories.txt (look at the "build" repository
declarations for example).

> Then people hive off their particular bits into submodules as and when
> they themselves decide and they are ready. It looks here that Boost
> has decided on skipping the intermediate stage and going straight to
> submodules. That sounds awfully fraught: it also demands a lot from
> those library maintainers not familiar with Git. Submodules are
> extremely different to SVN externals - they're a very light link, and
> in many occasions Git does not auto-update submodules even when you
> tell it (i.e. it'll fetch, but silently not checkout head if it thinks
> you might lose data).

Which is exactly the right behavior, except the silent part (although
I'd be surprised if someone hasn't fixed that by now)

You act as though we haven't been thinking about how to do this well and
working on it for years already. We're quite well-aware how submodules
work. The use of submodules by most developers is supposed to be a
temporary transitional measure, since modularization is designed to
allow you to work with only the parts of Boost relevant to your project.

> Also, how is Boost going to manage dependency breakage across multiple
> submodules?

What is "dependency breakage?"

> Are you planning to keep headers in one monolithic repo

No

> but keep anything resulting in a DLL/SO in a submodule? Or are you
> planning to break out the headers into submodules too?

Yes; it's already done, modulo some expected adjustments due to the
review I am requesting.

> I can see it becoming quite easy for submodules or branches or forks
> thereof to get out of step with one another.

That's a *good* thing. We're trying to decouple. Most libraries will
be developed against the previous release of their dependencies.

> I can see the present submodule heavy approach introducing a lot of
> anti-Git workflow because those submodules inside the boostorg
> organization are treated as special compared to other submodules.

I don't have any idea what you mean here

> I'm confused. Sorry. I'm probably not making any sense. Is there a design
> rationale document somewhere? I can't see one up to date on Google.

Beman has been working on the big picture doc. Beman?

Dave Abrahams

unread,
May 7, 2013, 12:36:32 AM5/7/13
to bo...@lists.boost.org, Daniel Pfeifer

on Mon May 06 2013, Beman Dawes <bdawes-AT-acm.org> wrote:

> On Mon, May 6, 2013 at 1:31 AM, Dave Abrahams <da...@boostpro.com> wrote:
>>
>> on Sun May 05 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
>>
>>>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>>>> SVN commit in a Git repository. You can view the results at
>
>>>>
>>>> http://github.com/boostorg
>>>> http://bitbucket.org/boostorg
>>>>
>>>> or you can pull from these repositories and view them in your local
>>>> browser.
>>>
>>> What is the story on branches and branch names?
>>
>> I think I need a more precise question in order to give a good answer,
>> but I'll do my best. You can see the mapping of SVN paths to branch
>> names in https://github.com/ryppl/Boost2Git/blob/master/repositories.txt
>> As mentioned in my previous posting, to understand the mapping, read
>> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt
>
> One more precise question: Are we still planning to map trunk->develop
> and branches/release->master?

Presently, as you can see, the mapping is trunk->master and
branches/release->release. It could be easily changed, and I think that
might be a very good idea because otherwise we have to do a bulk branch
rename, which could cause havoc with developers. Daniel felt that the
branch names should reflect the names we've used historically, but I
think I disagree with him on that. I think there's too little benefit.
Daniel, shall we argue? :-)
I think it's probably doable. Another thing we might consider is mapping
them to tags/old-branches/...

These things are fairly easy to adjust.

>>> The Boost.System release branch is there, but
>>> https://github.com/boostorg/system shows 34 branches, and it looks
>>> like most of those are branches for other libraries that happened to
>>> include a Boost.System component.
>>
>> Yeah, quite a few branches seem to be empty in Boost.System, for example
>> https://github.com/boostorg/system/commit/b59e42803ca7b37e2715657ba60532f988ab4199
>> We've made some effort to prevent the creation of empty branches,
>> although it's difficult; clearly what we have isn't working in all
>> cases. I'll see what I can do about it.
>
> Seems like it would be helpful.
>
>>> I suppose Git has a way to blow the unwanted branches away from the
>>> system repo?
>>
>> Blowing away branches in Git is trivial, if a little obscure:
>>
>> git push <remote-name> :<branch-name>
>
> Good, and even easier with Jonathan's later suggestion.
>
> For a simple library like system, I'm only really interested in
> develop and master branches.

Hopefully we'll be able to automatically prune the irrelevant ones, but
if not, it's easy to write a script to prune everything but a set of
specified branches. We can give that out to maintainers and they can
make their own choices.

>>> How close are we to being able to experiment with modular boost? Do we
>>> know what the updated
>>> https://svn.boost.org/trac/boost/wiki/TryModBoost instructions are?
>>
>> As long as the goal is to do it with the current build system, we still
>> need to create the submodule references in
>> https://github.com/boostorg/boost that point to all the libraries and
>> tools. I'm working on that now. I think the instructions "should" be
>> pretty close to correct once we've done that, but am not certain. Daniel?
>
> There was a tentative change to the current build system that would
> allow b2/bjam to handle the "cmake -P forward_headers.cmake" step. I
> don't know if that change was ever finalized. If not we will have to
> pester the Boost.Build folks to get it working.

I don't know either.

--
Dave Abrahams

Beman Dawes

unread,
May 7, 2013, 9:57:22 AM5/7/13
to Boost Developers List
On Tue, May 7, 2013 at 12:28 AM, Dave Abrahams <da...@boostpro.com> wrote:
>
> on Mon May 06 2013, Niall Douglas <ndouglas-AT-blackberry.com> wrote:
>
>...
>> I'm confused. Sorry. I'm probably not making any sense. Is there a design
>> rationale document somewhere? I can't see one up to date on Google.
>
> Beman has been working on the big picture doc. Beman?

There are links to all the docs in the "Git and Modular Boost" section
of https://svn.boost.org/trac/boost/wiki/WikiStart

Might be useful to produce an "Overview" page, which may be what the
OP is asking for.

--Beman

Tom Kent

unread,
May 7, 2013, 12:17:54 PM5/7/13
to Boost Developers List
Looking forward to getting this working fully.

Are there changes to the regression script tools that would let the
regression tests run against the git repos? Have they been run already?

Tom

Stefan Seefeld

unread,
May 7, 2013, 9:25:12 PM5/7/13
to bo...@lists.boost.org, Dave Abrahams
Dave,

any idea what happened to the sandbox "module" ? It appears to be empty
(even though it is pretty big, i.e. contains a lot of history).
Coincidentally, `git log` reports the last commit in 2003. What happened
to the rest ?

Thanks,
Stefan

--

...ich hab' noch einen Koffer in Berlin...

Dave Abrahams

unread,
May 7, 2013, 10:58:30 PM5/7/13
to Stefan Seefeld, bo...@lists.boost.org
https://github.com/boostorg/sandbox/tree/trunk

HTH

Sent from my moss-covered three-handled family gradunza

Matus Chochlik

unread,
May 8, 2013, 1:32:29 AM5/8/13
to bo...@lists.boost.org
Wouldn't it be better if trunk was set as the main branch of the sandbox
repository to avoid confusion ?


On Wed, May 8, 2013 at 4:58 AM, Dave Abrahams <da...@boostpro.com> wrote:

> https://github.com/boostorg/sandbox/tree/trunk
>
> HTH
>
> Sent from my moss-covered three-handled family gradunza
>
> BR

Matus

Alex Perry

unread,
May 8, 2013, 4:49:02 AM5/8/13
to bo...@lists.boost.org
On 08 May 2013 03:59 Dave Abrahams [mailto:da...@boostpro.com] wrote :
>
> Sent from my moss-covered three-handled family gradunza
>

By a process of "Calculatus Eliminatus"[1] I suspect a MacGuffin[2] driving the plot forwards to the goal of modularisation.

If it was just a nice tag line in protest to all the "sent from my iphone" it certainly made me laugh

Alex

[1] http://en.wikipedia.org/wiki/The_Cat_in_the_Hat_(TV_special)
[2] http://en.wikipedia.org/wiki/MacGuffin

Stefan Seefeld

unread,
May 8, 2013, 7:13:34 AM5/8/13
to Dave Abrahams, bo...@lists.boost.org
On 05/07/2013 10:58 PM, Dave Abrahams wrote:
> https://github.com/boostorg/sandbox/tree/trunk
>
> HTH

Thanks ! Now I can indeed see some content in the web browser. However,
the suggested URL for cloning is sill displayed to be
https://github.com/boostorg/sandbox.git, which I used yesterday with
`git clone <url>`, which results in an empty directory. Something seems
wrong there...

Thanks for all your effort !

Matus Chochlik

unread,
May 8, 2013, 7:35:13 AM5/8/13
to bo...@lists.boost.org
On Wed, May 8, 2013 at 1:13 PM, Stefan Seefeld <ste...@seefeld.name> wrote:

> On 05/07/2013 10:58 PM, Dave Abrahams wrote:
> > https://github.com/boostorg/sandbox/tree/trunk
> >
> > HTH
>
> Thanks ! Now I can indeed see some content in the web browser. However,
> the suggested URL for cloning is sill displayed to be
> https://github.com/boostorg/sandbox.git, which I used yesterday with
> `git clone <url>`, which results in an empty directory. Something seems
> wrong there...
>
>
You need to do 'git checkout trunk'


Matus

Stefan Seefeld

unread,
May 8, 2013, 7:52:29 AM5/8/13
to bo...@lists.boost.org
On 05/08/2013 07:35 AM, Matus Chochlik wrote:
> On Wed, May 8, 2013 at 1:13 PM, Stefan Seefeld <ste...@seefeld.name> wrote:
>
>> On 05/07/2013 10:58 PM, Dave Abrahams wrote:
>>> https://github.com/boostorg/sandbox/tree/trunk
>>>
>>> HTH
>> Thanks ! Now I can indeed see some content in the web browser. However,
>> the suggested URL for cloning is sill displayed to be
>> https://github.com/boostorg/sandbox.git, which I used yesterday with
>> `git clone <url>`, which results in an empty directory. Something seems
>> wrong there...
>>
>>
> You need to do 'git checkout trunk'

Ah, indeed I do. It seems the normal 'git clone' call checks out the
'master' branch by default, but since this sandbox doesn't have one
under that name, I had to check out explicitly. Now all looks good.
Anyhow, this is just for archeology, as it was suggested we use the
(original) sandbox read-only, and develop / maintain individual projects
elsewhere.


Thanks,

Stefan

--

...ich hab' noch einen Koffer in Berlin...


Beman Dawes

unread,
May 8, 2013, 10:02:29 AM5/8/13
to Boost Developers List, Daniel Pfeifer
On Tue, May 7, 2013 at 12:36 AM, Dave Abrahams <da...@boostpro.com> wrote:
>
> on Mon May 06 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
>
>...
>> One more precise question: Are we still planning to map trunk->develop
>> and branches/release->master?
>
> Presently, as you can see, the mapping is trunk->master and
> branches/release->release. It could be easily changed, and I think that
> might be a very good idea because otherwise we have to do a bulk branch
> rename, which could cause havoc with developers.

Strongly agree.

We had previously agree to follow the Git Flow conventions, and so map
trunk -> develop and branches/release -> master. It doesn't matter
that the current mapping can be easily changed - the conversion isn't
ready for prime time until that is done.

OTOH, I'm not so worried about having some extra branches. It seems
reasonable to ask developers to clean out unwanted branches. If you
can clean out the empty ones, fine. Likewise for branches that have
seen no activity for years. But I wouldn't want to delay the
conversion waiting for such niceties.

--Beman

Daniel Pfeifer

unread,
May 8, 2013, 3:32:50 PM5/8/13
to Beman Dawes, Boost Developers List
2013/5/8 Beman Dawes <bda...@acm.org>

> On Tue, May 7, 2013 at 12:36 AM, Dave Abrahams <da...@boostpro.com> wrote:
> >
> > on Mon May 06 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
> >
> >...
> >> One more precise question: Are we still planning to map trunk->develop
> >> and branches/release->master?
> >
> > Presently, as you can see, the mapping is trunk->master and
> > branches/release->release. It could be easily changed, and I think that
> > might be a very good idea because otherwise we have to do a bulk branch
> > rename, which could cause havoc with developers.
>
> Strongly agree.
>
> We had previously agree to follow the Git Flow conventions, and so map
> trunk -> develop and branches/release -> master. It doesn't matter
> that the current mapping can be easily changed - the conversion isn't
> ready for prime time until that is done.
>

I am not opposed to master/develop. In fact, I am much in favor for Git
Flow.

The concern that I have with our current "release" branch is that it does
not correspond to "master" in the Git Flow model.

In Git Flow, you create a temporary release branch for stabilization. When
the stabilization is completed, you

1. merge to "master",
2. merge to "develop",
3. create a tag, and
4. drop the release branch.

In the end, "master" is stable at any point in time.

However, in SVN we currently use "release" as "release preparation" or
stabilization. This branch was not always guaranteed to be stable back in
history.

If we want to have a branch that corresponds to "master" in Git Flow, we
need to artificially recreate it based on the tags. This is certainly not
impossible, but requires further work.

-- Daniel

Daniel James

unread,
May 8, 2013, 7:00:20 PM5/8/13
to bo...@lists.boost.org, Daniel Pfeifer
On 6 May 2013 16:32, Beman Dawes <bda...@acm.org> wrote:
> One more precise question: Are we still planning to map trunk->develop
> and branches/release->master?

I'd hope not, the useful history is in trunk.

Beman Dawes

unread,
May 8, 2013, 9:54:03 PM5/8/13
to Boost Developers List, Daniel Pfeifer
On Wed, May 8, 2013 at 7:00 PM, Daniel James <dan...@calamity.org.uk> wrote:
> On 6 May 2013 16:32, Beman Dawes <bda...@acm.org> wrote:
>> One more precise question: Are we still planning to map trunk->develop
>> and branches/release->master?
>
> I'd hope not, the useful history is in trunk.

IIUC, It is really just a renaming, so the history is still there. If
you do a log on develop you will see all the history from what used to
be called trunk.

Part of why I consider the mapping (or, if you prefer, renaming) to be
so important is that we need to verify that the history follows along
and is not disrupted.

--Beman

Dave Abrahams

unread,
May 8, 2013, 10:31:49 PM5/8/13
to Stefan Seefeld, bo...@lists.boost.org

on Wed May 08 2013, Stefan Seefeld <stefan-AT-seefeld.name> wrote:

> On 05/07/2013 10:58 PM, Dave Abrahams wrote:
>> https://github.com/boostorg/sandbox/tree/trunk
>>
>> HTH
>
> Thanks ! Now I can indeed see some content in the web browser. However,
> the suggested URL for cloning is sill displayed to be
> https://github.com/boostorg/sandbox.git, which I used yesterday with
> `git clone <url>`, which results in an empty directory. Something seems
> wrong there...
>
> Thanks for all your effort !

OK, fixed.

--
Dave Abrahams

Dave Abrahams

unread,
May 8, 2013, 10:32:08 PM5/8/13
to bo...@lists.boost.org

on Tue May 07 2013, Matus Chochlik <chochlik-AT-gmail.com> wrote:

> Wouldn't it be better if trunk was set as the main branch of the sandbox
> repository to avoid confusion ?

Done.

> On Wed, May 8, 2013 at 4:58 AM, Dave Abrahams <da...@boostpro.com> wrote:
>
>> https://github.com/boostorg/sandbox/tree/trunk
>>
>> HTH
>>
>> Sent from my moss-covered three-handled family gradunza
>>
>> BR
>
> Matus
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

--
Dave Abrahams

Daniel James

unread,
May 9, 2013, 4:09:49 AM5/9/13
to bo...@lists.boost.org, Daniel Pfeifer
On 9 May 2013 02:54, Beman Dawes <bda...@acm.org> wrote:
> IIUC, It is really just a renaming, so the history is still there. If
> you do a log on develop you will see all the history from what used to
> be called trunk.

The problem is that if someone checks out a repo and run git annotate
or bisect, then it's quite likely they'll just find changes with the
title 'Merge to release' with lots of changes merged together. Which
is not very helpful.

> Part of why I consider the mapping (or, if you prefer, renaming) to be
> so important is that we need to verify that the history follows along
> and is not disrupted.

Can't we design a release system so that the current release branch is
labelled with something like 'svn-release' or just 'release' and that
it used until a developer tags/branches for a new release? I don't
know how it's planned to work, but I expect that's feasible.

Niall Douglas

unread,
May 9, 2013, 11:17:54 AM5/9/13
to bo...@lists.boost.org
> > I'm far more confused as to why? Surely the conventional approach is to
> > convert a SVN repo to GIT.
>
> Good luck with that, given the way SVN "branches" (really directories)
> have been used in Boost's history. No automated tool is able to
> determine how to map those things properly onto Git branches. You can
> see evidence of that in repositories.txt (look at the "build" repository
> declarations for example).

Me personally I'd just chuck away any unmappable historical information.
Chalk it up to a cost of transition. If they really need the history, they
can go fetch it from the read-only SVN repo.

> You act as though we haven't been thinking about how to do this well and
> working on it for years already. We're quite well-aware how submodules
> work. The use of submodules by most developers is supposed to be a
> temporary transitional measure, since modularization is designed to
> allow you to work with only the parts of Boost relevant to your project.

You have mistaken my confusion for criticism. I am not criticizing. I am
confused.

> > Also, how is Boost going to manage dependency breakage across multiple
> > submodules?
>
> What is "dependency breakage?"

Generally speaking Git submodules are for loosely coupled relations because
of how Git implements submodules. Some of Boost's constituent libraries do
meet that definition (BPL, BGL, Math). Some do not, as they are too
essential because so many other libraries depend on them (anything likely to
enter the standard with TR2, the MPL, the PPL etc).

I fear that if modularization is taken to its logical extreme, you could see
submodules get out of step with other submodules. You may of course have
already anticipated this and have implemented something I haven't realized.
As I said, I am confused.

> > Are you planning to keep headers in one monolithic repo
>
> No
>
> > but keep anything resulting in a DLL/SO in a submodule? Or are you
> > planning to break out the headers into submodules too?
>
> Yes; it's already done, modulo some expected adjustments due to the
> review I am requesting.

One solution is to require all findable headers to live inside the main
Boost repo [1], and only implementation to reside in submodules. That keeps
minds focused on dependency management.

[1]: By findable I mean that when Boost library users do #include
<boost/whatever> they get the main Boost repo version, not the submodule
version. I absolutely would expect an automated tool to pull headers from
submodules, check them for ABI breakage and push them into the main repo. My
point is that some sanity check ought to be there.

> > I can see it becoming quite easy for submodules or branches or forks
> > thereof to get out of step with one another.
>
> That's a *good* thing. We're trying to decouple. Most libraries will
> be developed against the previous release of their dependencies.
>
> > I can see the present submodule heavy approach introducing a lot of
> > anti-Git workflow because those submodules inside the boostorg
> > organization are treated as special compared to other submodules.
>
> I don't have any idea what you mean here

Git generally has you clone the authoritative master repo, you create a
branch, make your changes. Once you're finished, you fetch latest master,
merge into your branch until it's up to date, then generate a patch or pull
request from your branch to authoritative master. Whoever gatekeeps that
repo does the merge.

Where I'm getting confused - again - is that it would seem that the
submodule repos inside the Boost org section on github are the ones to be
regression tested and used to assemble releases. So, instead of treating all
git repos as one-among-many as say Linux would do, we've got a special
"high" gold candidate set of git repos and then all other git repos. This
appears to add an extra layer of syncing and merging, because surely Boost
admin has the power to modify any of the high gc set of repos.

I'm not saying that this choice isn't for very good reasons. It's just that
from my experience a submodule heavy git repo with lots of tightly coupled
submodules becomes a royal PITA for end users because you get all sorts of
weird obtuse breakages for apparently no reason at the time. Maybe of course
it's just me, and I have some mental block others don't have.

And I absolutely agree that other projects such as KDE have adopted your
approach and it seems to work for them. However Boost is not KDE. In my
mental model of the world, Boost more resembles the Linux kernel in terms of
interdependencies and size. KDE has a vastly more stable intra-module ABI
and much more loosely coupled submodules. This interpretation may be a
mistake of mine.

Alexander Lamaison

unread,
May 9, 2013, 1:12:34 PM5/9/13
to bo...@lists.boost.org
Niall Douglas <ndou...@blackberry.com> writes:

>> > I'm far more confused as to why? Surely the conventional approach is to
>> > convert a SVN repo to GIT.
>>
>> Good luck with that, given the way SVN "branches" (really directories)
>> have been used in Boost's history. No automated tool is able to
>> determine how to map those things properly onto Git branches. You can
>> see evidence of that in repositories.txt (look at the "build" repository
>> declarations for example).
>
> Me personally I'd just chuck away any unmappable historical information.
> Chalk it up to a cost of transition. If they really need the history, they
> can go fetch it from the read-only SVN repo.

I see you've not been keeping up with the list lately ;) Daniel et. al.
suggested doing just that a few months ago and was met with such a
chorus of criticism, they didn't really have a choice but to fix it.

Personally, I agree with the chorus. After all, the point of a VCS is
to have a history of the code's evolution to a point. The VCS, be it
SVN, Git, whatever, is just a means to get that history. Jetissoning
the ends for the means seems misguided.

What happens in a few years time when Git is replaced with the next big
thing? Do we lose the history again? And then again when that gets
replaced too?

>> You act as though we haven't been thinking about how to do this well and
>> working on it for years already. We're quite well-aware how submodules
>> work. The use of submodules by most developers is supposed to be a
>> temporary transitional measure, since modularization is designed to
>> allow you to work with only the parts of Boost relevant to your project.
>
> You have mistaken my confusion for criticism. I am not criticizing. I am
> confused.
>
>> > Also, how is Boost going to manage dependency breakage across multiple
>> > submodules?
>>
>> What is "dependency breakage?"
>
> Generally speaking Git submodules are for loosely coupled relations because
> of how Git implements submodules. Some of Boost's constituent libraries do
> meet that definition (BPL, BGL, Math). Some do not, as they are too
> essential because so many other libraries depend on them (anything likely to
> enter the standard with TR2, the MPL, the PPL etc).

A blob that is sometimes needed but must be supplied is a problem. A
blob that is always needed but does not have to be supplied is not a
problem.

> I fear that if modularization is taken to its logical extreme, you could see
> submodules get out of step with other submodules. You may of course have
> already anticipated this and have implemented something I haven't realized.
> As I said, I am confused.

Can you explain a bit more about what you mean by out-of-step? The whole
point of modularising the code is to *help* modules to get out-of-step
and therefore be easier to develop and test independent of what other
Boost libraries are doing. But perhaps you mean something else?

>> > Are you planning to keep headers in one monolithic repo
>>
>> No
>>
>> > but keep anything resulting in a DLL/SO in a submodule? Or are you
>> > planning to break out the headers into submodules too?
>>
>> Yes; it's already done, modulo some expected adjustments due to the
>> review I am requesting.
>
> One solution is to require all findable headers to live inside the main
> Boost repo [1], and only implementation to reside in submodules. That keeps
> minds focused on dependency management.
>
> [1]: By findable I mean that when Boost library users do #include
> <boost/whatever> they get the main Boost repo version, not the submodule
> version. I absolutely would expect an automated tool to pull headers from
> submodules, check them for ABI breakage and push them into the main repo. My
> point is that some sanity check ought to be there.

I'm not following why you would want to do this. Perhaps you can
explain what problem you are anticipating and how this would solve it?

I also don't get what 'findable' means. What would a non-findable
header be?

Alex

--
Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)

Niall Douglas

unread,
May 10, 2013, 11:50:52 AM5/10/13
to bo...@lists.boost.org
> Niall Douglas <ndou...@blackberry.com> writes:
>
> > Me personally I'd just chuck away any unmappable historical information.
> > Chalk it up to a cost of transition. If they really need the history,
> > they can go fetch it from the read-only SVN repo.
>
> I see you've not been keeping up with the list lately ;) Daniel et. al.
> suggested doing just that a few months ago and was met with such a chorus
of
> criticism, they didn't really have a choice but to fix it.

Never actually was on this list, not ever, until recently as it's a lot of
extra reading. Been involved with Boost for over a decade though.

> Personally, I agree with the chorus. After all, the point of a VCS is to
have a
> history of the code's evolution to a point. The VCS, be it SVN, Git,
whatever, is
> just a means to get that history. Jetissoning the ends for the means
seems
> misguided.

No, it's a cost of doing an upgrade. Those of you who ever migrated a large
CVS to SVN transition know what I mean: stuff gets lost, and actually it
isn't important enough to preserve that it requires a quadrupling of
transition effort when a read-only copy of the old technology repo is good
enough. Distributed SCM is much more dimorphic again, and you *have* to
accept some data loss.

Let me put this another way: those who want no history loss ought to be the
ones volunteering the additional time to preserve history. What actually
happens sadly is then the argument becomes we must stick with whatever the
old technology is, because people will choose a small reduction of
productivity every day over fixing tooling once and for all ("programmers
program, they don't do tooling"). I *still* find CVS in use in places
because "history must never ever be lost", and given the anti-productivity
nature of CVS that costs far more in present developer time than accepting a
5% or 10% history loss, most of which will never significantly matter anyway
because it occurs on the edges where SCMs dimorph.

> What happens in a few years time when Git is replaced with the next big
thing?
> Do we lose the history again? And then again when that gets replaced too?

That's exactly what happens. Bitrot is always inevitable in the long run.
Here call it "non-fatal bitrot" :)

My only red line is corruption of past and present source code releases. It
must *always* be possible to check out tag X and build release X. Other than
that, I'm flexible, including loss of branch integrity, because in the end
if that branch is really important its owner will manually fix up the
damage.

> > I fear that if modularization is taken to its logical extreme, you
> > could see submodules get out of step with other submodules. You may of
> > course have already anticipated this and have implemented something I
> haven't realized.
> > As I said, I am confused.
>
> Can you explain a bit more about what you mean by out-of-step? The whole
> point of modularising the code is to *help* modules to get out-of-step and
> therefore be easier to develop and test independent of what other Boost
> libraries are doing. But perhaps you mean something else?

Well, the Git way of helping stuff get out of step is that everyone gets
their own full copy of the whole repo, and their copy is just as important
as anyone else's copy. You clone, you develop and test, and optionally push
changes elsewhere which could be your friend, your team, your employer, or
of course some central authoritative copy. So I'm afraid I just don't get
the present design for a library as small and as tightly integrated as
Boost. Something huge and cleanly separated like KDE sure, but for Boost I
suspect it's overkill. Unless Boost plans to grow 10x in the next three
years that is.

> > [1]: By findable I mean that when Boost library users do #include
> > <boost/whatever> they get the main Boost repo version, not the
> > submodule version. I absolutely would expect an automated tool to pull
> > headers from submodules, check them for ABI breakage and push them
> > into the main repo. My point is that some sanity check ought to be
there.
>
> I'm not following why you would want to do this. Perhaps you can explain
what
> problem you are anticipating and how this would solve it?

Most of Boost is implemented in headers, very much unlike KDE or most other
C++ libraries. Moreover, those headers are quite brittle, unlike KDE or most
other C++ libraries. If broken into submodules, I can see an apparently
innocent change in submodule X appearing to compile and be okay in developer
X's set of submodule clones, but silently be a breaking change with a
simultaneous change in submodule Y by developer Y.

Why this matters is because when in git you go to push, git will force you
to merge before push and at that point you "see" the breakage by a conflict
appearing (hopefully) and if not then your immediate next compile will fail.
With the submodule approach that doesn't happen, so you *don't* see the
breakage till much later when the regression tests suddenly start failing.

I'm a great believer in refusing to let programmers commit code which breaks
other code rather than nagging them later to fix an earlier commit. The
point of failure notification ought to be as close to cause as possible.

> I also don't get what 'findable' means. What would a non-findable header
be?

Any internal header only findable by internal implementation. What I'm
basically suggesting is an approach where the master repo keeps a gold
candidate set of headers automatically extracted regularly from the
submodules. Then on push a hook can do appropriate black magic to force the
pusher to merge headers before the push. Then stuff which is broken appears
broken as soon as possible, rather than suddenly emerging many days later.

Does this make sense?

Alexander Lamaison

unread,
May 10, 2013, 2:05:40 PM5/10/13
to bo...@lists.boost.org
Niall Douglas <ndou...@blackberry.com> writes:

>> Niall Douglas <ndou...@blackberry.com> writes:
>>
>> > Me personally I'd just chuck away any unmappable historical information.
>> > Chalk it up to a cost of transition. If they really need the history,
>> > they can go fetch it from the read-only SVN repo.
>>
>> I see you've not been keeping up with the list lately ;) Daniel et. al.
>> suggested doing just that a few months ago and was met with such a chorus
> of
>> criticism, they didn't really have a choice but to fix it.

snip

>> Personally, I agree with the chorus. After all, the point of a VCS is to
> have a
>> history of the code's evolution to a point. The VCS, be it SVN, Git,
> whatever, is
>> just a means to get that history. Jetissoning the ends for the means
> seems
>> misguided.
>
> No, it's a cost of doing an upgrade. Those of you who ever migrated a large
> CVS to SVN transition know what I mean: stuff gets lost, and actually it
> isn't important enough to preserve that it requires a quadrupling of
> transition effort when a read-only copy of the old technology repo is good
> enough. Distributed SCM is much more dimorphic again, and you *have* to
> accept some data loss.

It's a question of degree. Daniel et. al.'s recent work shows that you
can keep that loss pretty low. "Where there's a will there's a way?"

snip
>> What happens in a few years time when Git is replaced with the next big
> thing?
>> Do we lose the history again? And then again when that gets replaced too?
>
> That's exactly what happens. Bitrot is always inevitable in the long run.
> Here call it "non-fatal bitrot" :)
>
> My only red line is corruption of past and present source code releases. It
> must *always* be possible to check out tag X and build release X.

So why have a VCS at all? Named tarballs meet your red line.

> Other than
> that, I'm flexible, including loss of branch integrity, because in the end
> if that branch is really important its owner will manually fix up the
> damage.
>
>> > I fear that if modularization is taken to its logical extreme, you
>> > could see submodules get out of step with other submodules. You may of
>> > course have already anticipated this and have implemented something I
>> haven't realized.
>> > As I said, I am confused.
>>
>> Can you explain a bit more about what you mean by out-of-step? The whole
>> point of modularising the code is to *help* modules to get out-of-step and
>> therefore be easier to develop and test independent of what other Boost
>> libraries are doing. But perhaps you mean something else?
>
> Well, the Git way of helping stuff get out of step is that everyone gets
> their own full copy of the whole repo, and their copy is just as important
> as anyone else's copy. You clone, you develop and test, and optionally push
> changes elsewhere which could be your friend, your team, your employer, or
> of course some central authoritative copy.

That works for a monolithic project. Boost is not monolithic but it's
single repo model was forcing it to act like one. As well as making
life a little easier for library developers, the modularisation effort
should help end users too. For a while now people have been wanting to
take only what they need from Boost and ignore the rest. It's
especially important in commercial environments where manager (rightly
or wrongly) are reluctant to depend on huge quantities of unreviewable
code with unknown interactions. Modularisation won't solve the problem
overnight but it helps.

> So I'm afraid I just don't get the present design for a library as
> small and as tightly integrated as Boost.

You must be talking about some other Boost.

> Something huge and cleanly separated like KDE sure, but for Boost I
> suspect it's overkill. Unless Boost plans to grow 10x in the next three
> years that is.
>
>> > [1]: By findable I mean that when Boost library users do #include
>> > <boost/whatever> they get the main Boost repo version, not the
>> > submodule version. I absolutely would expect an automated tool to pull
>> > headers from submodules, check them for ABI breakage and push them
>> > into the main repo. My point is that some sanity check ought to be
> there.
>>
>> I'm not following why you would want to do this. Perhaps you can explain
> what
>> problem you are anticipating and how this would solve it?
>
> Most of Boost is implemented in headers, very much unlike KDE or most other
> C++ libraries. Moreover, those headers are quite brittle, unlike KDE or most
> other C++ libraries. If broken into submodules, I can see an apparently
> innocent change in submodule X appearing to compile and be okay in developer
> X's set of submodule clones, but silently be a breaking change with a
> simultaneous change in submodule Y by developer Y.

I think the idea is that it allows you to mix your own
boost release by choosing the set of submodules to combine. The main
boostorg set is the official mix but not the only one. If the libraries
were just developed as clones on the boostorg repo, you would be forced
to choose the exact set of library version that one particular library
developed against.

> Why this matters is because when in git you go to push, git will force you
> to merge before push and at that point you "see" the breakage by a conflict
> appearing (hopefully) and if not then your immediate next compile will fail.
> With the submodule approach that doesn't happen, so you *don't* see the
> breakage till much later when the regression tests suddenly start
> failing.
>
> I'm a great believer in refusing to let programmers commit code which breaks
> other code rather than nagging them later to fix an earlier commit. The
> point of failure notification ought to be as close to cause as
> possible.

Surely the developer sees the breakage the moment they update the
versions of the other libraries they are working against. And then they
can fix it at their leisure while early adopters get to use their code
with the previous version of the conflicting library.

I can't find the details written down anywhere but I vaguely remember
there is a way the developers are supposed to signal that a particular
version is suitable for use with the master libraries or something like
that. It would be good to see this in black and white.

>> I also don't get what 'findable' means. What would a non-findable header
> be?
>
> Any internal header only findable by internal implementation. What I'm
> basically suggesting is an approach where the master repo keeps a gold
> candidate set of headers automatically extracted regularly from the
> submodules.

That's pretty much what happens. The boostorg repo will have the 'gold
candidate' of submodule hashes which are known to work together. I
don't get what 'extracting' the headers would add to that.

> Then on push a hook can do appropriate black magic to force the
> pusher to merge headers before the push. Then stuff which is broken appears
> broken as soon as possible, rather than suddenly emerging many days
> later.

When you are talking about breakages are you talking about merge
conflicts or compile/test-failures? Up till now I've assumed you mean
the latter. But now I'm thinking you might be asking what happens if
the developer of library X makes a change to header H and developer Y
also makes a change to _the same header_?

The way the modularisation works, I'm not even sure this is possible.
The libraries are meant to be self-contained so you only ever change
your own headers. It's a good question and one I hand't thought about.
Perhaps the people working on this can weigh in here.

Dave Abrahams

unread,
May 11, 2013, 1:39:10 AM5/11/13
to bo...@lists.boost.org

on Fri May 10 2013, Niall Douglas <ndouglas-AT-blackberry.com> wrote:

>> Niall Douglas <ndou...@blackberry.com> writes:
>>
>> > Me personally I'd just chuck away any unmappable historical information.
>> > Chalk it up to a cost of transition. If they really need the history,
>> > they can go fetch it from the read-only SVN repo.
>>
>> I see you've not been keeping up with the list lately ;) Daniel et. al.
>> suggested doing just that a few months ago and was met with such a chorus
> of
>> criticism, they didn't really have a choice but to fix it.
>
> Never actually was on this list, not ever, until recently as it's a lot of
> extra reading. Been involved with Boost for over a decade though.
>
>> Personally, I agree with the chorus. After all, the point of a VCS
>> is to have a history of the code's evolution to a point. The VCS, be
>> it SVN, Git, whatever, is just a means to get that history.
>> Jetissoning the ends for the means seems misguided.
>
> No, it's a cost of doing an upgrade. Those of you who ever migrated a large
> CVS to SVN transition know what I mean: stuff gets lost, and actually it
> isn't important enough to preserve that it requires a quadrupling of
> transition effort when a read-only copy of the old technology repo is good
> enough. Distributed SCM is much more dimorphic again, and you *have* to
> accept some data loss.

Nothing important need be lost. We're not losing any commits. SVN does
represent merge information as an exact set of contributing commits
rather than as a chain of history, and trying to preserve all that
exactly would be... dumb.

> Let me put this another way: those who want no history loss ought to be the
> ones volunteering the additional time to preserve history. What actually
> happens sadly is then the argument becomes we must stick with whatever the
> old technology is, because people will choose a small reduction of
> productivity every day over fixing tooling once and for all

I didn't want to do the job of modularizing history, but now that it's
close-to-finished, arguing for someone else to do it seems... pointless.

> My only red line is corruption of past and present source code releases. It
> must *always* be possible to check out tag X and build release X.

Which is why we're keeping a read-only copy of SVN. Modularized
repositories are never going to reassemble to form an exact picture of
history, because several files from the same SVN directory need to be
sorted into different repos.

--
Dave Abrahams

Jürgen Hunold

unread,
May 13, 2013, 4:28:04 AM5/13/13
to bo...@lists.boost.org
Hi Dave,

On Saturday, 4. May 2013 12:08:22 Dave Abrahams wrote:
> Hi All,
>
> After substantial work, including massive changes by me and Daniel
> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
> SVN commit in a Git repository.

Thanks for the work. Having done CVS -> svn myself, I really appreciate this.

> You can view the results at
>
> http://github.com/boostorg
> http://bitbucket.org/boostorg

I have some questions concerning the Boost.Build conversion.

The repository at

https://github.com/boostorg/build

contains a "buildv2" directory containing the Boost.Build .jam files and
"jamsrc" directory containing the c sources.

The original idee from Volodya was to completely remove the "v2" subdirectory
and put everything current in "tools/build/v2/" in the toplevel directory. I
would expect the contents of "tools/build/v2/" in the toplevel directory,
especially the c sources in their new location "engine".

I have scanned the conversion rules but can not find the relocation
information. Can you please take a look at this as this seems gone wrong
somehow?

The other idea would be to keep the "v2" directory for now and move everything
thing up vai git after the conversion.

Yours,

Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* voice: ++49 4257 300 ! Fährstraße 1
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany

Niall Douglas

unread,
May 13, 2013, 5:06:34 PM5/13/13
to bo...@lists.boost.org
> It's a question of degree. Daniel et. al.'s recent work shows that you
> can keep that loss pretty low. "Where there's a will there's a way?"

Agreed. With enough effort anything is possible. But tough on those tasked
with the grunt work.

> >> What happens in a few years time when Git is replaced with the next big
> > thing?
> >> Do we lose the history again? And then again when that gets replaced
too?
> >
> > That's exactly what happens. Bitrot is always inevitable in the long
run.
> > Here call it "non-fatal bitrot" :)
> >
> > My only red line is corruption of past and present source code releases.
It
> > must *always* be possible to check out tag X and build release X.
>
> So why have a VCS at all? Named tarballs meet your red line.

VCS is for workflow and collaboration. Optionally for blaming bugs through
automated regression analysis. They can be useful for generating some
interesting stats.

And yes, I do use named tarballs (or ZFS snapshots more recently) of the VCS
store exactly for history preservation. VCSs aren't for history
preservation, especially as they're usually not bitrot tolerant.

> > Well, the Git way of helping stuff get out of step is that everyone gets
> > their own full copy of the whole repo, and their copy is just as
important
> > as anyone else's copy. You clone, you develop and test, and optionally
push
> > changes elsewhere which could be your friend, your team, your employer,
or
> > of course some central authoritative copy.
>
> That works for a monolithic project. Boost is not monolithic but it's
> single repo model was forcing it to act like one. As well as making
> life a little easier for library developers, the modularisation effort
> should help end users too. For a while now people have been wanting to
> take only what they need from Boost and ignore the rest. It's
> especially important in commercial environments where manager (rightly
> or wrongly) are reluctant to depend on huge quantities of unreviewable
> code with unknown interactions. Modularisation won't solve the problem
> overnight but it helps.

I disagree with pretty much all of this. And I think modularization, as
currently proposed if I understand it correctly, actually makes those feared
interactions much worse and/or much more work to avoid.

> > So I'm afraid I just don't get the present design for a library as
> > small and as tightly integrated as Boost.
>
> You must be talking about some other Boost.

It's all subjective. I agree there are a chunk of Boost libraries which are
fairly standalone and I have no problem with those being turned into git
submodules.

> > I'm a great believer in refusing to let programmers commit code which
breaks
> > other code rather than nagging them later to fix an earlier commit. The
> > point of failure notification ought to be as close to cause as
> > possible.
>
> Surely the developer sees the breakage the moment they update the
> versions of the other libraries they are working against. And then they
> can fix it at their leisure while early adopters get to use their code
> with the previous version of the conflicting library.

My point is this isn't automatic. It requires programmers not to be lazy.
That hasn't been hugely successful in the past.

> When you are talking about breakages are you talking about merge
> conflicts or compile/test-failures? Up till now I've assumed you mean
> the latter. But now I'm thinking you might be asking what happens if
> the developer of library X makes a change to header H and developer Y
> also makes a change to _the same header_?

No that's easily detectable. My concern is when modifications to submodule A
and submodule B cause arbitrary C in some submodule D to suddenly fail.

> The way the modularisation works, I'm not even sure this is possible.
> The libraries are meant to be self-contained so you only ever change
> your own headers. It's a good question and one I hand't thought about.
> Perhaps the people working on this can weigh in here.

Git submodules are too loosely coupled for anything but loosely coupled,
fairly standalone libraries.

Daniel Pfeifer

unread,
May 14, 2013, 11:13:00 AM5/14/13
to Boost Developers List
2013/5/13 Jürgen Hunold <jhu...@gmx.eu>

> Hi Dave,
>
> On Saturday, 4. May 2013 12:08:22 Dave Abrahams wrote:
> > Hi All,
> >
> > After substantial work, including massive changes by me and Daniel
> > Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
> > SVN commit in a Git repository.
>
> Thanks for the work. Having done CVS -> svn myself, I really appreciate
> this.
>
> > You can view the results at
> >
> > http://github.com/boostorg
> > http://bitbucket.org/boostorg
>
> I have some questions concerning the Boost.Build conversion.
>
> The repository at
>
> https://github.com/boostorg/build
>
> contains a "buildv2" directory containing the Boost.Build .jam files and
> "jamsrc" directory containing the c sources.
>

It seems the path seperators got lost. I think I fixed that problem, the
conversion is currently rerunning.

The original idee from Volodya was to completely remove the "v2"
> subdirectory
> and put everything current in "tools/build/v2/" in the toplevel directory.
> I
> would expect the contents of "tools/build/v2/" in the toplevel directory,
> especially the c sources in their new location "engine".
>

Yes, I remember.


> I have scanned the conversion rules but can not find the relocation
> information. Can you please take a look at this as this seems gone wrong
> somehow?
>

Probably. We wanted to make sure that no commit is lost.


> The other idea would be to keep the "v2" directory for now and move
> everything
> thing up vai git after the conversion.
>

Another option would be to create a second repository "build2".

Andrey Semashev

unread,
May 15, 2013, 3:59:28 PM5/15/13
to bo...@lists.boost.org
On Sat, May 4, 2013 at 11:08 PM, Dave Abrahams <da...@boostpro.com> wrote:

>
> Daniel and I are ready to accept your feedback on the results of
> modularization, and especially your pull requests containing edits to
> the ruleset. I will the steering committee to establish a formal review
> period, though.
>

I'm not sure if this is a problem and should be fixed, but Boost.Log
currently uses some configuration projects that reside in Boost.Context
(while Boost.Context itself is not used). Basically,
libs/log/build/log-architecture.jam uses /boost/architecture//* projects
that happen to reside in libs/context/config. I know, hijacking configure
projects like that may not be a good thing but it seemed better than
duplicating them in Boost.Log. Also, /boost/architecture//* doesn't imply
any relation to Boost.Context in the first place. Additionally, I have a
few configure projects of my own in libs/log/config that might be useful
for other libraries.

So, what I'm asking is:

1. Will this work after modulatization? Does it mean that Boost.Log will
not compile without Boost.Context?
2. Maybe it's worth to move all such configure projects to some common
place accessible to all libraries? What that place could be?

Vladimir Prus

unread,
May 17, 2013, 11:07:30 AM5/17/13
to bo...@lists.boost.org
On 14.05.2013 19:13, Daniel Pfeifer wrote:
> 2013/5/13 Jürgen Hunold <jhu...@gmx.eu>
>
>> Hi Dave,
>>
>> On Saturday, 4. May 2013 12:08:22 Dave Abrahams wrote:
>>> Hi All,
>>>
>>> After substantial work, including massive changes by me and Daniel
>>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>>> SVN commit in a Git repository.
>>
>> Thanks for the work. Having done CVS -> svn myself, I really appreciate
>> this.
>>
>>> You can view the results at
>>>
>>> http://github.com/boostorg
>>> http://bitbucket.org/boostorg
>>
>> I have some questions concerning the Boost.Build conversion.
>>
>> The repository at
>>
>> https://github.com/boostorg/build
>>
>> contains a "buildv2" directory containing the Boost.Build .jam files and
>> "jamsrc" directory containing the c sources.
>>
>
> It seems the path seperators got lost. I think I fixed that problem, the
> conversion is currently rerunning.

It looks better now, but the root of the repo has 'build', which has 'v2',
which has actual content. Can we actually have content of 'v2' at the top
level?


- Volodya

Paul A. Bristow

unread,
May 17, 2013, 1:00:59 PM5/17/13
to bo...@lists.boost.org

> -----Original Message-----
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Vladimir Prus
> Sent: Friday, May 17, 2013 4:08 PM
> To: bo...@lists.boost.org
> Subject: Re: [boost] Git Modularization Ready for Review
>
> On 14.05.2013 19:13, Daniel Pfeifer wrote:
> > 2013/5/13 Jürgen Hunold <jhu...@gmx.eu>
> >
> >> Hi Dave,
> >>
> >> On Saturday, 4. May 2013 12:08:22 Dave Abrahams wrote:
> >>> Hi All,
> >>>
> >>> After substantial work, including massive changes by me and Daniel
> >>> Pfeifer to KDE's svn->Git conversion tool, we have captured every
> >>> Boost SVN commit in a Git repository.
> >>
> >> Thanks for the work. Having done CVS -> svn myself, I really
> >> appreciate this.
> >>
> >>> You can view the results at
> >>>
> >>> http://github.com/boostorg

I've looked at https://github.com/boostorg/math/tree/master/doc - where we have some recent commits from John and me.

Until I tried the history button they appear in alphabetical folder order which isn't so useful.

They seem to be there OK .

This sounds quite an achievement.

Keep up the Good Work

Paul

PS I note that (purely out of conceit of course ;-)

I clicked on my name and it shows my Open Source contributions - but only those outside Boost.Math - a miserable 6 in total, and no reference to Boost.Math.

That is a bit disappointing as I was hoping it would show I was a Jolly Good Chap!
But that's trivia...


---
Paul A. Bristow,
Prizet Farmhouse, Kendal LA8 8AB UK
+44 1539 561830 07714330204
pbri...@hetp.u-net.com

Jonathan Wakely

unread,
May 17, 2013, 1:11:17 PM5/17/13
to bo...@lists.boost.org
On 17 May 2013 18:00, Paul A. Bristow wrote:
>
>
> I've looked at https://github.com/boostorg/math/tree/master/doc - where we have some recent commits from John and me.
>
> Until I tried the history button they appear in alphabetical folder order which isn't so useful.

That's the Files view, showing the directory contents not the commits,
so it's showing the files in alphabetical order. It happens to show
the most recent commit to each file/dir on the right, but it's not the
best way to look at commits. Either use the history button or switch
to the Commits tab, above.

Paul A. Bristow

unread,
May 17, 2013, 1:19:05 PM5/17/13
to bo...@lists.boost.org
> -----Original Message-----
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Jonathan Wakely
> Sent: Friday, May 17, 2013 6:11 PM
> To: bo...@lists.boost.org
> Subject: Re: [boost] Git Modularization Ready for Review
>
> On 17 May 2013 18:00, Paul A. Bristow wrote:
> >
> >
> > I've looked at https://github.com/boostorg/math/tree/master/doc - where we have some recent
> commits from John and me.
> >
> > Until I tried the history button they appear in alphabetical folder order which isn't so useful.
>
> That's the Files view, showing the directory contents not the commits, so it's showing the files
in
> alphabetical order. It happens to show the most recent commit to each file/dir on the right, but
it's not
> the best way to look at commits. Either use the history button or switch to the Commits tab,
above.

Thanks.

As you will gather, I'm a GIT newbie in fumble mode :-(

Paul

---
Paul A. Bristow,
Prizet Farmhouse, Kendal LA8 8AB UK
+44 1539 561830 07714330204
pbri...@hetp.u-net.com






Daniel Pfeifer

unread,
May 17, 2013, 4:49:15 PM5/17/13
to Boost Developers List
2013/5/17 Vladimir Prus <gh...@cs.msu.su>

> On 14.05.2013 19:13, Daniel Pfeifer wrote:
>
>> 2013/5/13 Jürgen Hunold <jhu...@gmx.eu>
>>
>> Hi Dave,
>>>
>>> On Saturday, 4. May 2013 12:08:22 Dave Abrahams wrote:
>>>
>>>> Hi All,
>>>>
>>>> After substantial work, including massive changes by me and Daniel
>>>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>>>> SVN commit in a Git repository.
>>>>
>>>
>>> Thanks for the work. Having done CVS -> svn myself, I really appreciate
>>> this.
>>>
>>> You can view the results at
>>>>
>>>> http://github.com/boostorg
>>>> http://bitbucket.org/boostorg
>>>>
>>>
>>> I have some questions concerning the Boost.Build conversion.
>>>
>>> The repository at
>>>

>>> https://github.com/boostorg/**build <https://github.com/boostorg/build>


>>>
>>> contains a "buildv2" directory containing the Boost.Build .jam files and
>>> "jamsrc" directory containing the c sources.
>>>
>>>
>> It seems the path seperators got lost. I think I fixed that problem, the
>> conversion is currently rerunning.
>>
>
> It looks better now, but the root of the repo has 'build', which has 'v2',
> which has actual content. Can we actually have content of 'v2' at the top
> level?
>

The current layout reflects what is in subversion. Both 'build' and 'jam'
from the directory 'tools' are placed in the repository 'build'.
'jam' has been deleted somewhere in the history, so only 'build' remains.

If we simply use 'tools/build/v2' as the top level of the repository
'build', where shall we place the parts of Boost.Build's history that does
not match that path?

Vladimir Prus

unread,
May 18, 2013, 2:57:51 AM5/18/13
to bo...@lists.boost.org

I would be fine with /dev/null, to be honest, unless Rene and Steven have any
concerns. That is part of the history that is either not used at all, or has been
rewritten over and over, so it won't be much useful in future.

- Volodya

Dave Abrahams

unread,
May 20, 2013, 1:58:33 AM5/20/13
to bo...@lists.boost.org

on Fri May 17 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:

> On 14.05.2013 19:13, Daniel Pfeifer wrote:
>> 2013/5/13 Jürgen Hunold <jhu...@gmx.eu>
>>
>>> Hi Dave,
>>>
>>> On Saturday, 4. May 2013 12:08:22 Dave Abrahams wrote:
>
>>>> Hi All,
>>>>
>>>> After substantial work, including massive changes by me and Daniel
>>>> Pfeifer to KDE's svn->Git conversion tool, we have captured every Boost
>>>> SVN commit in a Git repository.
>>>
>>> Thanks for the work. Having done CVS -> svn myself, I really appreciate
>>> this.
>>>
>>>> You can view the results at
>>>>
>>>> http://github.com/boostorg
>>>> http://bitbucket.org/boostorg
>>>
>>> I have some questions concerning the Boost.Build conversion.
>>>
>>> The repository at
>>>
>>> https://github.com/boostorg/build
>>>
>>> contains a "buildv2" directory containing the Boost.Build .jam files and
>>> "jamsrc" directory containing the c sources.
>>>
>>
>> It seems the path seperators got lost. I think I fixed that problem, the
>> conversion is currently rerunning.
>
> It looks better now, but the root of the repo has 'build', which has 'v2',
> which has actual content. Can we actually have content of 'v2' at the top
> level?

We could, but that leaves open some questions:

1. What does history look like? At some point, there's the v1 code at
the top level. Decide how you want those branches and commits
distributed, and we can set it up.

2. Will this break scripts and other tools that invoke bjam?

--
Dave Abrahams

Dave Abrahams

unread,
May 20, 2013, 2:08:56 AM5/20/13
to bo...@lists.boost.org

on Fri May 17 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:

You're free to delete any branches you want after the conversion is
over (it's really easy), but until then, we want to make sure every
commit is accounted for. Any commits that aren't caught by the ruleset
end up in https://github.com/boostorg/svn2git-fallback and the build at
http://jenkins.boost.org fails.

You can make an arbitrarily obscure branch name
(e.g. old-branches/deleteme) and send unwanted history there, if you
like. Please try to submit a pull request with edits to
repositories.txt. An explanation of the semantics is at:
https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt

--
Dave Abrahams

Vladimir Prus

unread,
May 21, 2013, 11:40:44 AM5/21/13
to bo...@lists.boost.org, Dave Abrahams
On 20.05.2013 10:08, Dave Abrahams wrote:

>> I would be fine with /dev/null, to be honest, unless Rene and Steven have any
>> concerns. That is part of the history that is either not used at all, or has been
>> rewritten over and over, so it won't be much useful in future.
>
> You're free to delete any branches you want after the conversion is
> over (it's really easy), but until then, we want to make sure every
> commit is accounted for. Any commits that aren't caught by the ruleset
> end up in https://github.com/boostorg/svn2git-fallback and the build at
> http://jenkins.boost.org fails.
>
> You can make an arbitrarily obscure branch name
> (e.g. old-branches/deleteme) and send unwanted history there, if you
> like. Please try to submit a pull request with edits to
> repositories.txt. An explanation of the semantics is at:
> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt

Dave,

I have spent around 30 minutes trying to grok this format, and I cannot
quite figure how how to sent tools/build/v2 directory to master branch
of "build" repo, while sending everything else from trunk/tools/build
into some other branch.

If the change is easy, as you make it sound, can either you or Daniel make it?

Thanks,
Volodya

Dave Abrahams

unread,
May 22, 2013, 1:26:01 AM5/22/13
to Vladimir Prus, bo...@lists.boost.org, Daniel Pfeifer

on Tue May 21 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:

> On 20.05.2013 10:08, Dave Abrahams wrote:
>
>>> I would be fine with /dev/null, to be honest, unless Rene and Steven have any
>>> concerns. That is part of the history that is either not used at all, or has been
>>> rewritten over and over, so it won't be much useful in future.
>>
>> You're free to delete any branches you want after the conversion is
>> over (it's really easy), but until then, we want to make sure every
>> commit is accounted for. Any commits that aren't caught by the ruleset
>> end up in https://github.com/boostorg/svn2git-fallback and the build at
>> http://jenkins.boost.org fails.
>>
>> You can make an arbitrarily obscure branch name
>> (e.g. old-branches/deleteme) and send unwanted history there, if you
>> like. Please try to submit a pull request with edits to
>> repositories.txt. An explanation of the semantics is at:
>> https://github.com/ryppl/Boost2Git/wiki/Editing-repositories.txt
>
> Dave,
>
> I have spent around 30 minutes trying to grok this format, and I cannot
> quite figure how how to sent tools/build/v2 directory to master branch
> of "build" repo, while sending everything else from trunk/tools/build
> into some other branch.

That doesn't sound like a very accurate reflection of history. There
used to be code in tools/build/, directly. What "other branch" should
that go into, and at what path would you like it to be placed?

And there are lots of changes outside trunk/ that should get similar
treatment, whatever you decide to do. See:

https://github.com/boostorg/build/branches
https://github.com/boostorg/build/tags

One possibility is that you keep everything under tools/build (other
than tools/build/v2) in the branches where it currently resides, but put
it in a subdirectory called v1/ rather than at the top level. However,
there's probably a point in history where that stuff would *belong* at the
top level because, e.g., v2 didn't exist.

> If the change is easy, as you make it sound, can either you or Daniel
> make it?

We could... but I am a little concerned about making changes to history
that diverge too far from reality. Remember, you are free to move
things around and rename branches after modularization is
complete... but I'm not sure we should do too much revision of the
actual history. Thoughts?

--
Dave Abrahams

Andrey Semashev

unread,
May 22, 2013, 2:29:01 AM5/22/13
to bo...@lists.boost.org
Ping? Any opinions on this?
Reply all
Reply to author
Forward
0 new messages