[boost] Git Modularization Review no vote heads-up
Beman Dawes
today, I've vote no since I think that we aren't yet ready. Since my
concerns are apparently easy to fix technically, I'm mentioning them
here to give Dave and Daniel a chance to address them before the
actual Git Modularization Review starts.
Concerns:
1) The mapping of svn branch names to modular git branch names needs
to be revised: Svn "trunk" needs to become modular git "develop", not
"master". Modular git needs to have a branch "master" that represents
the latest stable release. Whether the content is identical to the
last boost release or to branches/release at point of conversion needs
to be decided, as does what the history, if any, of this branch looks
like.
2) The procedures described in
https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated,
the dependency on CMake needs to be removed, and the procedures need
to work.
These are blocking issues because they prevent development and testing
of modular boost testing procedures, developers procedures, release
procedures, installation procedures, and documentation. Until they are
resolved, the entire modular boost conversion rests on the shoulders
of Dave and Daniel. Once they are resolved, others can pitch in and
help since from that point forward we will be in a pure git
environment and detailed knowledge of the conversion process from svn
is not required.
--Beman
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Bjørn Roald
> 2) The procedures described in
> https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated,
> the dependency on CMake needs to be removed, and the procedures need
> to work.
I tried to check this out, I think Steven Watanabe did some test with a
modification to b2 to automate this. So my intention was to look for it
and possibly try it or ping Steven.
However trying the steps posted on the wiki seems to fail on the
git submodule update --init
step.
There is no .gitmodules file in the top directory modular-boost master
branch HEAD, so I guess there is no way git submodule know what and
where to fetch from. It silently completes as there is nothing to do.
There must be something that is disconnected here.
--
Bjørn Roald
Beman Dawes
> On Thu, 2013-05-09 at 07:43 -0400, Beman Dawes wrote:
>> 2) The procedures described in
>> https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated,
>> the dependency on CMake needs to be removed, and the procedures need
>> to work.
>
> I tried to check this out, I think Steven Watanabe did some test with a
> modification to b2 to automate this. So my intention was to look for it
> and possibly try it or ping Steven.
>
> However trying the steps posted on the wiki seems to fail on the
>
> git submodule update --init
>
> step.
>
> There is no .gitmodules file in the top directory modular-boost master
> branch HEAD, so I guess there is no way git submodule know what and
> where to fetch from. It silently completes as there is nothing to do.
> There must be something that is disconnected here.
individual libraries is now working nicely with full history for all
branches, but the modular-boost master hasn't been handling submodules
for quite a while now.
--Beman
Nasos Iliopoulos
It seems to me also that there should be some more iterations or some
more relevant documentation.
I would feel more comfortable using submodules, but without them my best
attempt up to now is (ublas as an example here):
git clone https://github.com/boostorg/boost modular-boost
cd modular-boost
{manually relpace /v2 with /buildv2 in boost-build.jam and bootstrap.sh}
cd tools
git clone https://github.com/boostorg/build
cd ..
./bootstrap.sh
cd libs/numeric
git clone https://github.com/boostorg/ublas
cd ../../
./b2 include=libs/numeric/ublas/include libs/numeric/ublas/bench1
(to compile bench1 for example)
This relies on the assumption that the rest of boost is in an accessible
location in the system.
-Nasos
Dave Abrahams
I personally find the subject line of this posting needlessly alarming.
on Thu May 09 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
> If the steering committee's Git Modularization Review vote
that word. The intention was to give people a chance to make
corrections to the way things were being modularized so we could move on
to the next step. https://github.com/ryppl/Boost2Git/wiki#vetting-period
> were held today, I've vote no since I think that we aren't yet
> ready. Since my concerns are apparently easy to fix technically, I'm
> mentioning them here to give Dave and Daniel a chance to address them
> before the actual Git Modularization Review starts.
>
> Concerns:
>
> 1) The mapping of svn branch names to modular git branch names needs
> to be revised: Svn "trunk" needs to become modular git "develop", not
> "master". Modular git needs to have a branch "master" that represents
> the latest stable release. Whether the content is identical to the
> last boost release or to branches/release at point of conversion needs
> to be decided, as does what the history, if any, of this branch looks
> like.
avoid the threat of a "no vote"
> 2) The procedures described in
> https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated,
> the dependency on CMake needs to be removed, and the procedures need
> to work.
You wrote the web page; are you going to update it?
https://svn.boost.org/trac/boost/wiki/TryModBoost?action=history
> These are blocking issues because they prevent development and testing
> of modular boost testing procedures, developers procedures, release
> procedures, installation procedures, and documentation. Until they are
> resolved, the entire modular boost conversion rests on the shoulders
> of Dave and Daniel. Once they are resolved, others can pitch in and
> help since from that point forward we will be in a pure git
> environment and detailed knowledge of the conversion process from svn
> is not required.
not asking people to approve an immediate switchover; we're just asking
for people to make fixes and requests regarding how things are sorted
into modules.
--
Dave Abrahams
Beman Dawes
>
> I personally find the subject line of this posting needlessly alarming.
going into a review soon with the critical "develop" and "master"
branches still not having the right names, and the
https://github.com/boostorg/boost super-project for modular boost not
usable for evaluation (or at least "git submodule update --init"
apparently no longer working).
> on Thu May 09 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
>
>> If the steering committee's Git Modularization Review vote
>
> What vote? We asked for a review period; maybe I shouldn't have used
> that word.
vote from the review submitter.
> The intention was to give people a chance to make
> corrections to the way things were being modularized so we could move on
> to the next step. https://github.com/ryppl/Boost2Git/wiki#vetting-period
>
>> were held today, I've vote no since I think that we aren't yet
>> ready. Since my concerns are apparently easy to fix technically, I'm
>> mentioning them here to give Dave and Daniel a chance to address them
>> before the actual Git Modularization Review starts.
>>
>> Concerns:
>>
>> 1) The mapping of svn branch names to modular git branch names needs
>> to be revised: Svn "trunk" needs to become modular git "develop", not
>> "master". Modular git needs to have a branch "master" that represents
>> the latest stable release. Whether the content is identical to the
>> last boost release or to branches/release at point of conversion needs
>> to be decided, as does what the history, if any, of this branch looks
>> like.
>
> Decided by whom? If it's up to us, we'll do whatever is expedient to
> avoid the threat of a "no vote"
First, "Whether the content [of Master] is identical to the last boost
candidate is build. That becomes the point in time of conversion to
git. IOW, the last boost release == branches/release at point of
conversion to git, so the question becomes moot. Can we reach a
consensus on that, and if necessary get it ratified by the Steering
Committee?
Second, "What does the history, if any, of master look like?"
I've heard three possibilities:
1) History based on the difference between release tags.
2) History of the current svn branches/release.
3) No history at all. (The tags are still available, so if anyone
cares they diff the differences.)
(1) or (2) are my preference, but (3) is also acceptable. I wouldn't
want the choice to delay the schedule. My feeling is that you should
make the call, after consulting with Daniel. You have a broad overview
of the needs of the community and also understand the history
mechanism much better than most of us.
>
>> 2) The procedures described in
>> https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated,
>> the dependency on CMake needs to be removed, and the procedures need
>> to work.
>
> Where did these prerequisites come from?
https://svn.boost.org/trac/boost/wiki/TryModBoost.
> You wrote the web page; are you going to update it?
> https://svn.boost.org/trac/boost/wiki/TryModBoost?action=history
working, and not knowing what the replacement is for the "cmake -P
forward_headers.cmake" step.
>> These are blocking issues because they prevent development and testing
>> of modular boost testing procedures, developers procedures, release
>> procedures, installation procedures, and documentation. Until they are
>> resolved, the entire modular boost conversion rests on the shoulders
>> of Dave and Daniel. Once they are resolved, others can pitch in and
>> help since from that point forward we will be in a pure git
>> environment and detailed knowledge of the conversion process from svn
>> is not required.
>
> I understand that these are blocking issues for the switchover. We're
> not asking people to approve an immediate switchover; we're just asking
> for people to make fixes and requests regarding how things are sorted
> into modules.
are also blocking issues for making progress on documentation and
developing test and release procedures.
That said, the separation of libraries into separate modules with full
history is looking very much improved!
--Beman
Dave Abrahams
on Sun May 12 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
> On Fri, May 10, 2013 at 9:52 AM, Dave Abrahams <da...@boostpro.com> wrote:
>>
>> I personally find the subject line of this posting needlessly alarming.
>
> From my viewpoint, it seems at least somewhat alarming that we are
> going into a review soon with the critical "develop" and "master"
> branches still not having the right names,
It's clear that you are somewhat alarmed, yes.
The "review" that we're asking for is there in part so that the
community can decide what "the right names" are. As Daniel pointed out
quite nicely I think, there is no single obvious right answer. That
said, if you think the right answer is obvious, we can make the
necessary change at a moment's notice. Just tell us what you want.
Anything we do is easily reversed.
> and the https://github.com/boostorg/boost super-project for modular
> boost not usable for evaluation (or at least "git submodule update
> --init" apparently no longer working).
That's because when people asked for modularized history, what we were
doing previously—just creating a snapshot of the submodules where they
are, going forward—was no longer the correct answer. I could easily
provide you with a 10-line bash script that checks the head of all the
submodules out at their proper locations, but getting them to be
correctly represented though the entire history of Boost is requiring a
lot of coding (I think I'm very close, though).
>> on Thu May 09 2013, Beman Dawes <bdawes-AT-acm.org> wrote:
>>
>>> If the steering committee's Git Modularization Review vote
>>
>> What vote? We asked for a review period; maybe I shouldn't have used
>> that word.
>
> The reviews usually end with a Yes, No, or perhaps Conditionally Yes
> vote from the review submitter.
Yes, "review" was the wrong word; I see that now.
I don't understand how this relates to what we're asking people to
evaluate. We want to know if they're happy with where the files are
going in the Git repos and what branches and/or tags they're being
directed to. If people aren't happy, we want to make adjustments.
A perfect example of the kind of information we need is in
http://boost.2283326.n4.nabble.com/Git-Modularization-Ready-for-Review-tp4646659p4646739.html
where Vicente Botet wrote,
| Could I move the following from repository thread to core?
|
| "boost/detail/atomic_redef_macros.hpp" :
| "include/boost/detail/atomic_redef_macros.hpp";
| "boost/detail/atomic_undef_macros.hpp" :
| "include/boost/detail/atomic_undef_macros.hpp"; I added these
| files. They are used now by Boost.Thread, but it should be used by
| SmartPtr (See https://svn.boost.org/trac/boost/ticket/6842 and
| https://svn.boost.org/trac/boost/ticket/6843).
> Second, "What does the history, if any, of master look like?"
Wait; don't we care about the history of any of the other branches?
Daniel and I have spent months getting that right because the community
insisted on it.
> I've heard three possibilities:
>
> 1) History based on the difference between release tags.
> 2) History of the current svn branches/release.
> 3) No history at all. (The tags are still available, so if anyone
> cares they diff the differences.)
>
> (1) or (2) are my preference,
2 is trivially accomplished. 1 would take more work.
> but (3) is also acceptable. I wouldn't want the choice to delay the
> schedule. My feeling is that you should make the call, after
> consulting with Daniel. You have a broad overview of the needs of the
> community and also understand the history mechanism much better than
> most of us.
Well, I don't personally think history matters that much, and I think
having a history that matches git-flow's usage matters even less, since
after all we haven't actually been using git-flow, so it would be
artificial. It would be nice to do a "perfect" job, but I'm not sure
it's worth the effort.
>>> 2) The procedures described in
>>> https://svn.boost.org/trac/boost/wiki/TryModBoost need to be updated,
>>> the dependency on CMake needs to be removed, and the procedures need
>>> to work.
>>
>> Where did these prerequisites come from?
>
> By looking at the prerequisites for each of the command line steps in
> https://svn.boost.org/trac/boost/wiki/TryModBoost.
>
>> You wrote the web page; are you going to update it?
>> https://svn.boost.org/trac/boost/wiki/TryModBoost?action=history
>
> I'm being blocked by the "git submodule update --init" step not
> working, and not knowing what the replacement is for the "cmake -P
> forward_headers.cmake" step.
These things are both important for making the transition, but they're
not important to the particular review we were asking for.
>>> These are blocking issues because they prevent development and testing
>>> of modular boost testing procedures, developers procedures, release
>>> procedures, installation procedures, and documentation. Until they are
>>> resolved, the entire modular boost conversion rests on the shoulders
>>> of Dave and Daniel. Once they are resolved, others can pitch in and
>>> help since from that point forward we will be in a pure git
>>> environment and detailed knowledge of the conversion process from svn
>>> is not required.
>>
>> I understand that these are blocking issues for the switchover. We're
>> not asking people to approve an immediate switchover; we're just asking
>> for people to make fixes and requests regarding how things are sorted
>> into modules.
>
> It isn't just that they are blocking issues for the switchover. They
> are also blocking issues for making progress on documentation and
> developing test and release procedures.
Sure. I'm working hard on it.
> That said, the separation of libraries into separate modules with full
> history is looking very much improved!
Well, I know it isn't glamorous and it doesn't satisfy the need to say "look
it builds," but *that* is what we're asking people to evaluate now.
--
Dave Abrahams
Vladimir Prus
>> That said, the separation of libraries into separate modules with full
>> history is looking very much improved!
>
> Well, I know it isn't glamorous and it doesn't satisfy the need to say "look
> it builds," but *that* is what we're asking people to evaluate now.
So what are the plans for "look it builds"? At the very least, I would prefer that
requirement to use cmake to put together symlinks be removed. What's the plan for that?
Jürgen has posted a patch that teaches Boost.Build to do that, and I can give it
a try, but before that I think cloning of the master repository should work (others
already reported some issue with chrono) and at least Boost.Build should have its
final layout.
Also, what would be the process for tweaking any infrastructure code? Commit to
SVN and wait until git mirror is updated? Or create a special branch/repo in git
and do changes there, rebasing against new versions?
Thanks,
Volodya
Dave Abrahams
on Sat May 18 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:
> On 13.05.2013 09:00, Dave Abrahams wrote:
>
>>> That said, the separation of libraries into separate modules with full
>>> history is looking very much improved!
>>
>> Well, I know it isn't glamorous and it doesn't satisfy the need to say "look
>> it builds," but *that* is what we're asking people to evaluate now.
>
> So what are the plans for "look it builds"? At the very least, I would prefer that
> requirement to use cmake to put together symlinks be removed. What's
> the plan for that?
Steven Watanabe is working on that. It looks like Jürgen Hunold and
Eric Cornelius may have picked up where he left off? Jürgen has asked
Eric to post about this on the boost-build list, so I presume you've
seen some of it.
> Jürgen has posted a patch that teaches Boost.Build to do that, and I can give it
> a try, but before that I think cloning of the master repository should work
It does work.
> (others already reported some issue with chrono)
I don't know what's being referred to here.
> and at least Boost.Build should have its final layout.
I think maybe I disagree. If you want to rearrange Boost.Build in SVN
before the transition, then be my guest. Modularization will capture
those changes and you'll have a corresponding layout in Git. However, I
don't think it's reasonable to build a fake *history* in Git that
doesn't reflect the way things went in SVN. That would make those
running the Git transition responsible for any adjustments required for
your preferred layout. Make sense?
> Also, what would be the process for tweaking any infrastructure code?
> Commit to SVN and wait until git mirror is updated?
That would be my preference. The mirror updates fairly quickly. If
rules or modularization code doesn't change, it's very fast (about 3.5
minutes). Otherwise, it takes about an hour, but we're not pushing
ruleset or modularization code changes very often.
> Or create a special branch/repo in git and do changes there, rebasing
> against new versions?
The process essentially clobbers and reconstructs the git repos
continuously, so if you want to work in Git using the modularized repos,
expect to have to rebase against new branch heads, and don't expect to
be able to push to the github repos.
--
Dave Abrahams
Daniel Pfeifer
> on Sat May 18 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:
>
> > and at least Boost.Build should have its final layout.
> I think maybe I disagree.
Currently, we direct both "tools/build" and "tools/jam" to the "build"
repository, then we want that repository to be a submodule in boost at the
location "build".
Hence, the directory "tools/build/v2" will appear as "tools/build/build/v2"
which is not the same layout as in SVN.
We could either use "tools/build" or "tools/build/v2" the root of
repository "build". But we need to put the submodule at that exact location.
Everything that is not matched (eg. jam/, index.html), can remain in the
meta-repository "boost".
Which one do you prefer?
cheers, Daniel
Jürgen Hunold
On Monday, 20. May 2013 10:48:20 Daniel Pfeifer wrote:
> 2013/5/20 Dave Abrahams <da...@boostpro.com>
>
> > on Sat May 18 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:
> > > and at least Boost.Build should have its final layout.
> >
> > I think maybe I disagree.
>
> We definetly need to make some changes to the Boost.Build layout.
> Currently, we direct both "tools/build" and "tools/jam" to the "build"
> repository, then we want that repository to be a submodule in boost at the
> location "build".
> Hence, the directory "tools/build/v2" will appear as "tools/build/build/v2"
> which is not the same layout as in SVN.
It is not in the current svn layout anymore. Seems to be some v1 relict. Can
you investigate this?
> We could either use "tools/build" or "tools/build/v2" the root of
> repository "build". But we need to put the submodule at that exact location.
> Everything that is not matched (eg. jam/, index.html), can remain in the
> meta-repository "boost".
wonder if we should do the final move after the transition to git. Or just move
the files _now_ and adjust the toplevel scripts before the transition.
> Which one do you prefer?
The main culprit is that I don't even have a "tools/build" submodule at the
moment.
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* voice: ++49 4257 300 ! Fährstraße 1
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany
Bjørn Roald
> HI Daniel.
>
> On Monday, 20. May 2013 10:48:20 Daniel Pfeifer wrote:
> > 2013/5/20 Dave Abrahams <da...@boostpro.com>
> >
> > > on Sat May 18 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:
> > > > and at least Boost.Build should have its final layout.
> > >
> > > I think maybe I disagree.
> >
> > We definetly need to make some changes to the Boost.Build layout.
>
> The question is, when to do them.
do them before or after conversion to git, not as part of it.
> > Currently, we direct both "tools/build" and "tools/jam" to the "build"
> > repository, then we want that repository to be a submodule in boost at the
> > location "build".
> > Hence, the directory "tools/build/v2" will appear as "tools/build/build/v2"
> > which is not the same layout as in SVN.
>
> Yes, this is clearly not what we want. But where does "tools/jam" come from?
> It is not in the current svn layout anymore. Seems to be some v1 relict. Can
> you investigate this?
>
> > We could either use "tools/build" or "tools/build/v2" the root of
> > repository "build". But we need to put the submodule at that exact location.
> > Everything that is not matched (eg. jam/, index.html), can remain in the
> > meta-repository "boost".
>
> We wanted "tools/build" to contain the current "tools/build/v2" of svn. I
> wonder if we should do the final move after the transition to git. Or just move
> the files _now_ and adjust the toplevel scripts before the transition.
do you have a plan for how the top level scripts can work in both the
svn and the git environments prior to transition if you change
tools/build/v2 to tools/build?
> > Which one do you prefer?
>
> Nevertheless +1 for "tools/build" from my side if this works smoothly.
I propose to keep tools/build/v2 until all work is transitioned to git.
Then moving content of tools/build/v2 to tools/build is simple within
build submodule repository and the corresponding boost meta repository
can have modified top level scripts.
> The main culprit is that I don't even have a "tools/build" submodule at the
> moment.
yes, why is that submodule not included in .gitmodules?
It seems to be listed as submodule of boost in
ryppl/Boost2Git/repositories.txt and there is a build repository in
http://github.com/boostorg/build
I guess, for now you may add that one manually to a branch in your
clone:
git checkout -b feature/fix-broken-build
git submodule add http://github.com/boostorg/build tools/build
but it suffers from the restructure of tools/build into
tools/build/build :-(
# you could edit the submodule for your own needs - whatever they
# are, e.g:
cd tools/build
git checkout -b feature/fix-broken-build
git mv build/* .
git commit -m "only tools/build, not tools/build/build"
# then add the changed build submodule to index and commit
# the boost meta repository
cd ../..
git add tools/build
git commit -m "only tools/build, not tools/build/build"
# then later, when you get updates upstream you could rebase your
# changes on top of upstream commits from Boost2Git conversion
# with something like
git fetch
git rebase origin/master
--
Bjørn
Dave Abrahams
on Sun May 19 2013, Dave Abrahams <dave-AT-boostpro.com> wrote:
>> Jürgen has posted a patch that teaches Boost.Build to do that, and I can give it
>> a try, but before that I think cloning of the master repository should work
>
> It does work.
>
>> (others already reported some issue with chrono)
>
> I don't know what's being referred to here.
OK, I've seen those posts, but I think the problem is fixed now.
Vladimir Prus
> do you have a plan for how the top level scripts can work in both the
> svn and the git environments prior to transition if you change
> tools/build/v2 to tools/build?
Even with my limited bash-fu, that is perfectly doable.
>>> > >Which one do you prefer?
>> >
>> >Nevertheless +1 for "tools/build" from my side if this works smoothly.
>
> I propose to keep tools/build/v2 until all work is transitioned to git.
> Then moving content of tools/build/v2 to tools/build is simple within
> build submodule repository and the corresponding boost meta repository
> can have modified top level scripts.
To be honest, I don't trust git move (or lack thereof) and not really looking
forward to spending an evening with company of 'git filter-branch'. If an
infrastructure for svn to git conversion is already developed, why not create
the layout that is right?
Thanks,
Volodya
Dave Abrahams
on Tue May 21 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:
> On 21.05.2013 01:21, Bjørn Roald wrote:
>> do you have a plan for how the top level scripts can work in both the
>> svn and the git environments prior to transition if you change
>> tools/build/v2 to tools/build?
>
> Even with my limited bash-fu, that is perfectly doable.
>
>>>> > >Which one do you prefer?
>>> >
>>> >Nevertheless +1 for "tools/build" from my side if this works smoothly.
>>
>> I propose to keep tools/build/v2 until all work is transitioned to git.
>> Then moving content of tools/build/v2 to tools/build is simple within
>> build submodule repository and the corresponding boost meta repository
>> can have modified top level scripts.
>
> To be honest, I don't trust git move (or lack thereof)
Well, there's nothing really we can do about that lack of trust. It's a
simple matter to move files in Git.
> and not really looking forward to spending an evening with company of
> 'git filter-branch'.
Why do you want to revise history?
> If an infrastructure for svn to git conversion is already developed,
> why not create the layout that is right?
How do you define "right?"
--
Dave Abrahams
Jürgen Hunold
On Tuesday, 21. May 2013 22:30:10 Dave Abrahams wrote:
> on Tue May 21 2013, Vladimir Prus <ghost-AT-cs.msu.su> wrote:
> > On 21.05.2013 01:21, Bjørn Roald wrote:
> > To be honest, I don't trust git move (or lack thereof)
>
> Well, there's nothing really we can do about that lack of trust. It's a
> simple matter to move files in Git.
rewriting and start making things right after the conversion. git "move" just
works fine, even with git svn ;-) And please remember that svn still has no
atomic "mv", but just "rm/add" combination which is quite error prone,
especially with svn merge. Been there, still doing that.
> > and not really looking forward to spending an evening with company of
> > 'git filter-branch'.
>
> Why do you want to revise history?
>
> > If an infrastructure for svn to git conversion is already developed,
> > why not create the layout that is right?
>
> How do you define "right?"
transition, I presume.
But this can easily be done after the conversion.
I think we already lost too much time with failed rewrite attempts, so let us
get a working repository first. Then Volodya can test git mv v2/* . and report
results. Afterwards, we can always try rewriting in a separate clone.
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* voice: ++49 4257 300 ! Fährstraße 1
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany
Daniel James
> And please remember that svn still has no
> atomic "mv", but just "rm/add" combination which is quite error prone,
then removes the original. Using subversion as if it is git will
certainly create a mess.
Dave Abrahams
on Wed May 22 2013, Daniel James <daniel-AT-calamity.org.uk> wrote:
> On 22 May 2013 08:56, Jürgen Hunold <jhu...@gmx.eu> wrote:
>> And please remember that svn still has no
>> atomic "mv", but just "rm/add" combination which is quite error prone,
>
> That's what git does. Subversion's move command copies the file and
> then removes the original.
Daniel is correct.
--
Dave Abrahams
Dave Abrahams
on Wed May 22 2013, Jürgen Hunold <jhunold-AT-gmx.eu> wrote:
> But this can easily be done after the conversion.
>
> I think we already lost too much time with failed rewrite attempts, so let us
> get a working repository first. Then Volodya can test git mv v2/* . and report
> results. Afterwards, we can always try rewriting in a separate clone.
Rewriting _published_ history is most strongly discouraged by the Git
people, for several good reasons. *If* there is to be any
rearrangement, it should happen before the switchover, so it doesn't
bork people who are doing work based on the history originally
published.
However, I am loath to do any rearrangement that doesn't (reasonably)
faithfully reflect how things were set up in the past. Otherwise,
someone will check out an old state of the super-module and find that
things have the wrong path relationships. Of course, path relationships
will not match SVN anyway (because we don't have a modular layout in
SVN), but people on this list made it quite clear that modularizing
history was important to them, so I presume they want the Git history to
reflect reality with maximal fidelity.
That said, if the consensus is that things should be rearranged in the
build repository, we can do that. We just need clear and explicit
instructions that cover what to do with *all* the paths that have been
used in Boost.Build (including branches and tags) throughout time. As
you can see from
https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261
and following lines, just preserving the existing structure was
nontrivial.
--
Dave Abrahams
"Jürgen Hunold"
> Gesendet: Donnerstag, 23. Mai 2013 um 07:28 Uhr
> Von: "Dave Abrahams" <da...@boostpro.com>
> An: bo...@lists.boost.org
> Betreff: Re: [boost] Git Modularization Review no vote heads-up
>
>
> on Wed May 22 2013, Jürgen Hunold <jhunold-AT-gmx.eu> wrote:
>
> > But this can easily be done after the conversion.
> >
> > I think we already lost too much time with failed rewrite attempts, so let us
> > get a working repository first. Then Volodya can test git mv v2/* . and report
> > results. Afterwards, we can always try rewriting in a separate clone.
Done this myself yesterday. Works like a charm, git blame shows undisrupted history after git commit shows "rename" with 100% on each file moved.
> Rewriting _published_ history is most strongly discouraged by the Git
> people, for several good reasons. *If* there is to be any
> rearrangement, it should happen before the switchover, so it doesn't
> bork people who are doing work based on the history originally
> published.
Yes, the basic rule of git.
> However, I am loath to do any rearrangement that doesn't (reasonably)
> faithfully reflect how things were set up in the past. Otherwise,
> someone will check out an old state of the super-module and find that
> things have the wrong path relationships. Of course, path relationships
> will not match SVN anyway (because we don't have a modular layout in
> SVN), but people on this list made it quite clear that modularizing
> history was important to them, so I presume they want the Git history to
> reflect reality with maximal fidelity.
Correct.
> That said, if the consensus is that things should be rearranged in the
> build repository, we can do that. We just need clear and explicit
> instructions that cover what to do with *all* the paths that have been
> used in Boost.Build (including branches and tags) throughout time. As
> you can see from
> https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261
> and following lines, just preserving the existing structure was
> nontrivial.
That one starts with
repository build : common_branches
{
submodule of "boost" : "tools/build";
content
{
"tools/build/" : "build";
"tools/jam/" : "jam";
}
right?
So, this now creates "tools/build/build" with the contents of "tools/build" and
"tools/build/jam" with the contents of "tools/jam" which is long gone. The former is wrong, as it does not preserve the current structure despite the thing you wrote above.
I think that a one-to-one mapping like:
repository build : common_branches
{
submodule of "boost" : "tools/build";
content
{
"tools/build/" ;
}
is the way to go now. If something goes wrong, please put "tools/jam/" somewhere else.
Btw. would
repository build : common_branches
{
submodule of "boost" : "tools/build";
content
{
"tools/build/v2" ;
}
be the right way to move v2 one directory up?
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold ! juergen...@gmx.eu
* voice: 0049 4257 300 ! Fährstraße 1
* fax : 0049 4257 300 ! 31609 Balge/Sebbenhausen
* mobil: 0049 178 186 1566 ! Germany
Daniel James
> Rewriting _published_ history is most strongly discouraged by the Git
> people, for several good reasons. *If* there is to be any
> rearrangement, it should happen before the switchover, so it doesn't
> bork people who are doing work based on the history originally
> published.
repo, which would retain perfect history, and the working repo, which
would have the desired layout. The working repo could be created after
the conversion, and have enough history to be useful for general
development. The meta project could switch its reference over once the
new repository has been set up. Would also prevent any extra delay to
deal with this.
Since the odeint developers want to use their git repository
(https://github.com/headmyshoulder/odeint-v2) rather than the one
created by the conversion, that distinction might be required there
anyway.
"Jürgen Hunold"
> Gesendet: Donnerstag, 23. Mai 2013 um 10:12 Uhr
> Von: "Daniel James" <dan...@calamity.org.uk>
> An: bo...@lists.boost.org
> Betreff: Re: [boost] Git Modularization Review no vote heads-up
>
> On 23 May 2013 06:28, Dave Abrahams <da...@boostpro.com> wrote:
> > Rewriting _published_ history is most strongly discouraged by the Git
> > people, for several good reasons. *If* there is to be any
> > rearrangement, it should happen before the switchover, so it doesn't
> > bork people who are doing work based on the history originally
> > published.
>
> Maybe that could be avoided by having two repositories: the historical
> repo, which would retain perfect history, and the working repo, which
> would have the desired layout. The working repo could be created after
> the conversion, and have enough history to be useful for general
> development. The meta project could switch its reference over once the
> new repository has been set up. Would also prevent any extra delay to
> deal with this.
>
> Since the odeint developers want to use their git repository
> (https://github.com/headmyshoulder/odeint-v2) rather than the one
> created by the conversion, that distinction might be required there
> anyway.
Well, odeint is a rather new library. And for those it makes sense to
just add the existing repository as a submodule on acceptance.
AFAIK the odeint maintainers are just waiting for this to be possible.
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold ! juergen...@gmx.eu
* voice: 0049 4257 300 ! Fährstraße 1
* fax : 0049 4257 300 ! 31609 Balge/Sebbenhausen
* mobil: 0049 178 186 1566 ! Germany
_______________________________________________
Dave Abrahams
Caveat: what I mean by "preserving existing structure" is that branches
that include just a slice of the tools/build + tools/jam hierarchy have
all the files located in the same places where they appear under
trunk/. That's the appropriate way to map things for Git.
Dave Abrahams
on Thu May 23 2013, "Jürgen Hunold" <jhunold-AT-gmx.eu> wrote:
> Dave wrote:
>> if the consensus is that things should be rearranged in the
>> build repository, we can do that. We just need clear and explicit
>> instructions that cover what to do with *all* the paths that have been
>> used in Boost.Build (including branches and tags) throughout time. As
>> you can see from
>> https://github.com/ryppl/Boost2Git/blob/master/repositories.txt#L2261
>> and following lines, just preserving the existing structure was
>> nontrivial.
>
> That one starts with
>
> repository build : common_branches
> {
> submodule of "boost" : "tools/build";
> content
> {
> "tools/build/" : "build";
> "tools/jam/" : "jam";
> }
>
> right?
Yes, but note that there are five or so "repository" declarations for
the build repo.
> So, this now creates "tools/build/build" with the contents of
> "tools/build" and "tools/build/jam" with the contents of "tools/jam"
> which is long gone. The former is wrong, as it does not preserve the
> current structure despite the thing you wrote above.
It preserves *a fragment of* the current structure *within the build
repo*. That's presumably the important one. I expect the submodules
themselves to have a future as independent repositories, but not
necessarily as submodules.
> I think that a one-to-one mapping like:
>
> repository build : common_branches
> {
> submodule of "boost" : "tools/build";
> content
> {
> "tools/build/" ;
> }
>
> is the way to go now. If something goes wrong,
It will, because we'd no longer be catching tools/jam throughout
history. That will cause modularization to fail.
> please put "tools/jam/" somewhere else.
Yeah, that's not specific enough. What is "somewhere else?"
We could put it in the "engine" /v2/engine/ subdirectory of the build
repository, but that would be utterly inaccurate for older history. We
can also have it move into /v2/engine from somewhere else (as it
actually does in SVN), but again, what is "somewhere else?"
...and—just checking—do you speak for the Boost.Build project, or is
Volodya likely to countermand your requests?
> Btw. would
>
> repository build : common_branches
> {
> submodule of "boost" : "tools/build";
> content
> {
> "tools/build/v2" ;
> }
>
> be the right way to move v2 one directory up?
That would move the contents of v2 to the top level of the build repo,
but it would also drop all the commits that historically went directly
into tools/build/ (Boost.Build v1 mostly). Like I said, we need
decisions about where *everything* that was in Boost.Build—throughout
history—belongs.
> Yours,
>
> Jürgen
--
Dave Abrahams
Dave Abrahams
on Thu May 23 2013, Daniel James <daniel-AT-calamity.org.uk> wrote:
> On 23 May 2013 06:28, Dave Abrahams <da...@boostpro.com> wrote:
>> Rewriting _published_ history is most strongly discouraged by the Git
>> people, for several good reasons. *If* there is to be any
>> rearrangement, it should happen before the switchover, so it doesn't
>> bork people who are doing work based on the history originally
>> published.
>
> Maybe that could be avoided by having two repositories: the historical
> repo, which would retain perfect history, and the working repo, which
> would have the desired layout. The working repo could be created after
> the conversion, and have enough history to be useful for general
> development. The meta project could switch its reference over once the
> new repository has been set up. Would also prevent any extra delay to
> deal with this.
everyone would have been happy with the original plan to graft an
accurate SVN history when people want to refer to it, no? Please don't
tell me that the last several months of work were wasted!
> Since the odeint developers want to use their git repository
> (https://github.com/headmyshoulder/odeint-v2) rather than the one
> created by the conversion, that distinction might be required there
> anyway.
process with the one they're using,, IMO.
--
Dave Abrahams
Rene Rivera
>
> on Thu May 23 2013, "Jürgen Hunold" <jhunold-AT-gmx.eu> wrote:
>
> > Dave wrote:
>
> I think that a one-to-one mapping like:
> >
> > repository build : common_branches
> > {
> > submodule of "boost" : "tools/build";
> > content
> > {
> > "tools/build/" ;
> > }
> >
> > is the way to go now. If something goes wrong,
>
> It will, because we'd no longer be catching tools/jam throughout
> history. That will cause modularization to fail.
>
> > please put "tools/jam/" somewhere else.
>
> Yeah, that's not specific enough. What is "somewhere else?"
>
path = "jam/"
We could put it in the "engine" /v2/engine/ subdirectory of the build
> repository, but that would be utterly inaccurate for older history. We
> can also have it move into /v2/engine from somewhere else (as it
> actually does in SVN), but again, what is "somewhere else?"
>
> ...and—just checking—do you speak for the Boost.Build project, or is
> Volodya likely to countermand your requests?
>
> > Btw. would
> >
> > repository build : common_branches
> > {
> > submodule of "boost" : "tools/build";
> > content
> > {
> > "tools/build/v2" ;
> > }
> >
> > be the right way to move v2 one directory up?
>
> That would move the contents of v2 to the top level of the build repo,
> but it would also drop all the commits that historically went directly
> into tools/build/ (Boost.Build v1 mostly). Like I said, we need
> decisions about where *everything* that was in Boost.Build—throughout
> history—belongs.
>
branch = historical
path = "build/"
Or more descriptive..
1. Put anything that ever existed in the current build/v2/* files at the
root of the new build repo.
2. Put anything else in some form of "historical" branch using the path
from "boost-root/" as the path in the new repo. (I don't care that much
about the actual branch names other than to tell that they are there only
as history).
But since I don't know git sufficiently.. I don't know if that's
practically possible. Specifically I don't know if one can follow history
back across branches. if it's not possible then I would say change #2 above
to:
2. Put anything else at historical/jam, and historical/build in whatever
branches you have now.
I'll let Volodya and Jürgen comment and concur as they see fit :-)
--
--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Bjørn Roald
> on Thu May 23 2013, "Jürgen Hunold" <jhunold-AT-gmx.eu> wrote:
> > I think that a one-to-one mapping like:
> >
> > repository build : common_branches
> > {
> > submodule of "boost" : "tools/build";
> > content
> > {
> > "tools/build/" ;
> > }
> >
> > is the way to go now. If something goes wrong,
>
> It will, because we'd no longer be catching tools/jam throughout
> history. That will cause modularization to fail.
>
> > please put "tools/jam/" somewhere else.
>
> Yeah, that's not specific enough. What is "somewhere else?"
Why not in a separate repository called jam that are referenced only in
relevant past commits of boost meta repository? Boost.Build would not
be the only submodule with dependencies to other submodules, would it...
This build --> jam dependency is also gone in current boost, so I don't
understand the need to bundle all in one repository. It complicates the
conversion and changes file structure in history.
It is better to try as far as feasible to make the conversion an
accurate reflection of history while getting a sensible set of
repositories representing modularized libraries. Build is no a library,
and as a tool the modularization is good enough with two historic
repositories and a build --> jam dependency in the past. Current build
repository is self contained if I understand this correctly.
> > Btw. would
> >
> > repository build : common_branches
> > {
> > submodule of "boost" : "tools/build";
> > content
> > {
> > "tools/build/v2" ;
> > }
> >
> > be the right way to move v2 one directory up?
>
> That would move the contents of v2 to the top level of the build repo,
> but it would also drop all the commits that historically went directly
> into tools/build/ (Boost.Build v1 mostly). Like I said, we need
> decisions about where *everything* that was in Boost.Build—throughout
> history—belongs.
Agree that this is not smart or needed to be done in the conversion.
This step could be done with git mv at a convenient time after the
conversion. No real need to do that as part of the conversion - it
complicates and obfuscates.
Other build related files are in their respective repositories, this
could (or should) also be the case for all top level scripts that is
part of boost meta repository. Is it really a pure meta repository
anyway?
only my $0.02
--
Bjørn
Dave Abrahams
on Thu May 23 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
> On Thu, May 23, 2013 at 8:44 AM, Dave Abrahams <da...@boostpro.com> wrote:
>
>>
>> on Thu May 23 2013, "Jürgen Hunold" <jhunold-AT-gmx.eu> wrote:
>>
>> > Dave wrote:
>>
>> I think that a one-to-one mapping like:
>> >
>> > repository build : common_branches
>> > {
>> > submodule of "boost" : "tools/build";
>> > content
>> > {
>> > "tools/build/" ;
>> > }
>> >
>> > is the way to go now. If something goes wrong,
>>
>> It will, because we'd no longer be catching tools/jam throughout
>> history. That will cause modularization to fail.
>>
>> > please put "tools/jam/" somewhere else.
>>
>> Yeah, that's not specific enough. What is "somewhere else?"
>>
>
> branch = historical
> path = "jam/"
AFAICT, that doesn't work. tools/jam (or its equivalent) has appeared
in many different SVN branches, including trunk. You can't just squash
them all into one branch and come out with anything meaningful.
>> We could put it in the "engine" /v2/engine/ subdirectory of the build
>> repository, but that would be utterly inaccurate for older history. We
>> can also have it move into /v2/engine from somewhere else (as it
>> actually does in SVN), but again, what is "somewhere else?"
>>
>> ...and—just checking—do you speak for the Boost.Build project, or is
>> Volodya likely to countermand your requests?
>>
>> > Btw. would
>> >
>> > repository build : common_branches
>> > {
>> > submodule of "boost" : "tools/build";
>> > content
>> > {
>> > "tools/build/v2" ;
>> > }
>> >
>> > be the right way to move v2 one directory up?
>>
>> That would move the contents of v2 to the top level of the build repo,
>> but it would also drop all the commits that historically went directly
>> into tools/build/ (Boost.Build v1 mostly). Like I said, we need
>> decisions about where *everything* that was in Boost.Build—throughout
>> history—belongs.
>
> Anything before v2 existed should go to:
>
> branch = historical
> path = "build/"
As before, you can't meaningfully squash everything into one branch.
> Or more descriptive..
>
> 1. Put anything that ever existed in the current build/v2/* files at the
> root of the new build repo.
Easy enough
> 2. Put anything else in some form of "historical" branch using the path
> from "boost-root/"
? There's no "boost-root/" in SVN AFAIK
> as the path in the new repo. (I don't care that much about the actual
> branch names other than to tell that they are there only as history).
>
> But since I don't know git sufficiently.. I don't know if that's
> practically possible. Specifically I don't know if one can follow history
> back across branches.
Branches in Git are merely (reference-counted) labels for commits, each
of which is the root of a history DAG.
When you merge branch A into branch B, you can follow history from B to
the last commit on A when it was merged, and thence to all of that
commit's ancestors. If no further commits are made to A, it looks like
this:
/--> A --> A~1 ...
B --> B~1 --> B~2 ... B~N <
\--> B~(N+1) --> ...
If you mean something else by "follow history," I guess you'd better
explain.
> if it's not possible then I would say change #2 above to:
>
> 2. Put anything else at historical/jam, and historical/build in whatever
> branches you have now.
Are those (historical/jam and historical/build) supposed to be branch
names or paths?
--
Dave Abrahams
Dave Abrahams
on Thu May 23 2013, Bjørn Roald <bjorn-AT-4roald.org> wrote:
> On Thu, 2013-05-23 at 06:44 -0700, Dave Abrahams wrote:
>> on Thu May 23 2013, "Jürgen Hunold" <jhunold-AT-gmx.eu> wrote:
>
>> > I think that a one-to-one mapping like:
>> >
>> > repository build : common_branches
>> > {
>> > submodule of "boost" : "tools/build";
>> > content
>> > {
>> > "tools/build/" ;
>> > }
>> >
>> > is the way to go now. If something goes wrong,
>>
>> It will, because we'd no longer be catching tools/jam throughout
>> history. That will cause modularization to fail.
>>
>> > please put "tools/jam/" somewhere else.
>>
>> Yeah, that's not specific enough. What is "somewhere else?"
>
> Why not in a separate repository called jam that are referenced only in
> relevant past commits of boost meta repository?
We can do that.
> Boost.Build would not
> be the only submodule with dependencies to other submodules, would
> it...
Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's
hard to justify separating them.
> This build --> jam dependency is also gone in current boost,
I don't think so:
http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine
> so I don't understand the need to bundle all in one repository. It
> complicates the conversion and changes file structure in history.
I don't know what you mean by "changes file structure in history" or how
you conclude that it complicates the conversion.
> It is better to try as far as feasible to make the conversion an
> accurate reflection of history while getting a sensible set of
> repositories representing modularized libraries. Build is no a library,
> and as a tool the modularization is good enough with two historic
> repositories and a build --> jam dependency in the past. Current build
> repository is self contained if I understand this correctly.
It's only self-contained because it contains jam in its "engine"
directory. If Boost.Jam is going to be there in the present, its
revision history should be there in the past, so you can follow it.
>
>> > Btw. would
>> >
>> > repository build : common_branches
>> > {
>> > submodule of "boost" : "tools/build";
>> > content
>> > {
>> > "tools/build/v2" ;
>> > }
>> >
>> > be the right way to move v2 one directory up?
>>
>> That would move the contents of v2 to the top level of the build repo,
>> but it would also drop all the commits that historically went directly
>> into tools/build/ (Boost.Build v1 mostly). Like I said, we need
>> decisions about where *everything* that was in Boost.Build—throughout
>> history—belongs.
>
> Agree that this is not smart or needed to be done in the conversion.
> This step could be done with git mv at a convenient time after the
> conversion. No real need to do that as part of the conversion - it
> complicates and obfuscates.
>
> Other build related files are in their respective repositories, this
> could (or should) also be the case for all top level scripts that is
> part of boost meta repository.
I don't know what you mean here either. Specifics, please.
> Is it really a pure meta repository anyway?
I'm afraid it's not pure at all.
--
Dave Abrahams
Daniel James
>
> on Thu May 23 2013, Daniel James <daniel-AT-calamity.org.uk> wrote:
>>
>> Maybe that could be avoided by having two repositories: the historical
>> repo, which would retain perfect history, and the working repo, which
>> would have the desired layout. The working repo could be created after
>> the conversion, and have enough history to be useful for general
>> development. The meta project could switch its reference over once the
>> new repository has been set up. Would also prevent any extra delay to
>> deal with this.
>
> If that were going to be acceptable to the community, presumably
> everyone would have been happy with the original plan to graft an
> accurate SVN history when people want to refer to it, no? Please don't
> tell me that the last several months of work were wasted!
acceptable to another. It should ultimately be up to the maintainers
of individual modules. Also, you would still have an accurate history
in the historical module(s). Which was the point, it was an attempt at
a compromise between two conflicting desires (to have an accurate
history, and to have a repository with a different directory layout).
>> Since the odeint developers want to use their git repository
>> (https://github.com/headmyshoulder/odeint-v2) rather than the one
>> created by the conversion, that distinction might be required there
>> anyway.
>
> It would be simpler to just replace the one generated by the conversion
> process with the one they're using,, IMO.
will refer to changesets generated by the svn to git conversion.
Bjørn Roald
> on Thu May 23 2013, Bjørn Roald <bjorn-AT-4roald.org> wrote:
>
> > Boost.Build would not
> > be the only submodule with dependencies to other submodules, would
> > it...
>
> Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's
> hard to justify separating them.
I understand that point of view, but if historic Jam could be used by
itself, why not have them as separate modules?
> > This build --> jam dependency is also gone in current boost,
>
> I don't think so:
> http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine
I was not aware there is a single line of history here - in case it is
it may be worth trying to re-join the histories if that is done by the
conversion. but it seems odd to re-introduce tools/jam in
tools/build/jam, a new place where it has never been -- and move
tools/build to tools/build/build, where it has never been.
> > so I don't understand the need to bundle all in one repository. It
> > complicates the conversion and changes file structure in history.
>
> I don't know what you mean by "changes file structure in history" or how
> you conclude that it complicates the conversion.
Well, are you saying I can check out any boost historic version from
http://github.com/boostorg/boost and run the instructions of that
release to successfully build? Those instructions will fail as
structure of those historic commits are changed beyond moving the
headers. The changes in tools/build is a prime example.
> > It is better to try as far as feasible to make the conversion an
> > accurate reflection of history while getting a sensible set of
> > repositories representing modularized libraries. Build is no a library,
> > and as a tool the modularization is good enough with two historic
> > repositories and a build --> jam dependency in the past. Current build
> > repository is self contained if I understand this correctly.
>
> It's only self-contained because it contains jam in its "engine"
> directory. If Boost.Jam is going to be there in the present, its
> revision history should be there in the past, so you can follow it.
so are you are saying you will have some commit the build repository
history show how files are moved from tools/build/jam where they never
have been to tools/build/build/v2/engine where they never went? Even if
you manage to make such a commit, how is that preserving file structure
in the history?
> >> > Btw. would
> >> >
> >> > repository build : common_branches
> >> > {
> >> > submodule of "boost" : "tools/build";
> >> > content
> >> > {
> >> > "tools/build/v2" ;
> >> > }
> >> >
> >> > be the right way to move v2 one directory up?
> >>
> >> That would move the contents of v2 to the top level of the build repo,
> >> but it would also drop all the commits that historically went directly
> >> into tools/build/ (Boost.Build v1 mostly). Like I said, we need
> >> decisions about where *everything* that was in Boost.Build—throughout
> >> history—belongs.
> >
> > Agree that this is not smart or needed to be done in the conversion.
> > This step could be done with git mv at a convenient time after the
> > conversion. No real need to do that as part of the conversion - it
> > complicates and obfuscates.
> >
> > Other build related files are in their respective repositories, this
> > could (or should) also be the case for all top level scripts that is
> > part of boost meta repository.
>
> I don't know what you mean here either. Specifics, please.
Ok, it is just the obvious I guess I am trying to confirm, sorry for
not being clear here and having to explain these things you know much
better than me - feels very odd. But I will try to explayn what I mean.
There are jam files all over the place -- to find the files that have
jam in name. The command
@ find | grep -i jam | grep -v tools/build
does not provide complete list of all build related scripts outside of
tools/build, but it will list a lot of build stuff that will not be part
of build repository I presume, some of these are in the top level
directory
./project-config.jam
./Jamfile.v2
./Jamroot.jam
./boostcpp.jam
and they may end up staying there, i.e. not become part of any of the
new submodules, candidates are :
in addition bootstrap.(sh|bat) are part of the build system. With file
structure changes in history for tools/build and tools/jam, these and
their past siblings are surly broken.
In addition files in lib/*/build/ directories that belong to the
respective lib/* submodules does not go into build repository, right?
Nevertheless I struggle with understanding the intension of some of the
entries for the build repository in
https://github.com/ryppl/Boost2Git/blob/master/repositories.txt so there
may be intention to do more than the current converted repositories
reflect and I expect here.
--
Bjørn
Rene Rivera
>
> on Thu May 23 2013, Rene Rivera <grafikrobot-AT-gmail.com> wrote:
>
> > On Thu, May 23, 2013 at 8:44 AM, Dave Abrahams <da...@boostpro.com>
> wrote:
> >
> >>
> >> on Thu May 23 2013, "Jürgen Hunold" <jhunold-AT-gmx.eu> wrote:
> > Or more descriptive..
> >
> > 1. Put anything that ever existed in the current build/v2/* files at the
> > root of the new build repo.
>
> Easy enough
>
> > 2. Put anything else in some form of "historical" branch using the path
> > from "boost-root/"
>
> ? There's no "boost-root/" in SVN AFAIK
>
boost tree in svn (of which there are various). I.e. to just use the
subpath from the boost root (for example tools/jam, or tools/build).
>
> > as the path in the new repo. (I don't care that much about the actual
> > branch names other than to tell that they are there only as history).
> >
> > But since I don't know git sufficiently.. I don't know if that's
> > practically possible. Specifically I don't know if one can follow history
> > back across branches.
>
> Branches in Git are merely (reference-counted) labels for commits, each
> of which is the root of a history DAG.
>
> When you merge branch A into branch B, you can follow history from B to
> the last commit on A when it was merged, and thence to all of that
> commit's ancestors. If no further commits are made to A, it looks like
> this:
>
> /--> A --> A~1 ...
> B --> B~1 --> B~2 ... B~N <
> \--> B~(N+1) --> ...
>
> If you mean something else by "follow history," I guess you'd better
> explain.
>
versions of a single file. I.e. I only care about seeing *all* the diffs
for say the "jam.h" file regardless of what branch the history happened in.
In svn terms it would be following the history across copies (which is what
the branches are in svn). Hence I would see all the history here <
https://github.com/boostorg/build/commits/master/build/v2/engine/jam.h>
event if the commit did not occur in the master branch (because the file
didn't exist in the branch and was copied from another branch).
Which I guess means that.. Yes that's what I mean by history :-)
> if it's not possible then I would say change #2 above to:
> >
> > 2. Put anything else at historical/jam, and historical/build in whatever
> > branches you have now.
>
> Are those (historical/jam and historical/build) supposed to be branch
> names or paths?
>
--
--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Daniel Pfeifer
Jürgen, Volodya, all,
Would Rene's approach be acceptable?
- Send tools/build/v2/* to the top level of the build repository.
- Send the rest of tools/build to historic/build in the build repo.
- Send tools/jam to historic/jam in the build repo.
Pro:
- The change is quick and easy.
- I did it, so you can review it.
Contra:
- It does not reflect the history. You might not succeed to check out and
build a very old version of boost.
cheers, Daniel
Jürgen Hunold
On Saturday, 25. May 2013 14:35:23 Daniel Pfeifer wrote:
> Jürgen, Volodya, all,
>
> Would Rene's approach be acceptable?
>
> - Send tools/build/v2/* to the top level of the build repository.
> - Send the rest of tools/build to historic/build in the build repo.
> - Send tools/jam to historic/jam in the build repo.
> Pro:
> - The change is quick and easy.
> - I did it, so you can review it.
be correct. I'll poke around the history the next days. And try to get my
patches applying again.
> Contra:
> - It does not reflect the history. You might not succeed to check out and
> build a very old version of boost.
somewhere. This is sufficient at least for me.
The removal of "v2" requires some patching. I've attached to necessary patch
to bootstrap.{sh,bat}. Can this be applied automatically right after the
conversion so that the scripts are working on checkout? Otherwise the
patch(es) should be added to the wiki page.
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany
Jürgen Hunold
On Saturday, 25. May 2013 15:23:34 Jürgen Hunold wrote:
> The removal of "v2" requires some patching. I've attached to necessary patch
> to bootstrap.{sh,bat}. Can this be applied automatically right after the
> conversion so that the scripts are working on checkout? Otherwise the
> patch(es) should be added to the wiki page.
Next task is generating forward headers.
Daniel Pfeifer
> Hi,
>
> On Saturday, 25. May 2013 15:23:34 Jürgen Hunold wrote:
> > The removal of "v2" requires some patching. I've attached to necessary
> patch
> > to bootstrap.{sh,bat}. Can this be applied automatically right after the
> > conversion so that the scripts are working on checkout? Otherwise the
> > patch(es) should be added to the wiki page.
>
> Forgot to patch boost-build.jam. New patch attached.
> Next task is generating forward headers.
Hi Jürgen,
This patch does not apply to the complete history. I added the following
command instead:
$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i
-e 's|build[/\\]v2|build|g' {} \;"
Bjørn Roald
> 2013/5/25 Jürgen Hunold <jhu...@gmx.eu>
>
> > Hi,
> >
> > On Saturday, 25. May 2013 15:23:34 Jürgen Hunold wrote:
> > > The removal of "v2" requires some patching. I've attached to necessary
> > patch
> > > to bootstrap.{sh,bat}. Can this be applied automatically right after the
> > > conversion so that the scripts are working on checkout? Otherwise the
> > > patch(es) should be added to the wiki page.
> >
> > Forgot to patch boost-build.jam. New patch attached.
> > Next task is generating forward headers.
>
>
> Hi Jürgen,
>
> This patch does not apply to the complete history. I added the following
> command instead:
>
> $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i
> -e 's|build[/\\]v2|build|g' {} \;"
Nice - composing with tools the unix way certainly has it's strengths.
Given that the proposed changes to tools/build structure are made in git
history with the conversion, I think it make a lot of sense to change
these scripts in history as well, as Daniel suggest here.
That said, I am not much fan of changing history when not strictly
needed. I am not convinced all these changes in tools/build history is
needed, but I am not making that call.
For the record, here is another alternative:
- Send history of tools/jam to historic/jam in the build repo.
- After transition to git is complete, move tools/build/v2/* to
the top level of the build repository and change top
level scrips as a single git commit in build and boost.
Pro:
- Sending history of tools/jam to historic/jam to is quick and easy as
part of conversion to modularized boost.
- Reflect real history for build v1 and v2
Contra:
- It does not reflect the history for jam.
Jam is moved in history to get all into the build repository. This may
cause problems following legacy instructions for building historic
boost. However jam was normally built separately as a first manual step,
so it may be feasible and sufficient to have simple wiki or other howto
instructions on building historic boost from modularized git
repositories. How to deal with the staging of header files for the
build in the boost directory does also need to be described the same
place if such descriptions are created or needed.
--
Bjørn
Jürgen Hunold
On Saturday, 25. May 2013 23:00:48 Daniel Pfeifer wrote:
> This patch does not apply to the complete history. I added the following
> command instead:
>
> $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i
> -e 's|build[/\\]v2|build|g' {} \;"
running b2 first time results in:
~/src/devel/modular-boost$ ./b2 -j4 -q
/home/hunold/src/devel/modular-boost/tools/build/build/project.jam:287: in
load-jamfile from module project
error: Multiple Jamfiles found at '/home/hunold/src/devel/modular-boost'
error: Filenames are: Jamroot Jamroot.jam
From the contents of Jamroot.jam I guess this comes from one the sandboxes:
git log Jamroot.jam
commit 689e7d3d89c109a9ca1c59f0897071d0b35cf7ac
Author: John Maddock <jo...@johnmaddock.co.uk>
Date: Mon Jun 21 08:36:14 2010 +0000
Add use-project rule to suppress some BB issues.
svn path=/sandbox/Jamroot; revision=63182
Can you please remove this one from the toplevel repository?
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* voice: ++49 4257 300 ! Fährstraße 1
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany
Jürgen Hunold
On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:
> On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:
> > This patch does not apply to the complete history. I added the following
> > command instead:
> >
> > $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed -i
> > -e 's|build[/\\]v2|build|g' {} \;"
>
> Nice - composing with tools the unix way certainly has it's strengths.
> Given that the proposed changes to tools/build structure are made in git
> history with the conversion, I think it make a lot of sense to change
> these scripts in history as well, as Daniel suggest here.
> That said, I am not much fan of changing history when not strictly
> needed. I am not convinced all these changes in tools/build history is
> needed, but I am not making that call.
>
> For the record, here is another alternative:
>
> - Send history of tools/jam to historic/jam in the build repo.
> - After transition to git is complete, move tools/build/v2/* to
> the top level of the build repository and change top
> level scrips as a single git commit in build and boost.
>
> Pro:
> - Sending history of tools/jam to historic/jam to is quick and easy as
> part of conversion to modularized boost.
> - Reflect real history for build v1 and v2
> Contra:
> - It does not reflect the history for jam.
> Jam is moved in history to get all into the build repository. This may
> cause problems following legacy instructions for building historic
> boost.
repository was never the goal. You can always built them from read-only
subversion.
> However jam was normally built separately as a first manual step,
> so it may be feasible and sufficient to have simple wiki or other howto
> instructions on building historic boost from modularized git
> repositories. How to deal with the staging of header files for the
> build in the boost directory does also need to be described the same
> place if such descriptions are created or needed.
source tree rewriting needed for the modularisation itself is quite heavy.
This is the main reason why Dave proposed dropping the history altogether at
the first conversion attempt I think. The only "real" reason (at least for me)
for keeping history is the ability to use "git diff" and "git blame" and follow
edits and reading commit messages.
Those are my personal opinions, but I guees Rene and Volodya are on the same
track here.
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany
Rene Rivera
> On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:
> > Contra:
> > - It does not reflect the history for jam.
>
> And jam was moved into v2/engine several years ago.
>
Those are my personal opinions, but I guees Rene and Volodya are on the same
> track here.
>
--
--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Jürgen Hunold
On Saturday, 25. May 2013 15:38:07 Jürgen Hunold wrote:
> Next task is generating forward headers.
Jamroot and tools/build repository attached.
Caveats:
- Dependency analysis is screwed, symlinks are created _after_ compilation.
Just run compilation twice for the time being.
Repository layout:
Most targets fail with:
fatal error: boost/detail/workaround.hpp: No such file or directory
We have 7 workaround files after the conversion:
libs/math/include/boost/math/tools/workaround.hpp
libs/interprocess/include/boost/interprocess/detail/workaround.hpp
libs/test/include/boost/test/detail/workaround.hpp
libs/intrusive/include/boost/intrusive/detail/workaround.hpp
libs/container/include/boost/container/detail/workaround.hpp
libs/mpl/include/boost/mpl/aux_/config/workaround.hpp
but the missing one from boost/details seems to belong to mpl.
Any ideas?
Daniel Pfeifer
> Hi Bjørn,
>
> On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:
> > On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:
> > > This patch does not apply to the complete history. I added the
> following
> > > command instead:
> > >
> > > $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec
> sed -i
> > > -e 's|build[/\\]v2|build|g' {} \;"
> >
> > Nice - composing with tools the unix way certainly has it's strengths.
>
> Definetely ;-)
It is also pretty slow. Rewriting Boost's history that way takes about 4
hours.
I changed the mapping accoring to Bjørn's suggestion.
tools/build/v2 will end up in tools/build/v2. I think that makes most sense.
cheers, Daniel
Bjørn Roald
> 2013/5/26 Jürgen Hunold <jhu...@gmx.eu>
>
> > Hi Bjørn,
> >
> > On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:
> > > On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:
> > > > This patch does not apply to the complete history. I added the
> > following
> > > > command instead:
> > > >
> > > > $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec
> > sed -i
> > > > -e 's|build[/\\]v2|build|g' {} \;"
> > >
> > > Nice - composing with tools the unix way certainly has it's strengths.
> >
> > Definetely ;-)
>
>
> It is also pretty slow. Rewriting Boost's history that way takes about 4
> hours.
>
> I changed the mapping accoring to Bjørn's suggestion.
>
> tools/build/v2 will end up in tools/build/v2. I think that makes most sense.
Err -- although I still support my suggestion, boost build maintainers
seems to differ and they have more of a say here than I do and I
willingly yield. I feel like I am just getting in the way of progress
here.
--
Bjørn
Daniel Pfeifer
> Hi,
>
> On Saturday, 25. May 2013 15:38:07 Jürgen Hunold wrote:
> > Next task is generating forward headers.
>
> I got the patches applying and Boost.Build getting started. Patches for
> Jamroot and tools/build repository attached.
>
> Caveats:
> - Dependency analysis is screwed, symlinks are created _after_ compilation.
> Just run compilation twice for the time being.
>
> Repository layout:
>
> Most targets fail with:
>
> fatal error: boost/detail/workaround.hpp: No such file or directory
>
> We have 7 workaround files after the conversion:
>
> libs/math/include/boost/math/tools/workaround.hpp
> libs/interprocess/include/boost/interprocess/detail/workaround.hpp
> libs/test/include/boost/test/detail/workaround.hpp
> libs/intrusive/include/boost/intrusive/detail/workaround.hpp
> libs/container/include/boost/container/detail/workaround.hpp
> libs/mpl/include/boost/mpl/aux_/config/workaround.hpp
>
> but the missing one from boost/details seems to belong to mpl.
>
> Any ideas?
Many files from "boost/detail" were sent to a repository "core" that was
not mapped as a submodule.
I renamed the repository to "detail" and mapped it to "libs/detail".
Bjørn Roald
> Hi,
>
> On Saturday, 25. May 2013 15:38:07 Jürgen Hunold wrote:
> > Next task is generating forward headers.
>
> I got the patches applying and Boost.Build getting started. Patches for
> Jamroot and tools/build repository attached.
Great, I tried it and I get:
bjorn@frodo2:~/repo/boost (master)$ ./b2
link.jam: No such file or directory
Jamroot:144: in modules.load
ERROR: rule "link-directory" unknown in module
"Jamfile</home/bjorn/repo/boost>".
/home/bjorn/repo/boost/tools/build/build/project.jam:311: in
load-jamfile
/home/bjorn/repo/boost/tools/build/build/project.jam:64: in load
/home/bjorn/repo/boost/tools/build/build/project.jam:145: in
project.find
/home/bjorn/repo/boost/tools/build/build-system.jam:535: in load
/home/bjorn/repo/boost/../boost/tools/build/kernel/modules.jam:289: in
import
/home/bjorn/repo/boost/../boost/tools/build/kernel/bootstrap.jam:139: in
boost-build
/home/bjorn/repo/boost/boost-build.jam:67: in module scope
Looks to me like link-directory rule is missing. If you have added new
file or changes to git index, then you need to do git diff HEAD.
Without HEAD, git diff compare working dir to the index.
--
Bjørn
Daniel Pfeifer
> Hi Daniel,
>
> On Saturday, 25. May 2013 23:00:48 Daniel Pfeifer wrote:
> > This patch does not apply to the complete history. I added the following
> > command instead:
> >
> > $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed
> -i
> > -e 's|build[/\\]v2|build|g' {} \;"
>
>
> Okay, that "fixes" all the history. Now bootstrap.sh works out-of-the-box.
> But
> running b2 first time results in:
>
> ~/src/devel/modular-boost$ ./b2 -j4 -q
> /home/hunold/src/devel/modular-boost/tools/build/build/project.jam:287: in
> load-jamfile from module project
> error: Multiple Jamfiles found at '/home/hunold/src/devel/modular-boost'
> error: Filenames are: Jamroot Jamroot.jam
>
> From the contents of Jamroot.jam I guess this comes from one the sandboxes:
>
> git log Jamroot.jam
> commit 689e7d3d89c109a9ca1c59f0897071d0b35cf7ac
> Author: John Maddock <jo...@johnmaddock.co.uk>
> Date: Mon Jun 21 08:36:14 2010 +0000
>
> Add use-project rule to suppress some BB issues.
>
> svn path=/sandbox/Jamroot; revision=63182
>
> Can you please remove this one from the toplevel repository?
I see there are multiple files in the root directory that don't belong
there. Investigating...
Rene Rivera
> On Sun, 2013-05-26 at 21:25 +0200, Daniel Pfeifer wrote:
> > 2013/5/26 Jürgen Hunold <jhu...@gmx.eu>
> >
> > > Hi Bjørn,
> > >
> > > On Sunday, 26. May 2013 13:01:18 Bjørn Roald wrote:
> > > > On Sat, 2013-05-25 at 23:00 +0200, Daniel Pfeifer wrote:
> > > > > This patch does not apply to the complete history. I added the
> > > following
> > > > > command instead:
> > > > >
> > > > > $ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec
> > > sed -i
> > > > > -e 's|build[/\\]v2|build|g' {} \;"
> > > >
> > > > Nice - composing with tools the unix way certainly has it's
> strengths.
> > >
> > > Definetely ;-)
> >
> >
> > It is also pretty slow. Rewriting Boost's history that way takes about 4
> > hours.
> >
> > I changed the mapping accoring to Bjørn's suggestion.
> >
> > tools/build/v2 will end up in tools/build/v2. I think that makes most
> sense.
>
> Err -- although I still support my suggestion, boost build maintainers
> seems to differ and they have more of a say here than I do and I
> willingly yield. I feel like I am just getting in the way of progress
> here.
>
But if life is such a pain with squashing the v2 subtree.. I guess the b2
maintainers will just have to squash the v2 at the source.. I.e. make the
current Boost svn structure be tools/build/* moving all files and changing
the scripts.. and especially painful adjusting the testing regression
scripts. Which may impact the upcoming release schedules.
--
--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim - grafikrobot/yahoo
Bjørn Roald
> I was about to mention something.. So thanks for the comment :-)
>
> But if life is such a pain with squashing the v2 subtree.. I guess the b2
> maintainers will just have to squash the v2 at the source.. I.e. make the
> current Boost svn structure be tools/build/* moving all files and changing
> the scripts.. and especially painful adjusting the testing regression
> scripts. Which may impact the upcoming release schedules.
I may be misunderstanding you, but it was _not_ my suggestion for you to
change anything at all in svn. Nor do I think you need to move
anything in git from tools/build/v2 to tools/build until you feel ready
for it. E.g after next release. I am concerned we are have different
approaches in mind and think we are talking of the same approach.
Basically, what I have in mind is keeping everything that is in active
use as-is. Then when you are ready to restructure, at some time
convenient after the move to git, you simply do it in git. History is
traced across a commit where you do
git mv v2/* .
I think, at least if done properly this should work fine, I tried it -
had to move the duplicated css file out of the way first.
To some degree git tracing file history across moves depend on the tool
used for tracing history. There is not an explicit notion of a moved
file in git, just derived from commit changes. So to some degree I
think it may be safer to get the conversion to do it if the result is
all tools/build/v2 files history are in tools/build. But I would have
taken that risk and used git mv. It is up to you though.
Off-cause you may do it in svn if you like, but then you are on a tight
schedule as you point out, and I would be checking with Daniel if the
conversion scripts can actually maintain your file history for the files
moved. I would not suggest to fix history later in git after the boost
repository commits are public.
The last option, is to ignore fixing the history with
$ git filter-branch --tree-filter "find . -type f -maxdepth 1 -exec sed
-i -e 's|build[/\\]v2|build|g' {} \;"
all together, or just do it for the final conversion. We do not need it
for getting all the other stuff right. And it is questionable how
useful it is at all as special instructions or scripts are needed anyway
to build historic boost in modularized boost. Maybe simply have a wiki
page with a bootstrap-historic.sh script along the lines of:
find . -type f -maxdepth 1 -exec sed -i -e 's|build[/\\]v2|build|g' {}
\;
... make-header-symlinks and other stuff ...
./bootstrap.sh
--
Bjørn
>
>
> --
Beman Dawes
...>
> not mapped as a submodule.
> I renamed the repository to "detail" and mapped it to "libs/detail".
That should be instantly recognized by those familiar with Boost.
--Beman
Jürgen Hunold
On Sunday, 26. May 2013 21:44:26 Bjørn Roald wrote:
> Looks to me like link-directory rule is missing. If you have added new
> file or changes to git index, then you need to do git diff HEAD.
> Without HEAD, git diff compare working dir to the index.
Complete patch attached.
Jürgen Hunold
On Sunday, 26. May 2013 21:43:32 Daniel Pfeifer wrote:
> Many files from "boost/detail" were sent to a repository "core" that was
> not mapped as a submodule.
> I renamed the repository to "detail" and mapped it to "libs/detail".
boost/array.hpp:52:41: fatal error: boost/functional/hash_fwd.hpp: No such file
or directory
It seems that somehow libs/functional is missing the "hash" subdirectory too.
Another missing submodule?
As Steven has already committed most of the symlink logic, here two new
patches activating symlinks.
Daniel Pfeifer
> Hi Daniel,
>
> On Sunday, 26. May 2013 21:43:32 Daniel Pfeifer wrote:
> > Many files from "boost/detail" were sent to a repository "core" that was
> > not mapped as a submodule.
> > I renamed the repository to "detail" and mapped it to "libs/detail".
>
> Thanks. But now I get
>
> boost/array.hpp:52:41: fatal error: boost/functional/hash_fwd.hpp: No such
> file
> or directory
>
> It seems that somehow libs/functional is missing the "hash" subdirectory
> too.
>
> Another missing submodule?
>
Oh, "functional_hash" is not mapped as a submodule yet!
That is a tricky one. I'd prefer the boost metarepository to be the only
repository with submodules.
But since "libs/functional" is a submodule, "libs/functional/hash" could
only be a submodule of that one.
Would it be acceptable to keep Boost.Functional/Hash in the repository of
Boost.Functional?
-- Daniel
Daniel James
> Oh, "functional_hash" is not mapped as a submodule yet!
>
> That is a tricky one. I'd prefer the boost metarepository to be the only
> repository with submodules.
> But since "libs/functional" is a submodule, "libs/functional/hash" could
> only be a submodule of that one.
>
> Would it be acceptable to keep Boost.Functional/Hash in the repository of
> Boost.Functional?
different location it will break the documentation build, but that
isn't much of a problem.
Jürgen Hunold
On Monday, 27. May 2013 17:38:00 Daniel Pfeifer wrote:
> 2013/5/27 Jürgen Hunold <jhu...@gmx.eu>
> > too.
> >
> > Another missing submodule?
>
> Oh, "functional_hash" is not mapped as a submodule yet!
dynamic_bitset
function_types
logic
statechart
tuple
unordered
Found while trying to run the regression tests. See attached patch for the
commented out tests.
And assign is registered, but can't be checked out. Any progress on this one?
Michael Marcin
>
> And assign is registered, but can't be checked out. Any progress on this one?
>
HEAD is now at e36bf9c Resolve C++0x ambiguities. Fixes #3073
Vladimir Prus
> I even fear that building historic boost is from git is not even possible. The
> source tree rewriting needed for the modularisation itself is quite heavy.
> This is the main reason why Dave proposed dropping the history altogether at
> the first conversion attempt I think. The only "real" reason (at least for me)
> for keeping history is the ability to use "git diff" and "git blame" and follow
> edits and reading commit messages.
>
> Those are my personal opinions, but I guees Rene and Volodya are on the same
> track here.
and "git show" without hassle.
I *suspect* that "git mv" after conversion won't be quite as smooth, e.g. if I print a
previous commit it will have "v2/" prefix in all paths, which will slightly complicate
things. And practically speaking, the history is only important from the point where
jam moved into build/v2/engine.
Top-level scripts should not be a problem; we can code them to detect which layout is used (well,
I can do that), and I don't think that building previous Boost releases from git repository
is a use case were need to support -- even the structure of the checkout will be different,
so checking out from git, rearranging, and then building is not 100% reproduction of past builds
anyway. I.e. we probably need not bother with rewriting history of top-level scripts.
So where are we now? Daniel previously said:
- Send tools/build/v2/* to the top level of the build repository.
- Send tools/jam to historic/jam in the build repo.
and it looks like it was actually done? I don't know whether 'historic' is a branch
or directory name, but it does not matter much. However, it looks like further tweak
was made to disable this?
Thanks,
Volodya
Beman Dawes
>...
> ... I don't think that building previous Boost releases from
> is a use case were need to support -- even the structure of the checkout
> will be different,
> so checking out from git, rearranging, and then building is not 100%
> reproduction of past builds
> anyway.
For those not comfortable with going back to the old svn repo for
archeological research involving rebuilding an old release, there is
always SourceForge. They've got past releases back to 1.20.0 (February
2003) available.
--Beman
Jürgen Hunold
On Thursday, 30. May 2013 09:44:00 Vladimir Prus wrote:
> On 26.05.2013 17:15, Jürgen Hunold wrote:
> > I even fear that building historic boost is from git is not even possible.
> > The source tree rewriting needed for the modularisation itself is quite
> > heavy. This is the main reason why Dave proposed dropping the history
> > altogether at the first conversion attempt I think. The only "real"
> > reason (at least for me) for keeping history is the ability to use "git
> > diff" and "git blame" and follow edits and reading commit messages.
> >
> > Those are my personal opinions, but I guees Rene and Volodya are on the
> > same track here.
>
> Yep. I primarily care that in the new git repository, I can go "git blame"
> and "git log" and "git show" without hassle.
> I *suspect* that "git mv" after conversion won't be quite as smooth, e.g. if
> I print a previous commit it will have "v2/" prefix in all paths, which
> will slightly complicate things.
> And practically speaking, the history is
> only important from the point where jam moved into build/v2/engine.
> Top-level scripts should not be a problem; we can code them to detect which
> layout is used (well, I can do that), and I don't think that building
> previous Boost releases from git repository is a use case were need to
> support -- even the structure of the checkout will be different, so
> checking out from git, rearranging, and then building is not 100%
> reproduction of past builds anyway. I.e. we probably need not bother with
> rewriting history of top-level scripts.
even possible."
> So where are we now? Daniel previously said:
>
> - Send tools/build/v2/* to the top level of the build repository.
> - Send the rest of tools/build to historic/build in the build repo.
> - Send tools/jam to historic/jam in the build repo.
>
> and it looks like it was actually done?
> I don't know whether 'historic' is a
> branch or directory name, but it does not matter much. However, it looks
> like further tweak was made to disable this?
conversion time to 4 hours. That seems to be the current bottleneck.
But I still cannot get my adapded patch from Steven for symlinking to work
with the regression tests and at least 5 submodules are still missing.
My 2 cents:
Finalise the conversion and get the libraries etc. up and running.
Then enable the v2/ removal again and test if everything is still working.
Done.
Yours,
Jürgen
--
* Dipl.-Math. Jürgen Hunold !
* voice: ++49 4257 300 ! Fährstraße 1
* fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen
* jhu...@gmx.eu ! Germany
Vladimir Prus
> My 2 cents:
>
> Finalise the conversion and get the libraries etc. up and running.
> Then enable the v2/ removal again and test if everything is still working.
> Done.
that's very reasonable and practical suggestion.
If we have neither content in place, or infrastructure updates ready, then details of history is
of secondary importance.
Dave Abrahams
on Thu May 23 2013, Daniel James <daniel-AT-calamity.org.uk> wrote:
> On 23 May 2013 14:47, Dave Abrahams <da...@boostpro.com> wrote:
>>
>> on Thu May 23 2013, Daniel James <daniel-AT-calamity.org.uk> wrote:
>
>>>
>>> Maybe that could be avoided by having two repositories: the historical
>>> repo, which would retain perfect history, and the working repo, which
>>> would have the desired layout. The working repo could be created after
>>> the conversion, and have enough history to be useful for general
>>> development. The meta project could switch its reference over once the
>>> new repository has been set up. Would also prevent any extra delay to
>>> deal with this.
>>
>> If that were going to be acceptable to the community, presumably
>> everyone would have been happy with the original plan to graft an
>> accurate SVN history when people want to refer to it, no? Please don't
>> tell me that the last several months of work were wasted!
>
> What's acceptable to one person is quite different to what's
> acceptable to another.
No kidding!
> It should ultimately be up to the maintainers of individual
> modules.
If anyone expects the Boost master repository to contain accurate
submodule references through history, maintainers must not be allowed to
delete old branches or rewrite old history.
> Also, you would still have an accurate history in the historical
> module(s). Which was the point, it was an attempt at a compromise
> between two conflicting desires (to have an accurate history, and to
> have a repository with a different directory layout).
As I've said before, Git *cannot* represent exactly what SVN
represents. If you want an accurate history, use SVN to get it. The
best we can do with Git is to approximate actual history.
>>> Since the odeint developers want to use their git repository
>>> (https://github.com/headmyshoulder/odeint-v2) rather than the one
>>> created by the conversion, that distinction might be required there
>>> anyway.
>>
>> It would be simpler to just replace the one generated by the conversion
>> process with the one they're using, IMO.
>
> I thought that would mess up the history of the meta-module, as it
> will refer to changesets generated by the svn to git conversion.
It would, you're right.
--
Dave Abrahams
Dave Abrahams
on Fri May 24 2013, Bjørn Roald <bjorn-AT-4roald.org> wrote:
> On Thu, 2013-05-23 at 23:06 -0700, Dave Abrahams wrote:
>> on Thu May 23 2013, Bjørn Roald <bjorn-AT-4roald.org> wrote:
>>
>
>> > Boost.Build would not
>> > be the only submodule with dependencies to other submodules, would
>> > it...
>>
>> Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's
>> hard to justify separating them.
>
> I understand that point of view, but if historic Jam could be used by
> itself, why not have them as separate modules?
Because it makes everything harder. If the Boost.Build maintainers
would prefer to have them separated, we can do that, but I advise
against it.
>> > This build --> jam dependency is also gone in current boost,
>>
>> I don't think so:
>> http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine
>
> I was not aware there is a single line of history here - in case it is
> it may be worth trying to re-join the histories if that is done by the
> conversion. but it seems odd to re-introduce tools/jam in
> tools/build/jam, a new place where it has never been -- and move
> tools/build to tools/build/build, where it has never been.
You have to let go of the idea of the Boost super-repository having the
actual historical paths to anything.
>> > so I don't understand the need to bundle all in one repository. It
>> > complicates the conversion and changes file structure in history.
>>
>> I don't know what you mean by "changes file structure in history" or how
>> you conclude that it complicates the conversion.
>
> Well, are you saying I can check out any boost historic version from
> http://github.com/boostorg/boost and run the instructions of that
> release to successfully build?
No. That *will not* be the case. For that, use SVN.
> Those instructions will fail as structure of those historic commits
> are changed beyond moving the headers. The changes in tools/build is a
> prime example.
>
>> > It is better to try as far as feasible to make the conversion an
>> > accurate reflection of history while getting a sensible set of
>> > repositories representing modularized libraries. Build is no a library,
>> > and as a tool the modularization is good enough with two historic
>> > repositories and a build --> jam dependency in the past. Current build
>> > repository is self contained if I understand this correctly.
>>
>> It's only self-contained because it contains jam in its "engine"
>> directory. If Boost.Jam is going to be there in the present, its
>> revision history should be there in the past, so you can follow it.
>
> so are you are saying you will have some commit the build repository
> history show how files are moved from tools/build/jam where they never
> have been to tools/build/build/v2/engine where they never went? Even if
> you manage to make such a commit, how is that preserving file structure
> in the history?
I didn't say anything about preserving file structure in the history.
What I am saying is that you should be able to follow the changes to any
file that is part of Boost.Jam through its history, and if those files
are in the Build repository in the present but jump there from a different
repository some time in the past, you won't be able to do that.
>> > Agree that this is not smart or needed to be done in the conversion.
>> > This step could be done with git mv at a convenient time after the
>> > conversion. No real need to do that as part of the conversion - it
>> > complicates and obfuscates.
>> >
>> > Other build related files are in their respective repositories, this
>> > could (or should) also be the case for all top level scripts that is
>> > part of boost meta repository.
>>
>> I don't know what you mean here either. Specifics, please.
>
> Ok, it is just the obvious I guess I am trying to confirm, sorry for
> not being clear here and having to explain these things you know much
> better than me - feels very odd. But I will try to explayn what I mean.
>
> There are jam files all over the place -- to find the files that have
> jam in name. The command
>
> @ find | grep -i jam | grep -v tools/build
>
> does not provide complete list of all build related scripts outside of
> tools/build, but it will list a lot of build stuff that will not be part
> of build repository I presume, some of these are in the top level
> directory
>
> ./project-config.jam
> ./Jamfile.v2
> ./Jamroot.jam
> ./boostcpp.jam
>
> and they may end up staying there, i.e. not become part of any of the
> new submodules, candidates are :
>
> in addition bootstrap.(sh|bat) are part of the build system. With file
> structure changes in history for tools/build and tools/jam, these and
> their past siblings are surly broken.
>
> In addition files in lib/*/build/ directories that belong to the
> respective lib/* submodules does not go into build repository, right?
>
> Nevertheless I struggle with understanding the intension of some of the
> entries for the build repository in
> https://github.com/ryppl/Boost2Git/blob/master/repositories.txt so there
> may be intention to do more than the current converted repositories
> reflect and I expect here.
Sorry, I just don't know what to do with this input. I'm a bit
overwhelmed with everything going on in my life at the moment, so I
apologize.
Bjørn Roald
> on Fri May 24 2013, Bjørn Roald <bjorn-AT-4roald.org> wrote:
>
> > On Thu, 2013-05-23 at 23:06 -0700, Dave Abrahams wrote:
> >> on Thu May 23 2013, Bjørn Roald <bjorn-AT-4roald.org> wrote:
> >>
> >
> >> > Boost.Build would not
> >> > be the only submodule with dependencies to other submodules, would
> >> > it...
> >>
> >> Yeah, but as Boost.Build is really the *only* client of Boost.Jam, it's
> >> hard to justify separating them.
> >
> > I understand that point of view, but if historic Jam could be used by
> > itself, why not have them as separate modules?
>
> Because it makes everything harder. If the Boost.Build maintainers
> would prefer to have them separated, we can do that, but I advise
> against it.
Agree, its their choice. Especially in the future. My issue is more
about changing the past.
> >> > This build --> jam dependency is also gone in current boost,
> >>
> >> I don't think so:
> >> http://ci.boost.org/svn-trac/browser/trunk/tools/build/v2/engine
> >
> > I was not aware there is a single line of history here - in case it is
> > it may be worth trying to re-join the histories if that is done by the
> > conversion. but it seems odd to re-introduce tools/jam in
> > tools/build/jam, a new place where it has never been -- and move
> > tools/build to tools/build/build, where it has never been.
>
> You have to let go of the idea of the Boost super-repository having the
> actual historical paths to anything.
Ok, I can accept that, if that is how it is. It just escapes me what
the super-repository's purpose is then. But that is no big deal, I
guess it acts a holder of references to other boost code.
> >> > so I don't understand the need to bundle all in one repository. It
> >> > complicates the conversion and changes file structure in history.
> >>
> >> I don't know what you mean by "changes file structure in history" or how
> >> you conclude that it complicates the conversion.
> >
> > Well, are you saying I can check out any boost historic version from
> > http://github.com/boostorg/boost and run the instructions of that
> > release to successfully build?
>
> No. That *will not* be the case. For that, use SVN.
Or tar-balls I guess. This make sense as you say no global state of
history is kept or attempted to be kept, so that means no need to mangle
the history of the top level scripts in the modularized boost repo.
> > Those instructions will fail as structure of those historic commits
> > are changed beyond moving the headers. The changes in tools/build is a
> > prime example.
> >
> >> > It is better to try as far as feasible to make the conversion an
> >> > accurate reflection of history while getting a sensible set of
> >> > repositories representing modularized libraries. Build is no a library,
> >> > and as a tool the modularization is good enough with two historic
> >> > repositories and a build --> jam dependency in the past. Current build
> >> > repository is self contained if I understand this correctly.
> >>
> >> It's only self-contained because it contains jam in its "engine"
> >> directory. If Boost.Jam is going to be there in the present, its
> >> revision history should be there in the past, so you can follow it.
> >
> > so are you are saying you will have some commit the build repository
> > history show how files are moved from tools/build/jam where they never
> > have been to tools/build/build/v2/engine where they never went? Even if
> > you manage to make such a commit, how is that preserving file structure
> > in the history?
>
> I didn't say anything about preserving file structure in the history.
> What I am saying is that you should be able to follow the changes to any
> file that is part of Boost.Jam through its history,
Ok, In that case I follow the logic.
> and if those files
> are in the Build repository in the present but jump there from a different
> repository some time in the past, you won't be able to do that.
fair enough. I guess I expected more than was feasible when you had
added history to the modularized boost repositories. I understand the
need to restructure some, I guess it surprised me how many adaptations
has been done beyond the header file reorganization.
... snip...
>
> Sorry, I just don't know what to do with this input. I'm a bit
> overwhelmed with everything going on in my life at the moment, so I
> apologize.
no worries, I am happy with the replies given above and understands the
direction this is moving. I have mo major issues with the choices that
has been made.
Best regards,
--
Bjørn
Daniel James
>
> on Thu May 23 2013, Daniel James <daniel-AT-calamity.org.uk> wrote:
>
>> On 23 May 2013 14:47, Dave Abrahams <da...@boostpro.com> wrote:
>>>
>>> If that were going to be acceptable to the community, presumably
>>> everyone would have been happy with the original plan to graft an
>>> accurate SVN history when people want to refer to it, no? Please don't
>>> tell me that the last several months of work were wasted!
>>
>> What's acceptable to one person is quite different to what's
>> acceptable to another.
>
> No kidding!
as saying that it would be acceptable to the community.
>> It should ultimately be up to the maintainers of individual
>> modules.
>
> If anyone expects the Boost master repository to contain accurate
> submodule references through history, maintainers must not be allowed to
> delete old branches or rewrite old history.
Before the switch to the new layout, the master module would use that
for the submodule, aftewards it would use the working repo. Rather
than storing the historical record in a separate repo, it could be
stored in separate branches, if that's preferred. And I'm not
suggesting that every repository has such a split.
Btw. I think people will want to delete branches that aren't relevant
to their modules. It might be a good idea to tag them (say,
'svn-branch-name') so that the actual branches can be safely deleted.
>> Also, you would still have an accurate history in the historical
>> module(s). Which was the point, it was an attempt at a compromise
>> between two conflicting desires (to have an accurate history, and to
>> have a repository with a different directory layout).
>
> As I've said before, Git *cannot* represent exactly what SVN
> represents. If you want an accurate history, use SVN to get it. The
> best we can do with Git is to approximate actual history.
historical checkout would look equivalent to the corresponding
checkout from subversion.