on Sun Mar 31 2013, Stephen Kelly <steveire-AT-gmail.com> wrote:
> On Saturday, March 30, 2013 7:09:56 PM UTC+1, Dave Abrahams wrote:
>>
>>
>> on Thu Mar 28 2013, Stephen Kelly <steveire-AT-gmail.com> wrote:
>>
>> > Hi there,
>> >
>> > One of the annoying parts of trying to work on the boost cmake files
>> > is that small changes are impossible. I can't just change something
>> > in the generated cmake files for boost_any and boost_config,
>>
>> I'm sorry, but who's generating cmake files? As far as I know we are
>> creating cmake files entirely by hand.
>>
>
> I'm referring to the generation of files done by
> ryppl/cmake/Modules/RypplExport.cmake and testing modifications to that
> file.
OK. Sorry, I didn't design most of the CMake stuff in this system, so
I'm not really in a position to (usefully) comment on what makes working
on it annoying. That's really Daniel Pfeifer's department.
However, I do have a theory about what would make it much easier...
>> > The reason is that boost_config depends on boost_core, which depends
>> > on 35 other libraries:
>> >
http://thread.gmane.org/gmane.comp.programming.tools.ryppl.devel/9/focus=26
>> > So, I have to change 35 libraries if I want to try/test something.
>>
>> > On Thursday, March 28, 2013 4:05:07 PM UTC+1, Vladimir Prus wrote:
>> >
>> > Pardon my ignorance, but why do you need to *change* 35
>> > libraries? Or did you mean "checkout" or something else?
>> >
>> > Sorry, maybe I didn't make that clear.
>> >
>> > If I'm working on the ryppl.cmake file, how it works and what it
>> > does, then I'll need to also change the CMakeLists.txt files that
>> > include it. If it were possible to comment out everything except
>> > config and any, that would be easy, but if have to change 35
>> > libraries *just* to develop/test a change, it's easier for me to
>> > throw in the towel :).
>>
>> That would be a shame. Why don't you just use Ryppl.cmake to put
>> together some smaller representative examples?
>
> I guess that would be an option, but it would only delay real testing of
> the changes.
Actually, I think that's the problem. We don't have any "real testing"
for Ryppl. Real tests—as opposed to a single, really large (though
important) use-case—would make it much easier to make changes and know
that they're right. I, at least, would be very grateful if you could
write some test cases, and I'd set them up to run at
http://jenkins.boost.org.
>> On the other hand, a global search/replace in 35 CMakeLists.txt files
>> can't be that hard unless... (ahem) you don't have the right tools ;-)
Oh, c'mon, man! I deserve a "touché" for that quip, don't I????
>> > My boss has a catchphrase: "This is not a tools problem"…
And the appropriate version in this case is, "this is not a Ryppl
problem; it's a Boost problem." Boost is merely Ryppl's first, most
important customer. But Ryppl itself is not about Boost at all.
>> > My understanding is that the ryppl community already decided that the
>> > interdependencies between modules don't matter because ryppl will be
>> > configured to download all needed dependencies.
>>
>> Uhm, no. I think you have grossly misinterpreted the attitude of the
>> ryppl community.
>
> Great! Thanks for clearing this up. My understanding was coming from
>
>
http://thread.gmane.org/gmane.comp.programming.tools.ryppl.devel/9/focus=35
>
> and in particular "In the future, you won't clone the library with
> git, but setup a workspace with ryppl. Ryppl will clone the library
> *and* resolve dependencies." and "Ryppl will then create/update the
> CMakeLists.txt file in the top level." which indicated to me that all
> use of boost repos with cmake will require the ryppl tool for
> practical reasons. It also indicated to me that a sufficiently good
> tool makes any issues with interdependencies vanish.
Not *all* issues, but some of them. No matter how much the dependencies
of Boost are untangled I'm sure it will be extremely tedious to collect
all the dependendencies of some of these libraries without using a tool
like Ryppl.
> Thanks for clearing up that that's not the case.
>
>> Interdependencies certainly matter. Boost has too
>> many of them right now in part because the way we build and develop
>> Boost doesn't force people to confront them. By modularizing and making
>> the interdependencies explicit, we hope to encourage developers to
>> refactor. But we can only solve one problem at a time.
>
> Are you concerned about modularizing the repos first and the code second,
> and the effect that has on the history?
Not really.
> It's not really easy/practical to move files between different
> repos.
Sure it is. Of course you do end up with unpleasant discontinuities in
revision history. But from what I've seen, I guesstimated that most of
the dependency problems are not going to be solved by moving files, but
by splitting repositories into multiple parts.
That said, the module structure is being determined by
https://github.com/ryppl/Boost2Git/blob/master/repositories.txt and is
still (very) open to change. If you think files belong in different
repos, we can do that without disturbing Boost.
> It seems that with a 'one problem at a time' approach, the first
> problem that would need to be solved would be modularization of the
> code itself, not splitting into multiple repos. It seems you're saying
> that that would not be practical for social/community reasons?
It might be "practical" by some measure, but Boost is a big ship that
takes a long time to turn, so we're trying hard not to have Ryppl depend
on the decisions Boost makes (beyond switching to a modularized Git
structure).
> Any you're using split repos to force people to realize that the
> interdependencies exist?
Not exactly. Initially the split repos will still use the existing
Boost.Build procedure with a monolithic Boost "super-project." However,
integration with Ryppl forces you to declare your dependencies
explicitly. If/when Boost gets there, the dependency knots will become
much more apparent and it will be much easier to make a case for fixing
them.
> And you assume that will incentivize people to reduce such
> interdependencies?
Somewhat.
> Sorry for all the questions. Although I've been reading the boost
> mailing list for a few months, I'm not familiar with what it's like to
> try to get something done in the community when collaboration like
> that is needed.
Questions are good.
>> > However, in my view, this is not a tools problem. I'm sure many
>> > circular-dependency issues in boost modules can be solved. The ryppl
>> > tool can be helpful, but making it mandatory doesn't seem right.
>>
>> Making what mandatory?
>
> The ryppl tool. See the above quotes about how in the future it would be
> needed to clone the right 35 repos and generate a CMakeLists.txt file.
You can always do it manually, but, as I said, it would be tedious.
Even if you have only 5 dependencies. Don't forget, Ryppl will encode,
and use, version dependency information too.
>> > The result is not a 'modular boost' if all of the modules are
>> > interdependent.
>>
>> Of course not. But they're not *all* interdependent. There are,
>> unfortunately, a few strongly-connected components in the graph.
>>
>> > If all the interdependencies between the headers are going to remain,
>> > then why not put the 'core mesh' into one repo? If you want one of
>> > the 35 libraries in the list, you need all of them, right?
>>
>> If we do that, the dependencies would never become untangled.
>
> Indeedm but I didn't know that the plan was to modularize the code at all
> :).
I hope Boost will do that. I will encourage Boost to do that. But
Ryppl is a separate project.
> KDE is going through a similar modularization over the past couple of
> years,
I know; we're using a fork of their tool.
> which I've been involved with. We're modularizing the code directly in
> the kdelibs repo, and we'll eventually split it up into multiple
> independent git repos *after* we've done the modularization of the
> code.
BTW, I've found some serious bugs in their tool which I've reported but
it seems like the reports were ignored.
> I assumed, given the above quote again, and given my own opinion of the
> right order to do things (from my experience in the KDE efforts), that if
> you were splitting now, then you have already done all the modularization
> you intend to do. I can accept that that's not the case. Thanks for
> clearing that up.
It's not that simple. You can feel free to pursue things in the order
you think best. *I* barely have time for the Ryppl side of things.
>> > But a better question is why not remove the interdependencies?
>>
>> Good question. It would be great if you could do that!
>>
However, it has to be said, being able to handle awful dependency knots
is an important test case for Ryppl if people are going to use it to
distribute existing code.
>> > This seems like it should have been a prerequisite to splitting the
>> > code into multiple git repos, so I'm assuming the approach of
>> > actually modularizing the code was already considered and rejected
>> > for some reason?
>>
>> Individual maintainers "own" the individual libraries, and it seemed
>> like too much to take on negotiating all these changes at the same time
>> as doing the tools work.
>
> This is a little worrying :). It seems like something with obvious
> benefits to me, so I'd expect collaboration, not negotiation, unless
> some of the individual owning maintainers are opposed to the effort
> for some reason?
I don't know. I expect that such reorganization could require a lot of
work in some of the libraries, whose maintainers may not have a lot of
time right now, and I don't want it to be a blocker for Ryppl.
> If you would have do time-consuming negotiations, what chance do I
> have? :) I have no credibility in the boost community. I also don't
> have any familiarity with the codebase.
>
> I'm interested though, so I'll start a discussion there.
If you discover the changes that need to be made in the code, I'll be
happy to lend my credibility in support of your proposals to make the
changes.
--
Dave Abrahams