Rearranging SVN and separating the build scripts

[Jonathon Rossi]

I have a few questions before I start.

1. Will anyone object to me rearranging the SVN in the layout we have been talking about? The main reason I am asking is because it will make merging changes made in branches more difficult and we will have a broken build for a few days because the changes are too difficult to make in 1 commit.
2. Do we want to move DynamicProxy and NVelocity to the root or should they stay in the Tools directory?
3. All projects in /Components, /Facilities, /Services and /Tools should have their own trunk/tags/branches; do /Experiments need the same?
4. Should we drop the Samples directory in favour of storing the samples in the doc directories in each project?
5. Do we still need the 1.0.x, ComponentBurden, NH2.0 and Security branches?
6. Should we just drop the RC2, RC3 and Jan-06 tags? If someone still wants either of them they can get it from a revision before the changes.
7. Do we want a separate strong name key per project?
8. Do we want the common.xml and common-project.xml files in each project directory or is there a better way? Should we just have separate build scripts and nothing in a common directory?
9. Do we want the default.build in the root to build all projects in the trunk directories for now, which would be able to update the binaries in each project's lib directory with the dependant library?
10. What should the layout of the project lib directories look like since our current shared libraries directory has /mono, /net, /net/2.0, /net/3.5. I assume we want to have the same layout for each project, but only the required binaries, not everything.

Summary of the proposed layout:
/
  MonoRail
    trunk
      src
      lib
      doc
    tags
    branches
      1.0.4
        src
        lib
        doc
  ActiveRecord
    trunk
    tags
    branches
  Components
  Core
  Experiments
  Facilities
  InversionOfControl
    trunk
      Castle.MicroKernel
      Castle.Windsor
  MonoRail2
  Samples
  Services
  Setup
  SharedLibs
  Tools
    Castle.DynamicProxy2
      trunk
      lib
      doc
    NVelocity
  site
  docs (used for the API docs)

--
Jonathon Rossi

[Jonathon Rossi]

We also said we were dropping the VS2005 projects and solutions so I'll do that as I go.

--
Jonathon Rossi

[hammett]

On Sat, Nov 8, 2008 at 12:35 AM, Jonathon Rossi <jo...@jonorossi.com> wrote:
> 1. Will anyone object to me rearranging the SVN in the layout we have been
> talking about? The main reason I am asking is because it will make merging
> changes made in branches more difficult and we will have a broken build for
> a few days because the changes are too difficult to make in 1 commit.

I'm not sure the pros outweigh the cons with this change.
The pros:
- all projects will be self contained into a proper svn structure
(trunk, branches, tags)

The cons:
- Checking out /svn/castle/ will be hell
- All urls pointing to files on the current structure will be broken
- build files will be broken

I'd encourage that we take small steps and see how it goes. Like,
bringing the docs folder to each project folder. Then going from
there.

> 2. Do we want to move DynamicProxy and NVelocity to the root or should they
> stay in the Tools directory?

I'd say stay in Tools.

> 4. Should we drop the Samples directory in favour of storing the samples in
> the doc directories in each project?

The problem is that some samples encompass more than a single project.

> 6. Should we just drop the RC2, RC3 and Jan-06 tags? If someone still wants
> either of them they can get it from a revision before the changes.

Is it a problem to keep them around?

> 7. Do we want a separate strong name key per project?

Nope

--
Cheers,
hammett
http://hammett.castleproject.org/

[Jonathon Rossi]

I'm not sure the pros outweigh the cons with this change.
The pros:
- all projects will be self contained into a proper svn structure
(trunk, branches, tags)

If we don't have these it would make it very difficult to release each project separately because you can't branch or tag a single project.

The cons:
- Checking out /svn/castle/ will be hell

We can always just make a batch script for checking out just the trunk of the projects. I currently have the full castle repo checked out from the root and it is less than 1GB.

- All urls pointing to files on the current structure will be broken

Do you mean from blog posts and the like?

- build files will be broken

Yes, we brought up the idea of having separate builds in cruisecontrol.net and separate build drops. Maybe this is overkill for every single project. Maybe groups like facilities should become a bundle to keep the overhead down.

 

I'd encourage that we take small steps and see how it goes. Like,
bringing the docs folder to each project folder. Then going from
there.

I thought that bringing the documentation into each project would be one of the last things because the /site directory can stay at the root for now and still work. Following on from your suggestion, maybe the best place to start is by introducing the src, lib and doc directories into each project, update the build scripts to match and drop the SharedLibs directory. This would start to give us projects that are more separated.

> 4. Should we drop the Samples directory in favour of storing the samples in
> the doc directories in each project?

The problem is that some samples encompass more than a single project.

That's what I thought, maybe we just need to nominate what the sample best shows off and then point to the sample from other projects.

> 6. Should we just drop the RC2, RC3 and Jan-06 tags? If someone still wants
> either of them they can get it from a revision before the changes.

Is it a problem to keep them around?

I guess not, we can just leave them at the root.

--
Jonathon Rossi

[hammett]

On Sat, Nov 8, 2008 at 8:45 PM, Jonathon Rossi <jo...@jonorossi.com> wrote:
> If we don't have these it would make it very difficult to release each
> project separately because you can't branch or tag a single project.

What prevent us from having something like

/svn/castle/branches/DynamicProxy/ all dp branches here
/svn/castle/tags/DynamicProxy/ all dp tags here

> We can always just make a batch script for checking out just the trunk of
> the projects. I currently have the full castle repo checked out from the
> root and it is less than 1GB.

So you agree that the default experience will be painful.

> Do you mean from blog posts and the like?

Yes

> I thought that bringing the documentation into each project would be one of
> the last things because the /site directory can stay at the root for now and
> still work. Following on from your suggestion, maybe the best place to start
> is by introducing the src, lib and doc directories into each project,

Isnt that going to lead to duplication of dlls? I'm concerned that
updating say, NHibernate.dll would force me to update a bunch of
places.

[Jonathon Rossi]

Branching and tagging as you suggested is probably the best compromise of both ways. It gives you the ability to branch just one project or everything if you know you are going to be making changes all over the place.

Yes, it would mean updating NHibernate.dll in several places. It probably isn't a case we want to support, because you can't really have 2 different NHibernate.dll used by different projects if you want to use both of those projects.

After thinking about it more I think you are right in saying it is best to keep it as it is now. Keeping it all together will actually make building the web site a lot easier too.

--
Jonathon Rossi

[Jonathon Rossi]

I just realised I missed 2 things.

If you branch anywhere in the tree other than from the root (/castle/trunk) with the build scripts, then you won't be able to build any projects because your branch would be missing these files.

I think the main reason we need the lib directory is so that we can build project's against released versions of other castle projects, otherwise we are back to how we are now with not being able to release project's separately because they are all built against each other's trunk.

--
Jonathon Rossi

[hammett]

Agreed, but the problem hasn't surfaced. Let's deal with it when it
becomes a concrete problem.

[Jonathon Rossi]

Sure

--
Jonathon Rossi

[beep...@googlemail.com]

I'll start with a huge warning: I AM NOT VERY GOOD!
I'm not particularly experienced at Castle, nor do I know SVN very
well, but what the heck, I'l chuck in my thoughts! The worst that can
happen is I get ignored :-D

Can you not use the svn:externals (http://bit.ly/15Y8X) functionality
for this sort of situation? As I said, I don't know SVN very well, but
I've seen it used on projects like MsSqlSpatial (http://bit.ly/xH74)
so could each Castle project have it's own SVN repository, linking to
the relevant external ones where required (including a global "shared
libs"project, or even linking externally to the correct revision of
the external project if possible) and a final repository which is
essentially an amalgamation of all the other repositories with a build
script that calls all the other build scripts. Or something!

Just my 2p's worth.
Thanks for reading.

Ben

On 9 Nov, 20:51, "Jonathon Rossi" <j...@jonorossi.com> wrote:
> I just realised I missed 2 things.
>
> If you branch anywhere in the tree other than from the root (/castle/trunk)
> with the build scripts, then you won't be able to build any projects because
> your branch would be missing these files.
>
> I think the main reason we need the lib directory is so that we can build
> project's against released versions of other castle projects, otherwise we
> are back to how we are now with not being able to release project's
> separately because they are all built against each other's trunk.
>
>
>
> On Sun, Nov 9, 2008 at 6:33 PM, Jonathon Rossi <j...@jonorossi.com> wrote:
> > Branching and tagging as you suggested is probably the best compromise of
> > both ways. It gives you the ability to branch just one project or everything
> > if you know you are going to be making changes all over the place.
>
> > Yes, it would mean updating NHibernate.dll in several places. It probably
> > isn't a case we want to support, because you can't really have 2 different
> > NHibernate.dll used by different projects if you want to use both of those
> > projects.
>
> > After thinking about it more I think you are right in saying it is best to
> > keep it as it is now. Keeping it all together will actually make building
> > the web site a lot easier too.
>

> > On Sun, Nov 9, 2008 at 3:19 PM, hammett <hamm...@gmail.com> wrote:
>
> >> On Sat, Nov 8, 2008 at 8:45 PM, Jonathon Rossi <j...@jonorossi.com>

[Jonathon Rossi]

I think we wanted to avoid having separate repositories because it adds a lot of management and performance overhead.

Using svn externals will probably make things more difficult to know what revisions things are running from. It works well for 3rd party libraries but I don't think it would work very well for castle. Well that's my opinion, anyway.

--
Jonathon Rossi

[Symon Rottem]

Externals used against a tag might work.  When a new release of a component is made the svn:externals link could be updated to the new tag.

Symon.

--
Symon Rottem
http://blog.symbiotic-development.com

[Jonathon Rossi]

I'm not sure how svn externals provides anything better than storing the pre-built released binaries in a lib directory.

--
Jonathon Rossi

[Markus Zywitza]

svn:external doesn't require us to change the build script because the released project is still there like it is now, only that it points to a released version.

 

-Markus

2008/11/10 Jonathon Rossi <jo...@jonorossi.com>

[Jonathon Rossi]

Maybe I don't understand svn externals correctly. How can I have a working copy with all of castle, but still be able to build projects against tagged revisions. Won't the tagged revisions replace the trunk of the other projects in my working copy. That still means you have to rearrange the repository so that each project can have a working copy (tagged) of the source of all dependant projects. Won't the amount of source you have to checkout blow out in size?

--
Jonathon Rossi

[Markus Zywitza]

The idea is following:

 

When I release AR, I'll branch the trunk to ActiveRecord/1.0 and add appropriate tags. I also create a branch ActiveRecord/trunk for 2.0 development.

 

In the old trunk I can now remove AR and replace it by svn:external link to AR1.0 tagged revision. Once AR 1.1 is released, I will switch the externals to that revision.

 

If hammett does the same for MR (which has a dependency to AR), he will branch the trunk and have its own copy of the svn:external links to AR. He must decide when I update the trunk externals whether he merges the change to the MR 1.0 and MR trunk branches..

 

This way, everybody can work on his own branches and "project trunks" and the current trunk will stay as a reference checkout and build location.

[Ken Egozi]

I used the same approach in the past with git. (it's called submodules and Lee was the one pointing that out to me). It's looks like the best and simplest solution.

--
Ken Egozi.
http://www.kenegozi.com/blog
http://www.musicglue.com
http://www.castleproject.org
http://www.gotfriends.co.il

[Jonathon Rossi]

Interesting, I never thought of doing it like that.

Just to make sure I got my head around it, do you work in a branch or in the trunk?

--
Jonathon Rossi

[Markus Zywitza]

In branches. The trunk as it is now, will be consisting of build scripts and svn:externals only when all projects has been releases.

[Brian DeMarzo]

I like Markus's approach. I certainly seems to allow things to stay
the way they are (one unified build) while allowing people to also
look at each project separately.

In a sense, Castle is now a family of projects that can be used
together very easily, but can also be used separately. Keeping both of
that will probably help each project, while also keeping the whole
intact.

In other words, you won't need to visit the whole castle if you only
want to visit a few of the battlements -- but you could if you wanted
to.


- b