Moving uP3 to SVN
dmccallum
Recently the clearinghouse list has been discussing the possibility of
moving the uP3 source tree out of CVS and into SVN. I'd like to surface
that discussion to this list and ask for an up-or-down vote on the
issue, as well as solicit feedback on the specific steps I've proposed
for executing the migration. Talking with Andrew Petro yesterday, we
thought that it might be a good idea to target the migration for some
time Wed - Fri of next week when we'll have a high concentration of
interested developers gathered in one place.
To be clear, we're only talking about moving uP3 source at this point.
It's a good guinea pig: it's still sandboxed, its tagging and branching
are straightforward and current commit traffic is light. I've gone
through a dry run of the process on my workstation... cvs2svn
successfully generates a SVN dump file, which loads into SVN with what
looks to me like a perfectly valid log history. The dump preserves the
three milestone tags. If you'd like to take a look at the end result
yourself, the dump is available at:
URL: ftp://ftp.unicon.net/up3.dump
User: up3
(Note that this dump dates back to early August.)
Here's my shot at a high-level blow-by-blow for making it happen in the
real world:
1) Announce our intentions to jasig-...@unm.edu, jasi...@unm.edu,
jasig-upo...@googlegroups.com and
http://www.uportal.org/cvs.html/ and set a target cutover date. Again,
I'm thinking that late next week would be ideal. Jason Shao has
suggested that we identify an admin/support contact for project
migrations. I can volunteer to help support the uP3 migration, but I
think it's probably best if I were assisted by someone with more
historical knowledge of the JA-SIG hosting environment and established
standing in the JA-SIG community. I'm also willing to assist with
migrating any other JA-SIG project, with the same caveat and the
understanding that I need to make uP3 my near-term priority.
2) Get a SVN repo up and running behind Apache to verify SSL, authn/z
and any other configuration before attempting anything more
interesting. If necessary, configure backup processes to include SVN
data.
3) Grab the cvs2svn tarball. Can either install it or run it directly
from the exploded archive.
4) Take a backup of the CVS file system. (Not strictly necessary for
what we're doing, but better paranoid than sorry.)
5) Generate the SVN dump file. I found I needed to use the --use-cvs
switch.
We're just dumping uP3 code, so the command ends up looking something
like:
% cvs2svn --use-cvs --dump-only --dumpfile <dump-file>
<cvs-backup-home>/sandbox/up3
6) Load the dump into SVN. Assuming we want a top-level uP3 module with
the usual trunks, branches and tags dirs below that, the command should
look something like:
% svnadmin --parent-dir uP3 load <path-to-repo> < <dump-file>
7) Verify the load, then delete the uP3 module in SVN. This was just a
trial run.
8) On the appointed date, throw CVS into a read-only state (I'll humbly
defer someone with more CVS admin experience on the best approach.
Possible options might include a global commit refusal script or an
empty 'writers'
file). Repeat steps 4 - 6.
9) Re-enable CVS commit access. Move uP3 source into the attic with an
informative log message. Add a readme explaining where to find the
SVN'd source.
10) Announce completion to the channels notified in step 1.
Jason also suggested that we take a vote on a URL for the JA-SIG SVN
service. I'll take this opportunity to suggest
'https://developer.ja-sig.org/svn' to represent a global repository for
all JA-SIG SVN projects. uP3 code would be a first-class project, with
its trunk accessible at: 'https://developer.ja-sig.org/svn/uP3/trunk'.
I look forward to your feedback.
- Dan McCallum
Unicon, Inc
John Lewis
Eric Dalquist
on the clearing house box correct? Who will be the administrator for the
service?
As long as those two are figured out +1
-Eric
> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups "jasig-uportal3-dev" group.
> To post to this group, send email to jasig-upo...@googlegroups.com
> To unsubscribe from this group, send email to jasig-uportal3-...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/jasig-uportal3-dev
> -~----------~----~----~----~------~----~------~--~---
>
>
bolto...@gmail.com
> Content-Type: application/x-pkcs7-signature
> Content-Transfer-Encoding: base64
> Content-Disposition: inline;
> filename="smime.p7s"
> Content-Description: S/MIME Cryptographic Signature
> X-Google-AttachSize: 4462
Brad Szabo
Dan McCallum
I'm make no claims to be the world's greatest sys admin, but I will volunteer to take responsibility for the JA-SIG SVN service.
- Dan
Andrew Petro
> moving the uP3 source tree out of CVS and into SVN. I'd like to surface
> that discussion to this list and ask for an up-or-down vote
> To be clear, we're only talking about moving uP3 source at this point.
Eric Dalquist
clearinghouse mailing list? If not lets get you on there to coordinate
the install and migration.
-Eric
Dan McCallum wrote:
> Unless another box is available and/or we have a good reason to start
> looking for another box (like, ahem, running out of disk space every
> so often), I'd say sure, let's target the clearinghouse box.
>
> I'm make no claims to be the world's greatest sys admin, but I will
> volunteer to take responsibility for the JA-SIG SVN service.
>
> - Dan
>
> On 9/7/06, *Eric Dalquist* <eric.d...@doit.wisc.edu
> >> URL: ftp://ftp.unicon.net/up3.dump <ftp://ftp.unicon.net/up3.dump>
> >> User: up3
> >>
> >> (Note that this dump dates back to early August.)
> >>
> >> Here's my shot at a high-level blow-by-blow for making it
> happen in the
> >> real world:
> >>
> >> 1) Announce our intentions to jasig-...@unm.edu
> <mailto:jasig-...@unm.edu>, jasi...@unm.edu
> <mailto:jasi...@unm.edu>,
> >> jasig-upo...@googlegroups.com
> <mailto:jasig-upo...@googlegroups.com> and
Dan McCallum
- Dan
Jason Shao
+1 and I can probably free up some time to assist with implementation
on the Clearinghouse side in terms of any support you might need
there.
2 points.
1- currently we only have a certificate for www.ja-sig.org NOT
developer.ja-sig.org - so if the connection needs to be over SSL
(seems reasonable) we either need to use a self-signed, or purchase
another commercial certificate.
2- Echo Eric's point -- who is responsible for owning the service in
the longer term - e.g. upgrading the SVN install, checking backups,
managing accounts, etc?
Jason
Dan McCallum
But, I took a look at the clearinghouse box this morning and it looks like it's running Apache 1.3. To run SVN behind mod_dav we'd need Apache 2.x. I'd like to ask if anyone has any opinions around this issue. We have a few options:
1) Migrate 'everything' to Apache 2.x
2) Run Apache 1.3 and 2.x side-by-side, with the 2.x instance listening on some non-standard port for developer.ja-sig.org SSL connections
3) Worry about http/s later, and just expose access over svn+ssh. Something like Fisheye or WebSVN (http://websvn.tigris.org/) could provide web browsing
I don't know that there's really much to be gained or lost by going in one direction or another -- the SVN book lists the tradeoffs ( http://svnbook.red-bean.com/nightly/en/svn.serverconfig.html#svn.serverconfig.overview) -- but if our primary goal is just to get developers up and running, kicking SVN tires, option 3 is probably the shortest route to that goal. On the other hand, there's something to be said for the off-the-shelf (anonymous!) browsability of a DAV-fronted repo and unless we have a very good reason to start publishing scary svn+ssh repo URLs, it may be worth the up-front effort to get an Apache 2.0 instance working from the start. Since the choice of access URLs does affect file-system permissions for the repository itself, I'd vote to pursue up-front the approach we intend to take in the long term, which I assume to be Apache-powered DAV over https. If we do go this route, though, we'll need to deal with the issue of one Apache hosting SSL access to developer.ja-sig.org and another serving up port 80 access. Depending on the availability of IP addresses and the ease of making DNS configuration changes, I'd guess the simplest thing would be to request a new name and IP for the SVN service. So instead of developer.ja-sig.org/svn, we'd host the SVN service behind something like svn.ja-sig.org. Alternatively, we could configure the Apache 2.x instance to handle SSL requests on port XXXX and redirect https://developer.ja-sig.org/svn to https://developer.ja-sig.org:XXXX/svn, although I have not experimented with SVN clients to see how well they handle such redirects.
Please let me know if you have preferences or guidance on these issues.
Jason, thanks for your offer of clearinghouse assistance. I think the first steps to take, no matter what we eventually decide w/r/t Apache, will be to announce a target cutover date (I'm thinking Thursday night this week) and get SVN on that box so we can attempt a dry-run migration as soon as possible. With those goals in mind, I'd like to ask for your help on a few points:
1) Can you point me in the right direction in terms of negotiating whatever processes exist for putting new software on that box as well as updating http://www.uportal.org/cvs.html with an outage/migration notification?
2) Is Thursday night going to be a good timeslot for cutting uP3 over to SVN or are there standard maintenance windows for these sorts of things? We will at least incur a brief CVS commit outage and possibly an Apache outage depending on what we decide to do for an SVN access method.
3) What kind of existing backup infrastructure can we leverage for archiving our SVN repo?
I'm willing to own the SVN service for as long as I'm able or until somebody else decides they'd really rather take ownership themselves.
- Dan