[gentoo-dev] Moving packages to dev-vcs
Sebastian Pipping
So now that we have a new category "dev-vcs" we need to move suitable
stuff over there. Moving packages is complex and error prone:
This mail tries to guide you through and summarize the process, please
read on.
HINT: Please keep CVS' radius of operation small to reduce risks.
0. Prepare
==========
- Check if your package fits:
Is it a "utility focused on version control"?
- Ensure that you have permission (if you are not the main maintainer)
- Sync the whole tree including profiles
(as you'll be inspecting reverse dependencies)
1. Copy
=======
- Duplicate any traces of dev-util/${PN} in profiles/ to dev-vcs/${PN}
(fgrep -Rw "dev-util/${PN}" profiles/)
- Copy complete package dev-util/${PN} to dev-vcs/${PN}
(watch out CVS directories)
2. Switch
=========
- Update eclasses
fgrep -w "dev-util/${PN}" eclass/*.eclass
- Update reverse dependencies
http://tinderbox.dev.gentoo.org/misc/dindex/dev-util/${PN}
http://tinderbox.dev.gentoo.org/misc/rindex/dev-util/${PN}
^^^
fgrep -w "dev-util/${PN}" */*/*.ebuild
- Append "move dev-util/${PN} dev-vcs/${PN}" line to
profiles/updates/1Q-2010
(must come last, described in [1])
3. Remove
=========
- Remove package dev-util/${PN} from CVS (as in [2])
cd dev-util
cvs rm -Rf ${PN}
cvs ci -m "dev-util/${PN}: Remove (renamed to dev-vcs/${PN})" ${PN}
^^^^^
- Remove any traces of dev-util/${PN} in profiles/
4. Notify
=========
- Report back problems with this process
- Mail fellow maintainers of dev-util/${PN} about the move
- If ${PN} is a big one (Subversion, Git, you know the list)
- Update documentation (now or open a bug at least, please)
- Drop sping@g.o. a mail to update references in Layman
- (Notify users and developers through Planet Gentoo?)
5. Final words
==============
- Help for filling "profiles/updates" documentation is wanted,
it's currently _all empty_:
http://devmanual.gentoo.org/profiles/updates/index.html
- Feel free to CC yourself to dev-vcs bug #56967 [3]
Sebastian
[1]
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=5#doc_chap6
[2]
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=5#doc_chap7
[3] https://bugs.gentoo.org/show_bug.cgi?id=56967
Ulrich Mueller
> 2. Switch
> =========
> - Update reverse dependencies
> http://tinderbox.dev.gentoo.org/misc/dindex/dev-util/${PN}
> http://tinderbox.dev.gentoo.org/misc/rindex/dev-util/${PN}
> ^^^
> fgrep -w "dev-util/${PN}" */*/*.ebuild
Also check metadata.xml which may contain package references.
For example (from app-portage/layman/metadata.xml):
<flag name='git'>Support <pkg>dev-util/git</pkg> based overlays</flag>
Ulrich
Markos Chandras
guess
--
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org
Robin H. Johnson
> So now that we have a new category "dev-vcs" we need to move suitable
> stuff over there. Moving packages is complex and error prone:
> This mail tries to guide you through and summarize the process, please
> read on.
"cvs add" and the matching commit aren't mentioned anywhere...
--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : rob...@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85
Dirkjan Ochtman
> This contains a critical bug...
>
> "cvs add" and the matching commit aren't mentioned anywhere...
Well, it *is* a summary.
Thanks for the guide, that'll be useful.
Cheers,
Dirkjan
Brian Harring
> =======
> - Duplicate any traces of dev-util/${PN} in profiles/ to dev-vcs/${PN}
> (fgrep -Rw "dev-util/${PN}" profiles/)
>
> - Copy complete package dev-util/${PN} to dev-vcs/${PN}
> (watch out CVS directories)
>
>
> 2. Switch
> =========
> - Update eclasses
> fgrep -w "dev-util/${PN}" eclass/*.eclass
>
> - Update reverse dependencies
> http://tinderbox.dev.gentoo.org/misc/dindex/dev-util/${PN}
> http://tinderbox.dev.gentoo.org/misc/rindex/dev-util/${PN}
> ^^^
> fgrep -w "dev-util/${PN}" */*/*.ebuild
I'd suggest
pquery --revdep dev-util/${PN} --raw --repo $PORTDIR
instead- throw in a '--attr inherited' if you want to look for
eclasses that are implicated in addition, but it's a safer bet then
relying on fgrep (since it's literal metadata searching) and doesn't
have the potential for staleness tinderbox does. Speed wise it'll be
slower then grep, but not hugely so- reasonably up to date cache, it's
<9s on my hardware so... there's no reason to not do it properly.
> - Append "move dev-util/${PN} dev-vcs/${PN}" line to
> profiles/updates/1Q-2010
> (must come last, described in [1])
>
>
> 3. Remove
> =========
> - Remove package dev-util/${PN} from CVS (as in [2])
> cd dev-util
> cvs rm -Rf ${PN}
> cvs ci -m "dev-util/${PN}: Remove (renamed to dev-vcs/${PN})" ${PN}
Realistically, the dev *really* should be doing visibility scans
after their efforts- yes bones does them too, but it's better to do
the scan after to ensure any breakage is caught quickly. Specifically
via pkgcore-checks, I'd suggest running
pcheck -r $PORTDIR -c visibility '*'
prior to the move, and a dump of that commands output after the cvs rm
invocation above. Presuming everything was updated properly, the
line count should be the same- output will vary slightly (previous
complaints about dev-util/bzr being unavailable become dev-vcs/bzr
unavailable specifically).
At least for my hardware, it's a 3.5 minute scan- so there really
isn't a reason to *not* do this check (if the mips profiles were
cleaned up/out prior it's more like 2.5 minutes on a sidenote)...
If folks are aware of alternate tools for doing these sort of checks
in a timely fashion, feel free to pipe up also- I'm just suggesting
the tools I know.
Random sidenote, anyone looked at using an alternate vcs to do the
work, then proxy it back? Specifically thinking of workflow like svk
(or in this case hg cvs,
https://wiki.mozilla.org/Using_Mercurial_locally_with_CVS ). The
reason I ask is that via building the work up outside of cvs, then
proxying the add/remove/modifications back into it, it should be
possible to minimize the window of cvs breakage down to bare minimum
while still getting the same level of QA validation for the changes.
That's assuming hg/cvs behaves sanely of course, and doing a checkout
(tip as it were) from cvs into hg is fast enough to be viable. Just a
thought- djc, you got any comments on this usage of hg?
~harring
Sebastian Pipping
> This contains a critical bug...
>
> "cvs add" and the matching commit aren't mentioned anywhere...
That's a valid complaint, yes.
I left it out knowingly, maybe not for the better.
Sebastian
Ryan Hill
Sebastian Pipping <sp...@gentoo.org> wrote:
> So now that we have a new category "dev-vcs" we need to move suitable
> stuff over there. Moving packages is complex and error prone:
> This mail tries to guide you through and summarize the process, please
> read on.
>
> [stuff]
Also search through our documentation and fix any package references.
--
fonts, by design, by neglect
gcc-porting, for a fact or just for effect
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
Serkan Kaba
--------
repoman: dev-vcs is not an official category. Skipping QA checks in
this directory.
Please ensure that you add dev-vcs to
/home/firari/Desktop/çalışma/gentoo/gentoo-x86/profiles/categories
if it is a new category.
-----
After hitting it for the first time I updated the whole profiles dir and
still failing
Thanks in advance.
--
Sincerely,
Serkan KABA
Gentoo Developer
Christian Faulhammer
Serkan Kaba <ser...@gentoo.org>:
> I'm hitting a repoman failure
> --------
> repoman: dev-vcs is not an official category. Skipping QA checks in
> this directory.
> Please ensure that you add dev-vcs to
> /home/firari/Desktop/çalışma/gentoo/gentoo-x86/profiles/categories
> if it is a new category.
> -----
> After hitting it for the first time I updated the whole profiles dir
> and still failing
It is in profiles/categories, so everything should be fine.
V-Li
--
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode
Serkan Kaba
additionally) and the issue was fixed when I syced.
2010/3/5 Christian Faulhammer <fa...@gentoo.org>:
Jeroen Roovers
Sebastian Pipping <sp...@gentoo.org> wrote:
> 4. Notify
> =========
> - Report back problems with this process
>
> - Mail fellow maintainers of dev-util/${PN} about the move
>
> - If ${PN} is a big one (Subversion, Git, you know the list)
> - Update documentation (now or open a bug at least, please)
> - Drop sping@g.o. a mail to update references in Layman
> - (Notify users and developers through Planet Gentoo?)
This step should probably include correcting all open bug
reports' Summaries to point to the new category, so that CAT/PN can be
found using the "simple" search interface.
Regards,
jer
Sebastian Pipping
>> 4. Notify
>> =========
[..]
>
> This step should probably include correcting all open bug
> reports' Summaries to point to the new category, so that CAT/PN can be
> found using the "simple" search interface.
Good catch. Thanks for fixing those on monotone (and others I guess).
Sebastian
Sebastian Pipping
> Random sidenote, anyone looked at using an alternate vcs to do the
> work, then proxy it back? Specifically thinking of workflow like svk
> (or in this case hg cvs,
> https://wiki.mozilla.org/Using_Mercurial_locally_with_CVS ). The
> reason I ask is that via building the work up outside of cvs, then
> proxying the add/remove/modifications back into it, it should be
> possible to minimize the window of cvs breakage down to bare minimum
> while still getting the same level of QA validation for the changes.
Assuming the person using such a tool
- is fluent in using such tool
(to at least compensate extra abstraction. with plain CVS you
at least know for sure what's happening)
- is manually doing extra commits for manifest fixes
(like "repoman commit" would do for him otherwise)
Sebastian
Sebastian Pipping
> After the move is done could you please come up with a list of all the
> things you needed to take into account and then work with me for example
> to get it included in devmanual.
Good idea.
I opened a bug for it so we don't forget about it.
https://bugs.gentoo.org/show_bug.cgi?id=308151
Sebastian
Fabian Groffen
> 3. Remove
> =========
> - Remove package dev-util/${PN} from CVS (as in [2])
> cd dev-util
> cvs rm -Rf ${PN}
> cvs ci -m "dev-util/${PN}: Remove (renamed to dev-vcs/${PN})" ${PN}
> ^^^^^
> - Remove any traces of dev-util/${PN} in profiles/
Please take masks for packages you move into account for /all/ profiles.
--
Fabian Groffen
Gentoo on a different level
Sebastian Pipping
Still to be moved
=================
dev-util/aegis
dev-util/cola
dev-util/colorsvn
dev-util/cvs
dev-util/cvs2cl
dev-util/cvs2svn
dev-util/cvsd
dev-util/cvsgraph
dev-util/cvsps
dev-util/cvsq
dev-util/cvsutils
dev-util/easygit
dev-util/fossil
dev-util/git
dev-util/gitosis
dev-util/gitosis-gentoo
dev-util/hgsvn
dev-util/qgit
dev-util/rapidsvn
dev-util/subversion
dev-util/svk
dev-util/svn2cl
dev-util/tig
dev-util/tkcvs
Already in place
================
dev-vcs/archway
dev-vcs/bzr
dev-vcs/bzr-explorer
dev-vcs/bzr-gtk
dev-vcs/bzr-rewrite
dev-vcs/bzr-svn
dev-vcs/bzrtools
dev-vcs/colorcvs
dev-vcs/cssc
dev-vcs/cvsspam
dev-vcs/darcs
dev-vcs/giggle
dev-vcs/gitg
dev-vcs/git-sh
dev-vcs/guilt
dev-vcs/hg-git
dev-vcs/hgsubversion
dev-vcs/hgview
dev-vcs/kdesvn
dev-vcs/mcvs
dev-vcs/mercurial
dev-vcs/monotone
dev-vcs/mr
dev-vcs/qbzr
dev-vcs/qct
dev-vcs/qsvn
dev-vcs/rcs
dev-vcs/statcvs
dev-vcs/statsvn
dev-vcs/stgit
dev-vcs/svn2git
dev-vcs/svnmailer
Robin H. Johnson
> dev-util/cvs
> dev-util/cvs2cl
> dev-util/cvs2svn
> dev-util/cvsd
> dev-util/cvsgraph
> dev-util/cvsps
> dev-util/cvsq
> dev-util/cvsutils
> dev-util/gitosis
> dev-util/gitosis-gentoo
For myself as the Git maintainer, and the main cvs-utils guy, feel free
to move all of the above. Just shoot me a reminder after you do, because
part of the infra cfengine stuff will start spewing errors at me (it
doesn't handle package moves on it's own, so I need to update it).
Sebastian Pipping
>
> "cvs add" and the matching commit aren't mentioned anywhere...
I've been moving Git today, properly hopefully.
This is what I've been doing on the shell basically:
cd gentoo-x86
# Copy package to new location
cd dev-vcs
# Add empty dirs to please CVS
for dir in $(find ../dev-util/git/ -name CVS -prune -or -type d
-printf 'git/%P\n'); do
mkdir ${dir}
cvs add ${dir}
done
cd git
rsync -v --recursive --archive --cvs-exclude ../../dev-util/git/ .
find . -name CVS -prune -or -type f -print0 | xargs --null cvs add
echangelog 'New package (copied from dev-util/git)'
fgrep -Rw dev-util/git . | fgrep -v Header # Check for self-references
repoman manifest
repoman ci -m 'dev-vcs/git: New package (copied from dev-util/git)'
cd ..
cd ..
# Extend profiles
cd profiles
# Duplicate refs to old package
# ...
# Add move entry
echo 'move dev-util/git dev-vcs/git' >> updates/1Q-2010
echangelog 'Move dev-util/git to dev-vcs/git'
cvs ci -m 'Move dev-util/git to dev-vcs/git' ChangeLog updates/1Q-2010
cd ..
# Fix eclasses
nano -w eclass/git.eclass
cvs commit -m 'Move from dev-util/git to dev-vcs/git' eclass/git.eclass
# Fix packages depending on the old package
# For each package:
cd category/package
fgrep -Rwl dev-util/git . | xargs -n 1 sed
's|\bdev-util/git\b|dev-vcs/git|' -i
echangelog 'Propagate move of dev-util/git to dev-vcs/git'
repoman manifest
cvs diff -u | colordiff | less -r
repoman ci -m "$(basename $(dirname $PWD))/$(basename $PWD): Propagate
move of dev-util/git to dev-vcs/git"
cd ..
# Check if we missed anything
fgrep -Rwl dev-util/git . | fgrep -v ChangeLog | fgrep -v Repository
# Remove old package
cd dev-util
cvs rm -Rf git
cvs ci -m "dev-util/git: Remove (renamed to dev-vcs/git)" git
cd ..
# Clean-up profiles
cd profiles
# Remove refs to old package
# ...
cd ..
# EOF