history and forking in meta repositories for Lua distributions
David Manura
distributions and Batteries, how much attention should we pay to
preserving history in those individual package repos?
Observe that the LuaForge github page [1] has some similarities to the
LuaDist meta-repo [2]. Some modules like etree exist in both but with
independent history [3,4]. There's some issues in the LuaForge
conversion [5], particularly in the use of nested directories which
isn't what we want for deployed packages and which makes me second
guess whether to fork from it. There's also significant issues in the
LuaDist repos [6] that I think should be resolved more in the near
term.
I understand the LuaForge repos originally had a different purpose:
providing a straightforward conversion from the old CVS repos, with no
loss in history, and leaving it to the authors (or perhaps maintainers
like us) to move it elsewhere or fix it up with history rewriting if
they want. The LuaDist repos, on the other hand, currently lack much
in terms of history except for some modules like LuaCOM that have
already been maintained in git. The LuaDist repos rather were
originally intended as a shared space that distribution maintainers,
not just original module authors, can commit to in preparing
distributions like LuaDist and more recently LfW, Batteries, and
others. The LuaForge repos page has a more recognized name, although
it currently seems purposed with less ambitious goals than the LuaDist
repos.
Also, I wonder if it's worth forking our Lua repo [7] off of [8],
which has a more complete history back to Lua version 1.0. See more
recently [9,10], which also has branches for LuaJit, though it only
contains major snapshot's of Mike's history and is not directly linked
to Mike's git repo. Furthermore, Mike's repo doesn't go further back
than 2009-12-08 and doesn't branch from Lua's history.
Is this attention to history even worth it? The thing with git is
that you can't after-the-fact cleanly add newly discovered history or
remove useless history without affecting hashes and disrupting the
built-in merge algorithms. One thing that comes to mind is that
history can aid in debugging.
[1] https://github.com/luaforge/
[2] https://github.com/LuaDist/Repository
[3] https://github.com/luaforge/etree
[4] https://github.com/LuaDist/etree
[5] http://lua-users.org/lists/lua-l/2011-01/msg01234.html
[6] https://github.com/LuaDist/Repository/issues/#issue/32
[7] https://github.com/LuaDist/lua
[8] http://repo.or.cz/w/lua.git
[9] http://lua-users.org/lists/lua-l/2010-01/msg00280.html
[10] https://github.com/ittner/Lua-experimentation
Ryan Pusztai
Regards,
Ryan
steve donovan
> I think the less we have to "own" the code the better. Our focus should be
> on packaging not module development.
That seems generally a good idea, otherwise we end up as de facto
maintainers. Although, having started with a fairly clean slate (major
revisions maybe) then any patches should be tracked .. not an easy
one, I imagine that Linux distros have similar uneasy relations with
the upstream projects....
Perhaps it's a pragmatic thing - it's often faster for us to patch and
track than wake up the original maintainer, unless that person is in a
development burst.
steve d.
David Manura
> I don't think history matters that much. Only the main release milestones
> should be enough. I think the focus should be on adding any modules that are
> not on GitHub to Git and sub-module the rest.
I've spent some time cleaning up a couple dozen git repos in [1]. The
practice we're currently using is in [2]. We fork existing official
git(hub) repos (which as a side-effect imports all history). For
others, we create new git repos, importing or bridging what history we
have, and these new git repos may be forked by others as well. Then
in a separate branch, we apply our own CMake build file additions and
merge updates from the upstream branch(es).
So, for example, for an Lhf module like lpack there is now a git repo
[3] based on a couple tarballs. I encourage any git user wanting to
patch lpack to fork this repo, and patches can then be pulled between
the forks. Alternately, new official tarballs posted by the author
can be imported into the upstream branch and later merged.
[1] https://github.com/LuaDist/
[2] https://github.com/LuaDist/Repository/wiki/Dev:-Guidelines --
"Source code repositories of existing projects are imported into
LuaDist as follows"
[3] https://github.com/LuaDist/lpack
> Our focus should be on packaging not module development.
True. I do, however, see there being more effective collaboration
between those packaging and those doing module development. The
increased prevalence of git repos on github permits new workflows
between these parties.
Ryan Pusztai
On Fri, Mar 25, 2011 at 11:01 PM, Ryan Pusztai <rpus...@gmail.com> wrote:I've spent some time cleaning up a couple dozen git repos in [1]. The
> I don't think history matters that much. Only the main release milestones
> should be enough. I think the focus should be on adding any modules that are
> not on GitHub to Git and sub-module the rest.
practice we're currently using is in [2]. We fork existing official
git(hub) repos (which as a side-effect imports all history). For
others, we create new git repos, importing or bridging what history we
have, and these new git repos may be forked by others as well. Then
in a separate branch, we apply our own CMake build file additions and
merge updates from the upstream branch(es).
So, for example, for an Lhf module like lpack there is now a git repo
[3] based on a couple tarballs. I encourage any git user wanting to
patch lpack to fork this repo, and patches can then be pulled between
the forks. Alternately, new official tarballs posted by the author
can be imported into the upstream branch and later merged.
[1] https://github.com/LuaDist/
[2] https://github.com/LuaDist/Repository/wiki/Dev:-Guidelines --
"Source code repositories of existing projects are imported into
LuaDist as follows"
[3] https://github.com/LuaDist/lpack
> Our focus should be on packaging not module development.True. I do, however, see there being more effective collaboration
between those packaging and those doing module development. The
increased prevalence of git repos on github permits new workflows
between these parties.
Agreed, but I just worry about becoming the "perceived" expert, when our goal is to distribute and deploy modules. There is a fine line that we will have to figure out as we start moving forward.
--
Regards,
Ryan
steve donovan
> Agreed, but I just worry about becoming the "perceived" expert, when our
> goal is to distribute and deploy modules. There is a fine line that we will
> have to figure out as we start moving forward.
I think people mostly understand the distinction between 'maintainer'
and 'author'. (One of the things that the new LR rockspec format
needs is to capture this distinction.)
OK, it does mean we get problem reports, but we pass them on. We make
patches, and the upstream project can incorporate them.
It should all work out.
The alternative is to be always waiting for modifications from
upstream, especially for small fixes.
steve d.
Hisham
<steve.j...@gmail.com> wrote:
> On Sat, Mar 26, 2011 at 5:01 AM, Ryan Pusztai <rpus...@gmail.com> wrote:
>> I think the less we have to "own" the code the better. Our focus should be
>> on packaging not module development.
>
> That seems generally a good idea, otherwise we end up as de facto
> maintainers. Although, having started with a fairly clean slate (major
> revisions maybe) then any patches should be tracked .. not an easy
> one, I imagine that Linux distros have similar uneasy relations with
> the upstream projects....
My experience (having been on both sides of this fence) is that as
packagers you need to actively refrain yourself from becoming the de
facto maintainer. It often goes like this: you start applying bugfixes
and tracking patches; then there are things that are "bugs" to one
user (user: 'X does not do Y!') and are "features" to another
(upstream: 'but I didn't write X to do Y in the first place!'). Then
the packager decides to step up and make that call because the last
release of X was one year ago. Before you realize, you are patching
features in because of "overwhelming user requests" and voila, you're
the maintainer of an unofficial fork.
One day I woke up and found that a key I had added to htop's interface
had already been "taken" for a different feature in a Debian patch for
a previous version! And this was a patch I had received and rejected
myself. The user had submitted both to me (upstream) and to Debian, to
see who would accept... Remember also the confusion of incompatible
flags patched in to add bzip2 support to GNU tar years ago.
> Perhaps it's a pragmatic thing - it's often faster for us to patch and
> track than wake up the original maintainer, unless that person is in a
> development burst.
In the beginning it's faster to patch and track but that can lead to a
mess later on. It's better to pursue active collaboration with
upstream. Often it's in their interest as well to have their project
packaged by Lua for Windows. If upstream is unavailable, then it's
better to fork officially and handle the consequences.
It's part of a packager's duty to merge in bugfix patches, yes, but
this role is better taken _very_ conservatively. A good rule of thumb:
unless they're patches that affect the ability to build, or fix
reproducible crash bugs (not just foo() returns x when the API says
y), leave the patch out and report the bug upstream. Restrict the
scope like that and you'll keep yourself out of uneasy relations.
-- Hisham
(sorry about the late reply, I thought I had sent this, just found it
sitting in my Drafts folder)