There are various possibilities for how we could try to fix this:
* land the patch correctly and then merge it with mozilla-central
(I have this done locally, but I don't plan to push it right
now). This fixes the output of hg log -f, but does not seem to
fix hg annotate. (Does hg annotate work with renames anyway?)
* land the patch correctly and replay all of current
mozilla-central on top of that, and then do a dummy merge (I
think this is possible but don't know the details) that
essentially marks current m-c as merged into that history taking
all the details from the new history
If we want to do something to fix this, I think it's better to do
something sooner rather than later since merges into another
repository (like tracemonkey) could complicate things.
hg is pretty bad at showing information about renames anyway, so I'm
not even sure how useful fixing it is...
Thoughts?
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
> * land the patch correctly and replay all of current
> mozilla-central on top of that, and then do a dummy merge (I
> think this is possible but don't know the details) that
> essentially marks current m-c as merged into that history taking
> all the details from the new history
>
I _think_ this should be possible. I'm going to try it locally to see what
happens, and I'll post when I have more information.
--
Ehsan
<http://ehsanakhgari.org/>
Unfortunately, because hg annotate does not correctly follow up the
chain of revisions for a file, it means that this command will pretend
that our netwerk code became into being since revision d8dc49d5bd60
(see <http://hg.mozilla.org/mozilla-central/annotate/ac1ed3f6b2e7/netwerk/cache/nsCache.cpp>
for example.). On hgweb, revision logs are also broken, but you can
use hg log -f locally to get a correct revision log.
--
Ehsan
<http://ehsanakhgari.org/>
Just to be clear, "I'm very sorry."
Note to others, do not rebase with old HG versions.
(I'll soon be making a blog entry about my mistake).
--
~Justin Wood (Callek)
So.. What can we do to get a working annotate here? Not having that is
really a _huge_ problem, especially since we never fixed our hgweb
tooling to handle annotation sanely across the cvs/hg divide. As things
stand, figuring out why a particular piece of necko code is there just
became a _lot_ more time-consuming, and this will interfere in a
significant way with future necko work.
Do push hooks get access to the list of revision ids being pushed? Can
we strip the buggy push (and of course everything that landed on top of
it) from m-c, reland the things that were supposed to be there, and add
a hook that blocks any push containing the buggy revision?
> (see<http://hg.mozilla.org/mozilla-central/annotate/ac1ed3f6b2e7/netwerk/cache/nsCache.cpp>
> for example.). On hgweb, revision logs are also broken, but you can
> use hg log -f locally to get a correct revision log.
Revision log is a heck of a lot less useful than annotate.
-Boris
Alternately, would merging with the correct add/remove patch "the other
way" (with the first and second parent swapped) have helped with hg
annotate?
-Boris
Is this limited to old hg versions? If so, what constitutes old?
-Boris
Can a push hook block a push containing a changeset that has the same
leafname being both added and removed?
-Boris
I haven't read the patch, but it is very likely that this would have helped,
yes.
I am *very* concerned about how this affects the electrolysis merge.
--BDS
Yes, I understand that, and I tried my best to fix the issue, but my
other solution (which was putting the bad changeset on a dormant
branch and closing that branch) didn't work, because it's not possible
to put existing changesets in a new branch. That solution would fix
annotate, but it would cause the default branch to have two heads,
which was not acceptable with our push hooks in place.
> Do push hooks get access to the list of revision ids being pushed? Can we
> strip the buggy push (and of course everything that landed on top of it)
> from m-c, reland the things that were supposed to be there, and add a hook
> that blocks any push containing the buggy revision?
This was also another suggestion of mine, but Gavin didn't like it.
The problem that this solution creates is that anyone that has pulled
a changeset in the range that we split on the server should also split
the correct range, and pull again. This includes Tinderbox machines,
other project branches, developers, etc.
However, I personally still think that this could be manageable, and I
think that the inconvenience that it would create would only be a
one-time thing, whereas the inconvenience of the current state of
things is an ongoing pain (until hg annotate is fixed, that is.)
But if we decide to do this, we need to do it as soon as possible,
before too many people pull.
And I don't think that we need any special push hooks on the server,
because you can't push a changeset to a repo which doesn't have the
changeset's parent(s), right?
>> (see<http://hg.mozilla.org/mozilla-central/annotate/ac1ed3f6b2e7/netwerk/cache/nsCache.cpp>
>> for example.). On hgweb, revision logs are also broken, but you can
>> use hg log -f locally to get a correct revision log.
>
> Revision log is a heck of a lot less useful than annotate.
Yes. I was just pointing out the only remaining option.
--
Ehsan
<http://ehsanakhgari.org/>
ehsanakhgari:~/moz/src [01:31:34]$ hg log -f netwerk/cache/nsCache.cpp
changeset: 43173:ac1ed3f6b2e7
tag: qparent
parent: 43172:09f273f2c759
parent: 43143:c1162c2cc26b
user: Ehsan Akhgari <eh...@mozilla.com>
date: Sat Jun 05 21:23:26 2010 -0400
summary: Merge resolving the bad rename changeset landed as part
of bug 542222
changeset: 43144:2d90590dabe6
parent: 43112:28086cf6ede8
user: Mitchell Field <mitchel...@live.com.au>
date: Sat Jun 05 21:18:12 2010 -0400
summary: Bug 542222 - Reduce recursion in netwerk makefiles.
changeset: 43113:d8dc49d5bd60
user: Mitchell Field <mitchel...@live.com.au>
date: Wed Jun 02 01:56:00 2010 -0400
summary: Bug 542222 - Reduce recursion in netwerk makefiles. r=biesi
changeset: 29050:96c833c60acb
user: Arpad Borsos <arpad....@googlemail.com>
date: Tue May 26 10:53:15 2009 +0200
summary: Bug 467948 - fix some deprecated conversion from string
constant warnings; r=dbaron
changeset: 1:9b2a99adc05e
user: h...@mozilla.com
date: Thu Mar 22 10:30:00 2007 -0700
summary: Free the (distributed) Lizard! Automatic merge from CVS:
Module mozilla: tag HG_REPO_INITIAL_IMPORT at 22 Mar 2007 10:30 PDT,
If you annotate this file, you'll see all lines coming from
d8dc49d5bd60. But if you navigate the left-hand side of merge
upwards, that revision is not in the path at all.
I think this is a bug in hg annotate, and when it gets fixed, we'll
get correct annotation, because the history of the files is correct in
itself.
--
Ehsan
<http://ehsanakhgari.org/>
> _______________________________________________
> dev-tree-management mailing list
> dev-tree-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tree-management
>
FWIW, I tried merges in both ways, and neither helped.
> I am *very* concerned about how this affects the electrolysis merge.
How come?
--
Ehsan
<http://ehsanakhgari.org/>
Right. I'm not attacking you here!
> This was also another suggestion of mine, but Gavin didn't like it.
> The problem that this solution creates is that anyone that has pulled
> a changeset in the range that we split on the server should also split
> the correct range, and pull again. This includes Tinderbox machines,
> other project branches, developers, etc.
>
> However, I personally still think that this could be manageable
Agreed.
> But if we decide to do this, we need to do it as soon as possible,
> before too many people pull.
Also agreed. Who can make that call?
> And I don't think that we need any special push hooks on the server,
> because you can't push a changeset to a repo which doesn't have the
> changeset's parent(s), right?
But the parent of the original botched commit would still be in, no? Or
would we strip that too?
-Boris
Does hg merging deal well with renames even if they're properly
recorded as renames? The last time I was affected by renames, I
think it didn't, but that was a while ago.
cheers,
mike
Yes, merging works well with correctly-done renames.
--BDS
>> I am *very* concerned about how this affects the electrolysis merge.
>
> How come?
The electrolysis branch contains very significant changes to netwerk/, and
we're trying to converge to get this branch landed in mozilla-central this
week. If `hg merge` doesn't produce correct results here so that we end up
with .rej files, it will be a multi-day undertaking to fix it, or at least
that's what i think from prior experience.
--BDS
>> I am *very* concerned about how this affects the electrolysis merge.
>
> How come?
The electrolysis branch contains very significant changes to netwerk/, and
I think that suggests we should close the tree, strip changesets out
of mozilla-central, and push the correct replay.
Not sure how annotate should resolve two parents, other than throwing dices?
Axel
I'm not sure. But I did a null merge, which means that I kept "my"
changes and threw away "theirs", which should solve the problem. And
I think it does, judging by the fact that hg log -f knows how to
navigate the history across the renames now.
--
Ehsan
<http://ehsanakhgari.org/>
Actually, never mind that. I forgot about the issues that would
cause with the pushlog.
I'd *guess* that electrolysis might come out OK if you first merge
with the branch of transplants that Ehsan pushed yesterday (i.e.,
the branch that has the correct rename but not the incorrect one),
and then merge with tip. But I'm not sure.
It might be worth keeping the tree closed until we are.
(It's also worth somebody doing experiments to see if a different
order of merge parents would make annotate work better. I fear
annotate might be based on the order of revisions in the repo rather
than the shape of the graph, though.)
Oh, I wasn't trying to imply that! I just wanted to say that I did
whatever I could think of, but others may have better suggestions.
:-)
>> This was also another suggestion of mine, but Gavin didn't like it.
>> The problem that this solution creates is that anyone that has pulled
>> a changeset in the range that we split on the server should also split
>> the correct range, and pull again. This includes Tinderbox machines,
>> other project branches, developers, etc.
>>
>> However, I personally still think that this could be manageable
>
> Agreed.
>
>> But if we decide to do this, we need to do it as soon as possible,
>> before too many people pull.
>
> Also agreed. Who can make that call?
Not sure. I don't think this is something that we have predicted as
part of our review/superreview requirements, so I wasn't sure who to
talk to yesterday. All we did yesterday was the result of the
consensus of people on irc at that time, which included me, dbaron,
jesse, and gavin (sorry if I'm forgetting someone.)
>> And I don't think that we need any special push hooks on the server,
>> because you can't push a changeset to a repo which doesn't have the
>> changeset's parent(s), right?
>
> But the parent of the original botched commit would still be in, no? Or
> would we strip that too?
The parent of the original revision which caused the problem (i.e.,
28086cf6ede8) has two children, so I think we need to strip that too,
in order to be in a sane state afterwards. *All* changesets landed
after it (including d8dc49d5bd60, obviously) should be stripped. This
will mean that everyone should:
1. Check their local repository for revision 28086cf6ede8, using hg
log -r 28086cf6ede8).
2. If they don't have that changeset, they're not affected, they can
hg pull -u happily.
3. If they do have that changeset, they should preserve all their
local changes somehow (for example, by putting them in patch queues
and hg qpop -a), then run hg strip 28086cf6ede8, and then pull -u, and
then re-apply their local changes.
4. Tinderbox machines should be set to do a clobber build, which pulls
mozilla-central from scratch.
5. Nightly build machines are not affected (since they already do
clobber builds).
6. We should go through all the changesets which we're stripping, go
to their respective bugs, and put a comment in them stating the new
revision ID, in case someone needs to find that in future. I think
there are between 30-40 unique changesets in the range right now (not
counting the changesets that I replayed yesterday).
7. The entire thing should be documented somewhere other than this
thread, to give people a reference on what happened.
8. Project branches might possibly need to strip some changesets from
their repos as well, if they've pulled 28086cf6ede8 or any of its
descendants.
--
Ehsan
<http://ehsanakhgari.org/>
OK. Well, I just closed the tree for now until we sort this out....
It looks like at this point we'd need to strip out Ehsan's merge commit
and everything after it; there seem to be 8 changesets plus a
tracemonkey merge that just happened 20 minutes ago. Wish we'd closed
the tree when we discovered the problem... :(
So the open questions seem to be:
1) Can we sanely merge e10s in?
2) Can we recover sane annotate on the affected files, and how?
-Boris
Well, I personally don't have the bits to perform the measures I
proposed (running hg strip on the mozilla-central repository, for
example, or adding push hooks). So at some point when we come to a
decision we'll have to talk to whoever does about doing that...
In any case, seems like the issue of what to do is still open; tree is
closed for now, pending that getting decided.
-Boris
But that changeset is in clones all over the place that people have out
there. I doubt that we'll be successful in keeping that from getting
pushed back in eventually.
Axel
I may have a fix:
http://pastebin.mozilla.org/731830
Don't know why I couldn't get to work yesterday, but I'm working on
doing the same thing again now...
--
Ehsan
<http://ehsanakhgari.org/>
http://hg.mozilla.org/mozilla-central/annotate/5b3604a3cfbe/netwerk/cache/nsCache.cpp
vs
http://hg.mozilla.org/mozilla-central/annotate/28086cf6ede8/netwerk/cache/src/nsCache.cpp
Could we rely on this until "hg annotate" is fixed, or do we still
want to strip changesets from mozilla-central? (and now tracemonkey)
> Should be very easy. Filed bug 570411 for that.
I have a closely related but tangential question. I have been trying
to shepherd in some patches submitted by a community member. They
include a couple of file renames (and, unfortunately, simultaneous
modifications to the files) but the rename was not recorded properly so
if I import the patch with mq, I get a delete and an add. What is the
best way to fix this? (I don't know how the contributor generated the
diff.)
zw
>I have been trying to shepherd in some patches submitted by a community member. They include a couple of file renames (and, unfortunately, simultaneous modifications to the files) but the rename was not recorded properly so if I import the patch with mq, I get a delete and an add. What is the best way to fix this?
>
hg import --no-commit
hg rename --after
hg commit
I don't know what the mq equivalents are.
--
Warning: May contain traces of nuts.
Hmm... I would have thought strip could just remove a given node in the
DAG and all notes reachable from it without affecting anything else.
Maybe I don't understand the setup correctly, though....
> 1. Check their local repository for revision 28086cf6ede8, using hg
> log -r 28086cf6ede8).
> 2. If they don't have that changeset, they're not affected, they can
> hg pull -u happily.
> 3. If they do have that changeset, they should preserve all their
> local changes somehow (for example, by putting them in patch queues
> and hg qpop -a), then run hg strip 28086cf6ede8, and then pull -u, and
> then re-apply their local changes.
> 4. Tinderbox machines should be set to do a clobber build, which pulls
> mozilla-central from scratch.
> 5. Nightly build machines are not affected (since they already do
> clobber builds).
> 6. We should go through all the changesets which we're stripping, go
> to their respective bugs, and put a comment in them stating the new
> revision ID, in case someone needs to find that in future. I think
> there are between 30-40 unique changesets in the range right now (not
> counting the changesets that I replayed yesterday).
> 7. The entire thing should be documented somewhere other than this
> thread, to give people a reference on what happened.
> 8. Project branches might possibly need to strip some changesets from
> their repos as well, if they've pulled 28086cf6ede8 or any of its
> descendants.
Yes, exactly.
-Boris
That's where a push hook checking for that changeset comes in, no?
-Boris
Yes, but what workflow will lead you to the correct version and location
to be annotated? The normal workflow is "mxr search files for
nsCache.cpp, then click annotate link". What would happen after that to
give you the second url above? And how long would it take to perform
those steps?
> Could we rely on this until "hg annotate" is fixed
Assuming there's something to be fixed about it, right?
> or do we still want to strip changesets from mozilla-central? (and now tracemonkey)
I doubt we care about stripping this out of tracemonkey. We do want to
strip it from m-c, I think, unless the resulting pushlog confusion is
bad enough to kill the idea.
-Boris
I don't think we would need to strip the parent. Stripping one of
its children would just change it back to having one child.
What worries me is that the two branches created by its children are
later merged with each other, so stripping only one child would
probably strip all its children _and_ the merge revision (as well as
all its children), so we would end up with the other branch until the
merge point. I guess we _could_ replay the rest of the changesets on
top of that new tip, but anyways, the bigger problem is determining
whether stripping this way is safe. Which changeset to strip is not
really the main issue here.
--
Ehsan
<http://ehsanakhgari.org/>
Here's what needs to be done right now.
1. Go to mxr, and click the annotate link.
2. In the annotate page, click on one of the lines representing the
faulty revision.
3. Click the changeset link at the top of the page.
4. In the changeset page, click the parent link to go to the parent revision.
5. In the parent revision, click browse and navigate back to the same file.
It's a major pain. Things will be much worse if part of the thing
that you're huntung down is landed after the faulty changeset. In
that case, you should look at two annotate results, and merge them in
your mind.
>> Could we rely on this until "hg annotate" is fixed
>
> Assuming there's something to be fixed about it, right?
I don't really know how annotate is supposed to work, but it seems to
me that it's broken right now. We have the correct history, and
mercurial knows how to get it (hg log -f) but apparently annotate
works in an entirely different way.
Could we ask one of the mercurial folks to look at the problem? Maybe
they can at least tell us that this is a problem which is going to be
fixed in version X, or it's something that they're not looking at
fixing any time soon?
>> or do we still want to strip changesets from mozilla-central? (and now
>> tracemonkey)
>
> I doubt we care about stripping this out of tracemonkey. We do want to
> strip it from m-c, I think, unless the resulting pushlog confusion is bad
> enough to kill the idea.
If tracemonkey has the broken changeset (and any of its children),
those need to be stripped too. Otherwise, their next push would fail
(by the hook that we should put in place after stripping.)
--
Ehsan
<http://ehsanakhgari.org/>
I'm not sure if there are any. What I would do is to pop the mq
patch, hg import --no-commit and hg mv -A as you suggested, qrm the
old patch and qnew -f the new one.
--
Ehsan
<http://ehsanakhgari.org/>
Why would that happen? Doesn't pushlog use the information that
mercurial stores about the changesets?
--
Ehsan
<http://ehsanakhgari.org/>
No, it's using a separate database for the data about pushes.
Axel
I looked at the source of annotate, and I think it's actually right.
What it shows is "the latest modification to this line", and the merge
is ignored. Which I think is the right thing.
There are bugs in the code, like
http://mercurial.selenic.com/bts/issue1839, but that doesn't affect us
in our case, as we're merged. That might be affected by
http://mercurial.selenic.com/bts/issue1327.
Anyway, I think stripping changesets is a bad idea, and if I look at the
revisions of nsCache.cpp, it's not a good balance to shake our ecosystem
that badly.
I for example do use copies of repos and repo data in a separate
database to work around some of our limitations here, and updating that
data for a removed changeset is gonna be tedious. No idea who else is
running stuff like that.
Also:
Newer versions of hg are much nicer in exposing multiple parents, I ran
an hg serve locally, it's not all that tricky. It'd be a nice RFE to
have a graph display for the file log on the web. That works locally,
but not on the web afaict.
Axel
Yep. See bug 557210 for updating it to something newer.
- Blair
To be fair, "old" in this context means "15 months old".
http://mercurial.selenic.com/wiki/WhatsNew#Version_1.2.1_-_2009-03-20
I'll get a new MozillaBuild release out soon, there are a few other
things we need to update anyway.
-Ted
It's not just nsCache.cpp; we've had single-file fails like this before
and let them slide (might have been a mistake; that precommit hook Ehsan
filed a bug on would have saved us here if we'd put it in place then).
But in this case, we're talking about breakage for every single file
under netwerk/ except the toplevel Makefile.in.
-Boris
Yes, that's what it would have to do.
> I guess we _could_ replay the rest of the changesets on
> top of that new tip
Right.
-Boris
Note the same file, since it got added/removed. You have to navigate to
some other file at some other path, which you have to guess at.
> It's a major pain.
Amen.
> If tracemonkey has the broken changeset (and any of its children),
> those need to be stripped too. Otherwise, their next push would fail
> (by the hook that we should put in place after stripping.)
Tracemonkey doesn't do whole-repo pushes into m-c. They pull m-c
wholesale, but only push cherry-picked changesets. So they should, in
fact, be ok. But probably worth double-checking with sayrer.
-Boris
>On Sunday 2010-06-06 22:32 -0400, Boris Zbarsky wrote:
>
>
>>On 6/6/10 4:36 PM, Ehsan Akhgari wrote:
>>
>>
>>>The parent of the original revision which caused the problem (i.e., 28086cf6ede8) has two children, so I think we need to strip that too, in order to be in a sane state afterwards.
>>>
>>>
>>Hmm... I would have thought strip could just remove a given node in the DAG and all notes reachable from it without affecting anything else. Maybe I don't understand the setup correctly, though....
>>
>>
>I don't think we would need to strip the parent. Stripping one of its children would just change it back to having one child.
>
>
Assuming this is the approach taken, then am I correct in understanding
that people will fall in to one of the following categories:
1. Automatically destroys and reclones mozilla-central. Unaffected.
2. Not pulled any of the stripped changesets. Unaffected.
3. Pulled but not updated to any of the stripped changesets. Will
have local changesets which hg push will try (and fail) to push
which need to be stripped. (How?)
4. Updated to a stripped changeset, but uses mq or otherwise has no
local changes. Can update -C to the new head. (Or pull -rebase?)
5. Updated to a stripped changeset, and has local changes. Needs to
update -r 28086cf6ede8 then can update to the new head.
Pushlog has its own database for the push data; that's not stored in
the changesets. It then expects that the changesets that it has
data about are in the repository so that it can get the rest of the
information about them. I expect that if they're not in the
repository, it will get confused (but I'm not sure exactly how
badly). (I remember it being broken the last time we stripped a
changeset in mozilla-central. In that case, however, the problem
was that the changeset was somehow partially pushed, and we fixed
the problem by stripping the changeset and then re-pushing the exact
same changeset, so once the changeset was re-pushed it was in
mozilla-central again and pushlog was fine.)
hg strip rev
where "rev" is the broken changeset.
-Boris
Detail that's gtk if we really really go down this route, hg strip is
part of mq. That should be part of the messaging.
Axel
--
Ehsan Akhgari
eh...@mozilla.com
Sent from my Palm PreOn 7 Jun 2010 7:49 a.m., Mike Shaver <mike....@gmail.com> wrote:
On Sun, Jun 6, 2010 at 2:12 PM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
> On Sun, Jun 6, 2010 at 10:40 AM, Boris Zbarsky <bzba...@mit.edu> wrote:
>> On 6/5/10 10:30 PM, Justin Wood (Callek) wrote:
>>>
>>> Note to others, do not rebase with old HG versions.
>>
>> Is this limited to old hg versions? If so, what constitutes old?
>
> I think Callek said he was using 1.2.1...
Isn't that the version we're distributing in MozillaBuild? Oof.
Mike
cheers,
mike
Right.
> 2. Not pulled any of the stripped changesets. Unaffected.
Right.
> 3. Pulled but not updated to any of the stripped changesets. Will
> have local changesets which hg push will try (and fail) to push
> which need to be stripped. (How?)
hg strip bad-rev-id
> 4. Updated to a stripped changeset, but uses mq or otherwise has no
> local changes. Can update -C to the new head. (Or pull -rebase?)
hg update parent-of-bad-rev-id
hg strip bad-rev-id
hg pull -u
> 5. Updated to a stripped changeset, and has local changes. Needs to
> update -r 28086cf6ede8 then can update to the new head.
In addition to somehow save their local changes and reapply them later, yes.
--
Ehsan
<http://ehsanakhgari.org/>
-Ted
It should be possible to generate a mapping of old to new changesets
and replace them inside the pushlog db, right?
The only tricky part is mimicking the exact same changeset graph in
the repository after stripping, so that we would have a 1:1 mapping
between old and new changeset IDs. Is it possible to "strip" the
changeset that are going to be stripped in mozilla-central and then
let the new db be built the normal way?
I expect that such modifications would break tools which use use the
pushlog db, such tinderboxpushlog. :(
--
Ehsan
<http://ehsanakhgari.org/>
I had done this experiment in a user repo on Saturday:
<http://hg.mozilla.org/users/eakhgari_mozilla.com/542222-fixed-repo>
The only downside with this approach is that the default branch would
end up having two heads (e.g.,
http://hg.mozilla.org/users/eakhgari_mozilla.com/542222-fixed-repo/rev/c590aa6c1153
and http://hg.mozilla.org/users/eakhgari_mozilla.com/542222-fixed-repo/rev/fdb1e4bc853d).
We have a hook in place which prevents people from pushing multiple
heads for the default branch, which is probably a good thing. I
_think_ that hook can be modified to make an exception this one time.
The upside of this approach is that we won't have to deal with
stripping anything from mozilla-central, and we'd get the correct
annotate output. The downside of this approach is that
mozilla-central would end up having two heads on the default branch
forever, and people who have updated their clones to any revision
below the bad changeset in the tree would have to hg update -C the
next time (because they will be updating across branches) which
requires them to save their local modification somewhere. And
Tinderboxen should probbaly be set to a clobber build anyway.
What do people think about this approach?
--
Ehsan
<http://ehsanakhgari.org/>
> _______________________________________________
> dev-tree-management mailing list
> dev-tree-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tree-management
>
As far as updates go, for now we can only tell them that
mozilla-central is closed, and will remain closed until this issue is
resolved. But as far as I can tell, we don't yet have a solution
which satisfies everyone. But the meeting is in 2 hours, and a lot
can happen in that time!
--
Ehsan
<http://ehsanakhgari.org/>
OK, then how do we get usable blame? We've already been drifting down
the path of blame being broken for a few years (e.g. we never hooked up
blame across the hg/cvs divide, which we'd been going to do); it's
getting to the point where looking up blame is hard enough, apparently,
that people commonly put up patches that change code in ways that are
clearly wrong given the blame on the code... and then reviewers end up
having to do the time-consuming blame-hunting to r- the patch.
-Boris
> On 6/5/10 9:34 PM, Ehsan Akhgari wrote:
>> I ended up doing this, and pushed the results to mozilla-central.
>>
>> Unfortunately, because hg annotate does not correctly follow up the
>> chain of revisions for a file, it means that this command will
>> pretend
>> that our netwerk code became into being since revision d8dc49d5bd60
>
> So.. What can we do to get a working annotate here? Not having
> that is really a _huge_ problem, especially since we never fixed our
> hgweb tooling to handle annotation sanely across the cvs/hg divide.
> As things stand, figuring out why a particular piece of necko code
> is there just became a _lot_ more time-consuming, and this will
> interfere in a significant way with future necko work.
For what it's worth, you could use the git mirror at git://
bluishcoder.co.nz/git/mozilla-central.git to work around this problem.
Git tracks renames by content instead of with metadata so it doesn't
have the same problem mercurial does here.
Plus you can graft on the the cvs history from this repository http://people.mozilla.org/~jmuizelaar/mozilla-cvs-history.git
and get annotations back to the beginning of the history. See http://pastebin.mozilla.org/732143
-Jeff
>The downside of this approach is that mozilla-central would end up having two heads on the default branch forever, and people who have updated their clones to any revision below the bad changeset in the tree would have to hg update -C the next time (because they will be updating across branches) which requires them to save their local modification somewhere.
>
Can we start by begging everyone to update -r 28086cf6ede8 right now, so
that when we do sort something out they won't have to do anything special?
Sure. This is something that we can announce on the project meeting as well.
--
Ehsan
<http://ehsanakhgari.org/>
I don't think that we can go back in time, now that there has been a
tracemonkey merge. The bad revs have been pulled into TM and it would be
basically impossible to get them fully untangled now.
--BDS
I added this to the project meeting notes:
<https://wiki.mozilla.org/WeeklyUpdates/2010-06-07#Gecko>
--
Ehsan
<http://ehsanakhgari.org/>
> OK, then how do we get usable blame? We've already been drifting down
> the path of blame being broken for a few years (e.g. we never hooked up
> blame across the hg/cvs divide, which we'd been going to do); it's
As an aside, was there ever a bug on hooking up blame across the hg/
cvs divide (I've looked, but I can't find one)? This is easily one of
my biggest pain points with the Mozilla-Hg ecosystem, and I know I
don't use Gecko blame anywhere close to as often as Boris must.
Smokey
> As an aside, was there ever a bug on hooking up blame across the hg/
> cvs divide (I've looked, but I can't find one)? This is easily one of
> my biggest pain points with the Mozilla-Hg ecosystem, and I know I
> don't use Gecko blame anywhere close to as often as Boris must.
Yes, there was a bug. We have the data necessary to do it, but hooking it up
with hgweb or normal hg annotate is hard, and so we haven't actually done
it. It's pretty trivial to do it with an extension.
--BDS
What I think would really help is a rendition of
hg log -G -f somefile
Sadly, -G and -f (graph and follow) are not compat, at least in 1.5.4.
I'm not sure why, though.
Anyway, there could also be an external tool that does a more
computationally intensive blame computation than what's hg doing now,
detecting duplicate patches and backouts.
That seems to be helpful in more usecases (i.e., backouts), too, much
more than breaking into our revisions and creating hooks that keep
people from doing what hg tries to do by default.
Axel
I have been pondering about this for a bit more.
Here's my thinking as of now, subject to change on every other beer:
hg annotate as implemented right now gives you:
- When was this line of code changed last?
What folks are really interested here, and what the code doesn't deliver
in this scenario, is
- When was this code seen first?
I'm having a bit of an issue if "this code" is actually per-line, as a
single line might be doing something rather different with respect to
context, but I guess it'd be safe to assume that it's per line.
I'd be surprised if we couldn't hook up a view to hg via an extension
that would deliver that perspective. It'd be a good deal more expensive
than what hg is doing natively, as you don't have to just walk back the
graph until you got all lines covered, but you gotta always walk all
ancestors, and basically compare all lines to each other. Sounds like an
N^2*M problem, with N being the number of lines, M being the number of
revisions.
We got the pushlog hooks linked up, I guess we can get such a view
hooked up.
Axel
Question is, when can we have such an extension? If it's in a couple
of weeks, then I'd vote for opening up mozilla-central as it is right
now. If it's some indefinite point in the future, then I'd vote to
resolve this problem in a way that existing hg tools can handle it the
way that we want.
Unless we have an owner working on this and it's very high priority on
their list, I doubt that the answer is anything but "some indefinite
point in the future".
--
Ehsan
<http://ehsanakhgari.org/>
I should have an estimate for it tomorrow; I'm hoping that it can get
resolved in the next couple of weeks.
Mike
Here's the code that does it command-line-style for me:
import sys
from mercurial.ui import ui as _ui
from mercurial.hg import repository
from mercurial.node import hex
file = sys.argv[1]
r = repository(_ui())
fctx = r.filectx(file,'default')
lines = fctx.data().splitlines()
_h = fctx.rev()
blame = map(lambda l: (_h, l), lines)
for ancestor in reversed(list(fctx.ancestors())):
alines = ancestor.data().splitlines()
_h = ancestor.rev()
for i, line in enumerate(lines):
if line is None:
continue
if line in alines:
blame[i] = (_h, line)
lines[i] = None
for rev, line in blame:
print rev, line
PS: That code sucks, when it comes down to the "find that code" part,
should be written in terms of mercurial.bdiff.blocks.
Axel
Can we add a push hook to reject any pushes (with a message pointing
to a wiki page or something) with that changeset? Ideally we'd be
able to maintain a list of 'blacklisted' changsets so that we have a
way to deal with that in the future.
- Vlad
--
Ehsan
<http://ehsanakhgari.org/>
For some definition of "code" and "first". Typically what's needed in
your typical vcs archeology trip is figuring out why the code is the way
it is. This might mean "the last substantive change to this code"
(however substantive is defined), or "when the code was added". It
really depends on the situation.
> I'm having a bit of an issue if "this code" is actually per-line, as a
> single line might be doing something rather different with respect to
> context, but I guess it'd be safe to assume that it's per line.
So the question that you're proposing we answer is "what was the first
revision which had this line in the exact state it's in now"?
> I'd be surprised if we couldn't hook up a view to hg via an extension
> that would deliver that perspective. It'd be a good deal more expensive
> than what hg is doing natively, as you don't have to just walk back the
> graph until you got all lines covered, but you gotta always walk all
> ancestors, and basically compare all lines to each other. Sounds like an
> N^2*M problem, with N being the number of lines, M being the number of
> revisions.
That's really going to hurt on some files in our tree
(nsCSSFrameConstructor comes to mind: ~13000 lines, ~1000 revisions)....
And the point is that we don't need this information for the whole
file, typically; just for a small part of it. Does that help any?
-Boris
Actually, fixing the code changed that. I'm now actually taking diffs
across all ancestors to the current code, so it's detecting sequences of
lines that showed up like this. That's the same code pattern that hg
annotate uses, just reversed the order, and thus I might walk across
more revisions, I guess.
>> I'd be surprised if we couldn't hook up a view to hg via an extension
>> that would deliver that perspective. It'd be a good deal more expensive
>> than what hg is doing natively, as you don't have to just walk back the
>> graph until you got all lines covered, but you gotta always walk all
>> ancestors, and basically compare all lines to each other. Sounds like an
>> N^2*M problem, with N being the number of lines, M being the number of
>> revisions.
>
> That's really going to hurt on some files in our tree
> (nsCSSFrameConstructor comes to mind: ~13000 lines, ~1000 revisions)....
> And the point is that we don't need this information for the whole file,
> typically; just for a small part of it. Does that help any?
>
It helped to just unsuck the code, luckily. I filed
https://bugzilla.mozilla.org/show_bug.cgi?id=570676, which shows the
difference between what hg annotate does, and what my proposed code
does. It does show a slight slowdown for nsCSSFrameConstructor, but
moreso, it shows a different blame for it, I've put a review request on
that on you, if you don't mind. I guess it's someone with your
understanding of the history of that file than can tell whether that's a
bug or a feature.
Axel
Sounds good. Are we willing to open the tree now?
cheers,
David
>
>
>
>
>
>
> On Mon, Jun 7, 2010 at 10:21 PM, Vladimir Vukicevic <vladim...@gmail.com> wrote:
> > On Jun 6, 1:36 pm, Ehsan Akhgari <ehsan.akhg...@gmail.com> wrote:
> >> 3. If they do have that changeset, they should preserve all their
> >> local changes somehow (for example, by putting them in patch queues
> >> and hg qpop -a), then run hg strip 28086cf6ede8, and then pull -u, and
> >> then re-apply their local changes.
>
> > Can we add a push hook to reject any pushes (with a message pointing
> > to a wiki page or something) with that changeset? Ideally we'd be
> > able to maintain a list of 'blacklisted' changsets so that we have a
> > way to deal with that in the future.
>
> > - Vlad
> > _______________________________________________
> > dev-tree-management mailing list
> > dev-tree-managem...@lists.mozilla.org
> >https://lists.mozilla.org/listinfo/dev-tree-management
> Sounds good. Are we willing to open the tree now?
The tree has now been closed for close to two days. I've seen a bunch of solutions go by, and now we're waiting for a Mercurial fix. I really do not pretend to know the best solution, but it feels like we can just block the bad changesets, put a note on hg that gets sent to committers / pullers, and get back to work?
Can we get a clear answer about who owns the plan, what the plan is, who's driving, or maybe just a bug number tracking the issue, as well as an indication of when the tree is likely to re-open?
cheers,
mike
Mike
On Jun 8, 2010 9:06 AM, "Mike Beltzner" <belt...@mozilla.com> wrote:
> On 2010-06-08, at 8:47 AM, David Bolter wrote:
>
>> Sounds good. Are we willing to open the tree now?
>
> The tree has now been closed for close to two days. I've seen a bunch of
solutions go by, and now we're waiting for a Mercurial fix. I really do not
pretend to know the best solution, but it feels like we can just block the
bad changesets, put a note on hg that gets sent to committers / pullers, and
get back to work?
>
> Can we get a clear answer about who owns the plan, what the plan is, who's
driving, or maybe just a bug number tracking the issue, as well as an
indication of when the tree is likely to re-open?
>
> cheers,
> mike
Tree is OPEN.
cheers,
David
>
> Mike
>
> On Jun 8, 2010 9:06 AM, "Mike Beltzner" <beltz...@mozilla.com> wrote:> On 2010-06-08, at 8:47 AM, David Bolter wrote:
>
> >> Sounds good. Are we willing to open the tree now?
>
> > The tree has now been closed for close to two days. I've seen a bunch of
>
> solutions go by, and now we're waiting for a Mercurial fix. I really do not
> pretend to know the best solution, but it feels like we can just block the
> bad changesets, put a note on hg that gets sent to committers / pullers, and
> get back to work?
>
> > Can we get a clear answer about who owns the plan, what the plan is, who's
>
> driving, or maybe just a bug number tracking the issue, as well as an
> indication of when the tree is likely to re-open?
>
>
>
>
>
>
>
>
>
> > cheers,
> > mike
> > _______________________________________________
> > dev-tree-management mailing list
> > dev-tree-managem...@lists.mozilla.org
> >https://lists.mozilla.org/listinfo/dev-tree-management