Git mirror of mozilla-central down

[Ehsan Akhgari]

If you haven't noticed it before, mozilla-central's git mirror has
stopped updating since some time yesterday after the IonMonkey merge.
I've been working on fixing it since last night when I was notified of
the problem, but that was a huge merge which makes it very hard to
figure out what exactly went wrong. I'm making very slow progress on
fixing it, and will update you all when it's back in a good shape again.

Cheers,
Ehsan

[Ehsan Akhgari]

Sad news everyone. Since Tuesday night I have been trying to fix the
repository using almost anything that came to my mind, from trying to
recreate the missing commits manually one by one to updating hg and hg-git,
trying to reconvert only parts of the IonMonkey merge to narrow down the
problem, and eliminating all of the possible factors that could have an
impact on this conversion process, and all of my attempts have failed. I
have sort of narrowed down the range of the IonMonkey merge which causes
the problem, but narrowing it down further is a very time consuming process.

Unfortunately I don't have enough time for now to work on this even more,
so you should not expect any more updates to the mozilla-central mirror. I
may look at this again in the future and try other things that I can think
of (such as trying to narrow down the failure range even further, or
resorting to drastic measures such as redoing parts of the initial
conversion process from scratch), but I don't know when I will have enough
time to look at this again. If you're someone who knows things about git,
mercurial or hg-git and wants to lend a hand, I can provide you with more
information on how to reproduce the problem, and what I've tried and where
to go from here.

In the mean time, if you've been using this mirror and want to continue
using git, you may give this mirror a try:
https://github.com/mozilla/releases-mozilla-central. There are a few
downsides of doing so which you should be aware of:

* This repo is created by RelEng by a direct conversion of the mercurial
repository to git, which means that there is no CVS history for you to use.
* This repo does not have an inbound branch like my mirror did, so if you
want a commit which is on mozilla-inbound but not on mozilla-central yet, I
guess you should wait until it gets merged to mozilla-central.
* The sha1's in this mirror are obviously different than on mine, which
means that you should not just git pull it, and you need to rebase all of
your local branches on top of it after you add the new remote. jlebar
posted a detailed set of instructions on how to do this on dev-b2g
yesterday, so please take a look at his post if you need instructions on
how to do this.

I'll update you guys in case someone steps up to help and we make any
progress on this.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

[Ralph Giles]

On 12-09-13 2:49 PM, Ehsan Akhgari wrote:
> Sad news everyone. Since Tuesday night I have been trying to fix the
> repository using almost anything that came to my mind, from trying to
> recreate the missing commits manually one by one to updating hg and hg-git,
> trying to reconvert only parts of the IonMonkey merge to narrow down the
> problem, and eliminating all of the possible factors that could have an
> impact on this conversion process, and all of my attempts have failed. I
> have sort of narrowed down the range of the IonMonkey merge which causes
> the problem, but narrowing it down further is a very time consuming process.
>
> Unfortunately I don't have enough time for now to work on this even more,
> so you should not expect any more updates to the mozilla-central mirror.

That's sad news indeed. Thanks for working on it. I hope we can get it
back up eventually!

> * This repo is created by RelEng by a direct conversion of the mercurial
> repository to git, which means that there is no CVS history for you to use.

You you understand why this repo doesn't have the same problem with the
ionmonkey merge?

-r

[Ehsan Akhgari]

On 12-09-13 6:18 PM, Ralph Giles wrote:
>> * This repo is created by RelEng by a direct conversion of the mercurial
>> repository to git, which means that there is no CVS history for you to use.
>
> You you understand why this repo doesn't have the same problem with the
> ionmonkey merge?

I'm reasonably certain that I hit a bug in hg-git. I've tried their tip
version but that still doesn't fix it. I don't know the details of the
bug unfortunately (which prevents me from fixing it!), but I have no
reason to believe that it has anything to do with my setup, so I'm
pretty sure that the other repo will hit the same problem sooner or
later. Which means that it's really a good idea for the maintainers of
that repo to try to figure out what happened with my mirror before this
bug breaks their setup. :-)

Cheers,
Ehsan

[David Humphrey]

Thanks for your work, Ehsan, it's really appreciated.

Can we file a bug on this and not have it be something literally off the
side of your desk? Too many of us depend on it now to not see this
through, and if you can't get it, we need to find another solution soon.

Dave

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[Ehsan Akhgari]

On 12-09-13 6:30 PM, David Humphrey wrote:
> Thanks for your work, Ehsan, it's really appreciated.
>
> Can we file a bug on this and not have it be something literally off the
> side of your desk? Too many of us depend on it now to not see this
> through, and if you can't get it, we need to find another solution soon.

Not sure what kind of bug are you asking for. I filed bug 790710 to ask
for help from IT but that got wontfixed because they did not believe
that they should spend time on this given that RelEng is in the process
of duplicating my setup. Or are you asking for a different bug?

Ehsan

[David Humphrey]

Your original email included no bug, so I was asking for *any* bug to
get filed vs. this getting lost in a mailing list post.

I want to see this not die with your efforts, basically.

Dave

[Justin Lebar]

> * This repo does not have an inbound branch like my mirror did, so if you
> want a commit which is on mozilla-inbound but not on mozilla-central yet, I
> guess you should wait until it gets merged to mozilla-central.

Although that repository doesn't have an inbound branch, there is a
separate integration-mozilla-inbound repository, whose master branch
corresponds to m-i.

https://github.com/mozilla/integration-mozilla-inbound/

It's kind of annoying that you have to explicitly fetch from both
remotes, but in the grand scheme of things, it's not too bad.

> * This repo is created by RelEng by a direct conversion of the mercurial
> repository to git, which means that there is no CVS history for you to use.

This was /the/ killer feature of Ehsan's mirror, and I'm hopeful that
releng will include CVS history in their mirror soon (at the cost of
forcing us to rebase all our branches once again, of course).

In the meantime, it's not too hard to attach CVS history to the releng
repository, locally.

Pull Ehsan's repository into your new releng repository. Then do

$ git replace 2514a423 215f52d0

(If you examine those hashes, you should see that 215f52d0 is the last
CVS cset from Ehsan's repository, while 2514a423 is the first commit
in the releng repository. So we're effectively replacing "Free The
Distributed Lizard" with the last CVS commit.)

I've only been using this for a day, so caveat emptor, but it seems to work.

Crucially, the git replace command will not change any of your hashes.

You can even push and pull these replace revs, if that sort of thing
floats your boat. According to [1], the magic commands are

git push 'refs/replace/*'
git fetch 'refs/replace/*:refs/replace/*'

It's not great, but it's better than not having CVS history!

Happy hacking,
-Justin

[1] http://git.661346.n2.nabble.com/What-s-the-best-way-to-push-fetch-replace-refs-td7137925.html

On Thu, Sep 13, 2012 at 6:30 PM, David Humphrey
<david.h...@senecacollege.ca> wrote:
> Thanks for your work, Ehsan, it's really appreciated.
>
> Can we file a bug on this and not have it be something literally off the
> side of your desk? Too many of us depend on it now to not see this through,
> and if you can't get it, we need to find another solution soon.
>

[Ehsan Akhgari]

On 12-09-13 6:47 PM, David Humphrey wrote:
> On 12-09-13 6:40 PM, Ehsan Akhgari wrote:

>> On 12-09-13 6:30 PM, David Humphrey wrote:
>>> Thanks for your work, Ehsan, it's really appreciated.
>>>
>>> Can we file a bug on this and not have it be something literally off the
>>> side of your desk? Too many of us depend on it now to not see this
>>> through, and if you can't get it, we need to find another solution soon.
>>

>> Not sure what kind of bug are you asking for. I filed bug 790710 to
>> ask for help from IT but that got wontfixed because they did not
>> believe that they should spend time on this given that RelEng is in
>> the process of duplicating my setup. Or are you asking for a
>> different bug?
>>
>> Ehsan
>>
> Your original email included no bug, so I was asking for *any* bug to
> get filed vs. this getting lost in a mailing list post.
>
> I want to see this not die with your efforts, basically.

Sure, filed bug 791123 and put it in Firefox::General since I couldn't
think of a better place! :-)

Ehsan

[Ben Hearsum]

On 09/13/12 05:49 PM, Ehsan Akhgari wrote:
> In the mean time, if you've been using this mirror and want to continue
> using git, you may give this mirror a try:
> https://github.com/mozilla/releases-mozilla-central. There are a few
> downsides of doing so which you should be aware of:
>
> * This repo is created by RelEng by a direct conversion of the mercurial
> repository to git, which means that there is no CVS history for you to use.

I'm 90% sure that the RelEng/IT repository will contain CVS history at
some point. Hal would know for sure.

[cja...@gmail.com]

> Unfortunately I don't have enough time for now to work on this even more,
> so you should not expect any more updates to the mozilla-central mirror. I
> may look at this again in the future and try other things that I can think
> of (such as trying to narrow down the failure range even further, or
> resorting to drastic measures such as redoing parts of the initial
> conversion process from scratch), but I don't know when I will have enough
> time to look at this again. If you're someone who knows things about git,
> mercurial or hg-git and wants to lend a hand, I can provide you with more
> information on how to reproduce the problem, and what I've tried and where
> to go from here.

I had the same problem with my Git mirror and I have it fixed for me. Ping me on IRC (nick jacek) and I'm happy to assist you with the problem. It shouldn't take much of your time.

Cheers,
Jacek

[cja...@gmail.com]

> Unfortunately I don't have enough time for now to work on this even more,
> so you should not expect any more updates to the mozilla-central mirror. I
> may look at this again in the future and try other things that I can think
> of (such as trying to narrow down the failure range even further, or
> resorting to drastic measures such as redoing parts of the initial
> conversion process from scratch), but I don't know when I will have enough
> time to look at this again. If you're someone who knows things about git,
> mercurial or hg-git and wants to lend a hand, I can provide you with more
> information on how to reproduce the problem, and what I've tried and where
> to go from here.

[Ehsan Akhgari]

OK, time for some good news. John Schoenick helped me yesterday on IRC
to understand what has caused this issue and how to fix it. I tried
that and it indeed fixed the problem. I'm currently in the process of
converting the rest of the repo to get to the current tip of
mozilla-central, which should take a couple of hours, and then the
mirror will start to update like before!

Cheers,
Ehsan

On 12-09-13 5:49 PM, Ehsan Akhgari wrote:
> Sad news everyone. Since Tuesday night I have been trying to fix the
> repository using almost anything that came to my mind, from trying to
> recreate the missing commits manually one by one to updating hg and
> hg-git, trying to reconvert only parts of the IonMonkey merge to narrow
> down the problem, and eliminating all of the possible factors that could
> have an impact on this conversion process, and all of my attempts have
> failed. I have sort of narrowed down the range of the IonMonkey merge
> which causes the problem, but narrowing it down further is a very time
> consuming process.
>

> Unfortunately I don't have enough time for now to work on this even
> more, so you should not expect any more updates to the mozilla-central
> mirror. I may look at this again in the future and try other things
> that I can think of (such as trying to narrow down the failure range
> even further, or resorting to drastic measures such as redoing parts of
> the initial conversion process from scratch), but I don't know when I
> will have enough time to look at this again. If you're someone who
> knows things about git, mercurial or hg-git and wants to lend a hand, I
> can provide you with more information on how to reproduce the problem,
> and what I've tried and where to go from here.
>

> In the mean time, if you've been using this mirror and want to continue
> using git, you may give this mirror a try:
> https://github.com/mozilla/releases-mozilla-central. There are a few
> downsides of doing so which you should be aware of:
>
> * This repo is created by RelEng by a direct conversion of the mercurial
> repository to git, which means that there is no CVS history for you to use.

> * This repo does not have an inbound branch like my mirror did, so if
> you want a commit which is on mozilla-inbound but not on mozilla-central
> yet, I guess you should wait until it gets merged to mozilla-central.

[Mike Hommey]

On Fri, Sep 14, 2012 at 08:41:19AM -0400, Ehsan Akhgari wrote:
> OK, time for some good news. John Schoenick helped me yesterday on
> IRC to understand what has caused this issue and how to fix it. I
> tried that and it indeed fixed the problem. I'm currently in the
> process of converting the rest of the repo to get to the current tip
> of mozilla-central, which should take a couple of hours, and then
> the mirror will start to update like before!

Could you share the details, for posterity?

Mike

[Ehsan Akhgari]

I am writing a blog post about what happened for future reference.

Cheers,
Ehsan

[Ehsan Akhgari]

As promised:

<http://ehsanakhgari.org/blog/2012-09-14/mozilla-central-git-mirror-back-up-again>

Ehsan