PSA: Shutting down my mozilla git mirror in three weeks

[Ehsan Akhgari]

Dear all,

For the past two and a half years I have been maintaining a git mirror of
mozilla-central plus a lot of the other branches that people found useful
at https://github.com/mozilla/mozilla-central. Over the years this proved
to be too much work for me to keep up with, and given the existence of the
new git mirrors that are supported by RelEng, I'm planning to shut down the
update jobs for this repository on Friday, Dec 13, 2013 and take the
repository down.

I strongly suggest that if you have been using and relying on this
repository before, please consider switching to the RelEng repositories as
soon as possible. https://github.com/mozilla/gecko-dev is the main
repository where you can find branches such as trunk/aurora/b2g
branches/etc and https://github.com/mozilla/gecko-projects is a repository
which contains project branches and twigs. (Note that my repository hosts
all of these branches in a single location, but from now on if you use both
of these branch subsets you will need to have two upstream remotes added in
your local git clone.)

The RelEng repositories have a similar history line including the CVS
history but they have completely different commit SHA1s, so pulling that
repo into your existing clones is probably not what you want. If you don't
have a lot of non-merged branches, your safest bet is cloning from scratch
and then port your existing branches manually. If you have a lot of local
branches, you may want to wait for the script that John Schoenick is
working on in bug 929338 which will assist you in rebasing those branches
on top of the new history line.

Last but not least, I really hate to have to disrupt your workflow like
this. I do hope that three weeks is enough advance notice for everybody to
successfully switch over to the new mirrors, but if you have a reason for
me to delay this please let me know and I will do my best to accommodate.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

[Aki Sasaki]

Github has closed access to https://github.com/mozilla/mozilla-central :

This repository has been disabled.
Access to this repository has been disabled by GitHub staff.
Contact support to restore access to this repository.

We're currently trying to reach github support to reverse this decision;
it would be much better to do a scheduled rollout than have to scramble
like this.

In the meantime, if you rely on git-based mozilla-central and can move
to gecko-dev sooner, please do so. gecko-dev is also on
git.mozilla.org, so should the same thing happen there, we have more
options.

If you require support to switch, let's take over #git and see if we can
figure this out together.

[Nicholas Nethercote]

On Mon, Nov 25, 2013 at 1:46 PM, Aki Sasaki <a...@mozilla.com> wrote:
> Github has closed access to https://github.com/mozilla/mozilla-central :
>
> This repository has been disabled.
> Access to this repository has been disabled by GitHub staff.
> Contact support to restore access to this repository.
>
> We're currently trying to reach github support to reverse this decision;
> it would be much better to do a scheduled rollout than have to scramble
> like this.

Are there plans to move Gaia from Github onto Mozilla servers? Having
portions of our primary products on third-party servers is a terrible
idea.

Nick

[Gregory Szorc]

It looks like GitHub decided to disable Ehsan's repo and all forks
without notice. https://bugzilla.mozilla.org/show_bug.cgi?id=943132
tracks it from our end.

> _______________________________________________
> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>

[hw...@mozilla.com]

On Monday, November 25, 2013 2:59:13 PM UTC-8, Nicholas Nethercote wrote:
>
> Are there plans to move Gaia from Github onto Mozilla servers? Having
> portions of our primary products on third-party servers is a terrible
> idea.

Already done long ago. :)

All repositories used to build FFOS, including gaia, are present on
git.mozilla.org. All of our official builds, and our partners pull from
git.mozilla.org just in case there is any problem with github. (You can
see this in the sources.xml in an official build.)

At the moment, the devs prefer to commit via github for some of the
repositories. Gecko commits must, of course, be made to the repositories
on hg.mozilla.org.

--Hal

[Nicholas Nethercote]

On Tue, Nov 26, 2013 at 1:12 PM, <hw...@mozilla.com> wrote:
>>
>> Are there plans to move Gaia from Github onto Mozilla servers? Having
>> portions of our primary products on third-party servers is a terrible
>> idea.
>
> Already done long ago. :)
>
> All repositories used to build FFOS, including gaia, are present on
> git.mozilla.org. All of our official builds, and our partners pull from
> git.mozilla.org just in case there is any problem with github. (You can
> see this in the sources.xml in an official build.)
>
> At the moment, the devs prefer to commit via github for some of the
> repositories.

What is required to change that preference?

Aki said above "We have read-only mirrors of gaia on both hg.m.o and
git.m.o." The read-only aspect would cause difficulties if the github
repo became unavailable.

Nick

[Aki Sasaki]

On 11/25/13 2:59 PM, Nicholas Nethercote wrote:
> On Mon, Nov 25, 2013 at 1:46 PM, Aki Sasaki <a...@mozilla.com> wrote:

>> Github has closed access to https://github.com/mozilla/mozilla-central :
>>
>> This repository has been disabled.
>> Access to this repository has been disabled by GitHub staff.
>> Contact support to restore access to this repository.
>>
>> We're currently trying to reach github support to reverse this decision;
>> it would be much better to do a scheduled rollout than have to scramble
>> like this.

> Are there plans to move Gaia from Github onto Mozilla servers? Having
> portions of our primary products on third-party servers is a terrible
> idea.

We have read-only mirrors of gaia on both hg.m.o and git.m.o.

Making Mozilla-hosted repos our repositories-of-record is a larger
discussion; that may get added momentum due to issues like this.

[Gary Kwong]

>> All repositories used to build FFOS, including gaia, are present on
>> git.mozilla.org. All of our official builds, and our partners pull from
>> git.mozilla.org just in case there is any problem with github. (You can
>> see this in the sources.xml in an official build.)
>>
>> At the moment, the devs prefer to commit via github for some of the
>> repositories.
>
> What is required to change that preference?
>
> Aki said above "We have read-only mirrors of gaia on both hg.m.o and
> git.m.o." The read-only aspect would cause difficulties if the github
> repo became unavailable.

I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=943172 to
consider changing Gaia on git.m.o or hg.m.o to be read/write instead.

-Gary

[Ehsan Akhgari]

FWIW the repository is back up now!

[Juan Gómez]

Well, I have a fork of mozilla/mozilla-central and it is not disabled. I
can push commits, browse it via web or clone it.
Anyway, What do you suggest? moving to git.mozilla.org instead of
github/gecko-dev?

On Tue, Nov 26, 2013 at 2:05 AM, Gregory Szorc <g...@mozilla.com> wrote:

> It looks like GitHub decided to disable Ehsan's repo and all forks without
> notice. https://bugzilla.mozilla.org/show_bug.cgi?id=943132 tracks it
> from our end.
>
>

>> _______________________________________________
>> firefox-dev mailing list
>> firef...@mozilla.org
>> https://mail.mozilla.org/listinfo/firefox-dev
>>
>>

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Ehsan Akhgari]

On 11/26/2013, 2:13 PM, Juan G�mez wrote:
> Well, I have a fork of mozilla/mozilla-central and it is not disabled. I
> can push commits, browse it via web or clone it.
> Anyway, What do you suggest? moving to git.mozilla.org

> <http://git.mozilla.org> instead of github/gecko-dev?

That problem was resolved around 2 hours ago or so.

Ehsan

[Julien Wajsberg]

Note, for git lovers, the git remote hg helper seems to work quite well.
see http://felipec.wordpress.com/2012/11/13/git-remote-hg-bzr-2/

As an occasional gecko developer I'm giving it a try right now and see
how it goes :)

Le 26/11/2013 20:13, Juan Gómez a écrit :
> Well, I have a fork of mozilla/mozilla-central and it is not disabled. I
> can push commits, browse it via web or clone it.

> _______________________________________________
> dev-b2g mailing list
> dev...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-b2g

[Mike Hommey]

On Wed, Nov 27, 2013 at 06:02:42AM +0100, Julien Wajsberg wrote:
> Note, for git lovers, the git remote hg helper seems to work quite well.
> see http://felipec.wordpress.com/2012/11/13/git-remote-hg-bzr-2/
>
> As an occasional gecko developer I'm giving it a try right now and see
> how it goes :)

Note that since then, git remote-hg has made it to the official git
repository, in contrib/remote-helpers.

https://github.com/git/git/blob/master/contrib/remote-helpers/git-remote-hg

It still has the drawback that it uses a local mercurial clone under the
hood, so a git/hg clone takes between 2 and 3 times as much space as it
should.

Mike

[Ehsan Akhgari]

Gentle reminder that this is happening this Friday. Ideally you should
have migrated your local git clones to the new repositories by now, but if
you haven't, please take some time to do this before this Friday.

Again, if you have a reason why I should postpone this please speak up!

Cheers,

--
Ehsan
<http://ehsanakhgari.org/>

[Ehsan Akhgari]

Here's another reminder everyone. This is happening *this Friday* at 1pm,
EST. I will shut down my update scripts on Friday, and will push a
non-fast-forward commit to all of the branches on that repo on Monday so
that people who do not see this message before that time would have a way
of knowing about this change.

If you haven't done already, *please* take a few minutes to do this right
now. If you hit problems, please ping me over email or IRC, or hop on #git.

Thanks!

--
Ehsan
<http://ehsanakhgari.org/>

[John Schoenick]

Good news everyone! I've prepared a helper script that will guide you
through moving from the old to the new git mirror:
https://github.com/Nephyrin/moz-git-migrator

To run it:
wget
https://raw.github.com/Nephyrin/moz-git-migrator/master/moz-git-migrator.sh
./moz-git-migrator.sh ./my-git-repo

The script won't modify your repo directly, but instead analyze it and
tell you what needs to be done. It's a bit tricky, so if you run into
any issues or cases it doesn't seem to handle properly, let me know!

- John

> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning

[Aki Sasaki]

I think it also makes sense to use this same timeframe for EOLing the
following experimental repos:

https://github.com/mozilla/releases-mozilla-central
https://github.com/mozilla/releases-mozilla-aurora
https://github.com/mozilla/releases-mozilla-beta
https://github.com/mozilla/releases-mozilla-release

These introduced a 3rd set of SHAs, and the above branches are supported
in gecko-dev.
I don't know of anyone who relies on these.
Please let me know if you do rely on these and can't make the switch to
gecko-dev in time.

[Ehsan Akhgari]

I just finished this work. https://github.com/mozilla/mozilla-central will
not be updated any more, and I will replace its content with a short notice
explaining the situation on Monday.

Cheers,

--
Ehsan
<http://ehsanakhgari.org/>

[Aki Sasaki]

Similarly, the sync processes for these four repos have been stopped,
and we'll replace their content on Monday.

On 12/11/13 4:25 PM, Aki Sasaki wrote:
> I think it also makes sense to use this same timeframe for EOLing the
> following experimental repos:
>
> https://github.com/mozilla/releases-mozilla-central
> https://github.com/mozilla/releases-mozilla-aurora
> https://github.com/mozilla/releases-mozilla-beta
> https://github.com/mozilla/releases-mozilla-release
>
> These introduced a 3rd set of SHAs, and the above branches are supported
> in gecko-dev.
> I don't know of anyone who relies on these.
> Please let me know if you do rely on these and can't make the switch to
> gecko-dev in time.
>

[Gregory Szorc]

Ehsan,

I'm sure I'm speaking for many Mozillians when I say thank you for running this service - from building on Chris Double's initial work, keeping it running while dealing with GitHub shenanigans, and for assisting with a transition plan to its natural successor. You could easily have done nothing, but you stepped up and enabled countless people to access the Firefox source code on their terms. If that's not worthy of spamming dev-planning for, I don't know what is.

Thanks again,

Gregory

On Dec 14, 2013, at 5:28, Ehsan Akhgari <ehsan....@gmail.com> wrote:

> I just finished this work. https://github.com/mozilla/mozilla-central will not be updated any more, and I will replace its content with a short notice explaining the situation on Monday.
>