Data migration from ftp.mozilla.org to S3, 1pm Pacific Tuesday 20th October

Nick Thomas

unread,

Oct 20, 2015, 8:37:47 AM10/20/15

to

Starting today at 1pm a subset of directories will be migrated from the
FTP server to S3 - firefox, mobile, thunderbird, and calendar. The
window for this work is two hours long, and the trees will stay open
(unless things go very badly). This is part of a project lead by the
Cloud Ops group, assisted by Release Engineering .

Things you should know
* existing links to ftp.mozilla.org will continue to work, although
archive.mozilla.org is now considered the canonical domain
* while the data has been copied over, during the cutover the most
recent files may not available on the new system. Synchronization will
continue after the cutover to backfill them
* treeherder will continue to show the results of jobs, but the logs may
be affected by the previous point. Treeherder is designed to retrieve
the log again when necessary
* tests will continue to run because they use a separate storage system
(except for mozilla-beta, mozilla-release and mozilla-esr38)
* paths like firefox/tinderbox-builds/*/latest will stop being updated,
and later will be removed (they were originally added for gaia-try, and
that is now defunct)
* in the near future directories like firefox/releases/latest/ will be
emptied. README files with alternative options will be provided

Please contact the sheriff, or #releng, if you want to bring any issues
to our attention. bug 1211732 is tracking bug.

Nick Thomas (Release Engineering)

EE

unread,

Oct 20, 2015, 4:22:46 PM10/20/15

to

What is S3? Does that mean that the newer versions will be at some
other address, and if so, where?

Nick Thomas

unread,

Oct 21, 2015, 5:33:50 AM10/21/15

to EE

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 21/10/15 9:22 am, EE wrote:
> What is S3? Does that mean that the newer versions will be at
> some other address, and if so, where?

I mean Amazon S3, see http://aws.amazon.com/s3/ for more details.

Really we're just changing the backend storage to a cloud system. You
can continue to use https://ftp.mozilla.org to reach the usual places,
although we encourage people to move to https://archive.mozilla.org.
Same data, with a new name that doesn't default to the ftp protocol in
Firefox.

There will be some gaps in the data for a few days, notably in
firefox/nightly
firefox/tinderbox-builds
firefox/try-builds
These will disappear as data is synced up to S3.

Cheers
Nick Thomas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCgAGBQJWJ1vyAAoJENO8WgCvFBJMBPEH+wRUlo98atkH2Th2qVhpR653
Ua1F4CRYRtoCJWvbpDxgN/uKIxiL6xFA48YuaKpQaY+x8Usg19P5INY8LN+EAyNK
x8/rdyVdDqW3jPQqfIKkqWfB7RGOzjRr7/vUik7cH2MWjE9Aw2AcHGdkgPCo4nJw
/H9CsjG2BQ6DmtjP9VtsymSU9bCWlrtLJa6qbNBIyFOTGytvYqrKqC0B6IcJZXq2
96G+GjYSoM5j7fKCsHQeBqai3SehagaUuGUWId5+Bs/fSKTF7xLZZoYzHkf14oCt
6Pl9zeZaOZ1Xdzmuqad16yPqxQPAq4O8y7cJCKakf6y+dC4/bQvwRIdsH5/4dM4=
=U8gQ
-----END PGP SIGNATURE-----

Nick Thomas

unread,

Oct 21, 2015, 5:46:34 AM10/21/15

to

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

An update on how this went. We are now transitioned to S3 storage for
firefox, mobile, and thunderbird. calendar will move over in the near
future. There was a long tree closure, sorry for the length of that.

The major issues still in play are:
* data synchronization to S3 is ongoing, and should be complete in the
next few days. There are gaps in firefox/nightly,
firefox/tinderbox-builds, and firefox/try-builds
* to reach older builds you can use
http://ftp-origin-scl3.mozilla.org/pub/mozilla.org/firefox/
* please don't get comfortable with that, we'll just be using
archive.mozilla.org soon
* we apologize for any inconvience this may cause you, it was not the
intention before the migration

* try builds will fail to upload (and run tests!) unless you push your
changes on top of recent tip of inbound, specifically
https://hg.mozilla.org/integration/mozilla-inbound/rev/0ee21e8d5ca6 or
later (bug 1216907)

* the hazard builds aren't uploading their analysis files (bug 1216859)

* spidermonkey builds and logs are invisible until bug 1216859 is comple
te

* if you open a raw log in treeherder it will download instead of
shown in a new tab. This is part of a wider problem with the
Content-Type header not being set (bug 1216804).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCgAGBQJWJ171AAoJENO8WgCvFBJMlt0H/0iS7kJAIdOxqZwjAFnweMaA
aXHVhtcB3GbPWiwd9Yfxy2Q/lV+zVF6mVUFpoRV5kHRGZzCp9IdcRaIyNJ8fVdmo
y9hUZk9qcwEkRyd5SI/AH5gpzOL79e1ubq9Vv2DYtnrLvBBohK2M7zZ1lT4N6PJv
sPzISzwlMj9Kz8CMQKfgngJPqzHbrzKTL/JsR9Oc2JKKgUjYA0RJYCX62+AIpzJp
AuslJ9wruNQRkukyZdsZKWANO4A/XDKx8OsIzl/w7UoAWmcpqWy/ZnFaNs/1N6hW
F5Xcj2ML9BDgm8D9xpDZoPWYbSsET2C/r6Engu1+gmVQWezCJqmjbjH5gj+TZJQ=
=KdrQ
-----END PGP SIGNATURE-----

Henrik Skupin

unread,

Oct 21, 2015, 8:32:57 AM10/21/15

to Nick Thomas

Nick Thomas wrote on 10/21/2015 11:46 AM:

> * to reach older builds you can use
> http://ftp-origin-scl3.mozilla.org/pub/mozilla.org/firefox/
> * please don't get comfortable with that, we'll just be using
> archive.mozilla.org soon

For how long will the ftp-origin-scl3.mozilla.org host be online? We had
to temporarily switch mozdownload to make use of that host, to
circumvent the gaps of builds and the changed HTML structure for
parsing. It will take us some days to get it all fixed and I don't want
to use archive.mozilla.org again as long as not all builds have been
transfered over.

Thanks

--
Henrik

gary...@gmail.com

unread,

Oct 21, 2015, 12:32:54 PM10/21/15

to

When a file such as a .txt file is opened, as opposed to saved, via the download manager it is stored in the Temp folder as read-only. Is this by design because I don't think it worked this way before.

Gijs Kruitbosch

unread,

Oct 21, 2015, 1:00:22 PM10/21/15

to gary...@gmail.com

On 21/10/2015 17:32, gary...@gmail.com wrote:
> When a file such as a .txt file is opened, as opposed to saved, via the download manager it is stored in the Temp folder as read-only. Is this by design because I don't think it worked this way before.

This isn't really the right group to be asking this, but yes, this is by
design, see https://bugzilla.mozilla.org/show_bug.cgi?id=1009465 and
https://bugzilla.mozilla.org/show_bug.cgi?id=1022816 . It not working
like this for a while was a regression, which these bugs fixed.
Specifically, https://bugzilla.mozilla.org/show_bug.cgi?id=1009465#c9
indicates the rationale for Firefox behaving this way.

~ Gijs

Steve Wendt

unread,

Oct 21, 2015, 1:35:56 PM10/21/15

to

On 10/21/2015 2:33 AM, Nick Thomas wrote:

> although we encourage people to move to https://archive.mozilla.org.

The column header sorting (e.g. ?C=M;O=D) has been broken for a couple
weeks, and I see that both firefox and thunderbird don't even show dates
in the listings any more:
http://archive.mozilla.org/pub/firefox/releases/

SeaMonkey still has the nicer output, but with the broken sorting:
http://archive.mozilla.org/pub/mozilla.org/seamonkey/releases/?C=M;O=D

Will this be fixed, or is it going to continue to be a worse experience
than it ever was with FTP?

EE

unread,

Oct 21, 2015, 2:36:22 PM10/21/15

to

HTTPS does not default to FTP protocol even when the prefix is "ftp.",
so https:\\ftp.mozilla.org will use secure HTTP.

Philip Chee

unread,

Oct 23, 2015, 2:25:33 PM10/23/15

to

An Alphabet search indicates that you can run a FTP server using a S3
bucket as the backend. Just saying.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.