MXR permanently offline, please transition to DXR

[Lawrence Mandel]

Mozilla Cross-Reference, better known as MXR (https://mxr.mozilla.org), was
taken offline on June 13, 2016, to investigate a potential security issue.
After careful review of the codebase, we have decided to accelerate the
planned transition from MXR to its more modern equivalent, DXR (
https://dxr.mozilla.org), instead of bringing MXR back online. As far as we
know there was never a security compromise, but the unsupported legacy
codebase (forked from an old version of LXR) would require significant time
and effort to rewrite and bring up to spec.

Our transition plan is as follows:


-

Add an interstitial web page at https://mxr.mozilla.org that displays a
best-guess URL for the equivalent https://dxr.mozilla.org file data.
This will help interactive users retrieve data from historical links in
applications like Bugzilla.
-

Redirect certdata.txt and effective_tld_names.dat to their canonical
source code repositories instead of DXR. All other search queries and
automatic pulling of raw files by third parties will no longer be supported
at the https://mxr.mozilla.org URL.
-

Index the remaining repos listed in MXR in DXR for data parity, using
bug 1281443 to track progress. Repos will be indexed in the order listed
unless otherwise specified. If you need to prioritize the indexing of
specific repos, please open a bug and block against bug 1281443.


Our expectation is that the interstitial page will be in place and the
following remaining high-priority repos will be indexed by June 24th, 2016:

-

add-ons
-

servo
-

l10n


If you have concerns, questions, or requests, please open a new bug and
mark it as blocking bug 1281443 or add a comment to one of its existing
dependent bugs. Additional status updates will also be posted to bug
1281443 and its dependent bugs.

Lawrence

[Ms2ger]

On 22/06/16 20:30, Lawrence Mandel wrote:
> Mozilla Cross-Reference, better known as MXR (https://mxr.mozilla.org), was
> taken offline on June 13, 2016, to investigate a potential security issue.
> After careful review of the codebase, we have decided to accelerate the
> planned transition from MXR to its more modern equivalent, DXR (
> https://dxr.mozilla.org), instead of bringing MXR back online.

I wish the years of complaining about MXR had led to an equally useful
replacement for it by now.

Sad to see it go.

Ms2ger

[smaug]

searchfox.org actual is a very good alternative now. It has good integration with blame and at least so far it has
always given me the results what I've been looking for. (with dxr one may get occasionally rather unexpected results).
Just need to get addons/release/beta/aurora trees to searchfox.

[Cameron Kaiser]

As am I. I got a lot of miles out of it, especially for searching
historical source trees.

Cameron Kaiser
mxr? I dn't evn knw hr

[Edmund Wong]

Ditto. It's been my mainstay for searching code... couldn't dxr have
a similar ui and does it have to default to mozilla-central?

Edmund

[Philip Chee]

I wonder what is necessary to set up an instance of MXR (for comm-*) on
our own server (or vps). I would guess PERL, hg, and a Linux VM.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.

[glob]

Philip Chee wrote:
>
> I wonder what is necessary to set up an instance of MXR (for comm-*) on
> our own server (or vps). I would guess PERL, hg, and a Linux VM.

mxr was shutdown due to some very serious security issues; i strongly
advise against standing up your own instance unless you first put a lot
of time against securing it.

you'd be better served by deploying an alternative source browser.


-glob

[Mike Hoye]

On 2016-06-24 6:20 AM, Philip Chee wrote:
>
> I wonder what is necessary to set up an instance of MXR (for comm-*) on
> our own server (or vps). I would guess PERL, hg, and a Linux VM.

I've got the impression that comm-* has enough rocks to push up the
legacy-stack hill already.

- mhoye

[Philip Chee]

MXR uses black text (#000000) on a white background (#FFFFFF).
DXR uses grey goop that pretends to be text and gives me eye-strain.

MXR gives me a choice of {mozilla|comm}-{central|aurora|beta|release|esrNN}
Plus legacy repositories like mozilla-1.8 1.9 1.9.2 etc

DXR only gives me mozilla-central.

[Douglas Turner]

Philip,
Can you please file bugs. There is no need to discuss your specific eye
strain issues or what DXR should or should not index on dev-platform.

Also, the point that people are making here is that MXR wasn't safe to run
and we are fixing that by decommissioning. You're more than welcome to
deploy this old software but you're going to get shell pop'ed and have a
really bad day. And, trust me, those days really bad days suck.

Thanks,
Doug Turner

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Stefan Sitter]

On 25.06.2016 06:45, Philip Chee wrote:
> DXR only gives me mozilla-central.

You can choose different repositories using the Switch Tree drop-down
menu on the top right. Or keep a bookmark to start with a specific
repository like https://dxr.mozilla.org/comm-central/source/

/Stefan

[Joshua Cranmer 🐧]

On 6/24/2016 11:45 PM, Philip Chee wrote:
> DXR only gives me mozilla-central.

DXR has indexed comm-central for very nearly its entire time hosted at
Mozilla.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

[Edmund Wong]

Correction: boulders. :P

[Erik Rose]

Thanks for pointing this out! We've increased the contrast in https://github.com/mozilla/dxr/commit/a0b7afeb82bc9903d8c80494fb487b93ef280b70. Do feel free to file bugs in the future.

Cheers,
Erik

[Tanvi Vyas]

Is it possible to safely redirect mxr to dxr?

When I use my awesomebar and type "docshell", it pulls up
https://mxr.mozilla.org/mozilla-central/source/docshell/base/nsDocShell.cpp.
I click enter and end up at the mxr error page. So instead I do a dxr
search for docshell and scroll through a list of results, none of which
seem to give me a link to nsDocShell.cpp. It would be great if the mxr
link sent me directly to
https://dxr.mozilla.org/mozilla-central/source/docshell/base/nsDocShell.cpp.

Alternatively, I could create an addon that does this or try and write a
script that would change my history entries.

Thanks!

~Tanvi

On 6/22/16 11:30 AM, Lawrence Mandel wrote:
>
> Mozilla Cross-Reference, better known as MXR
> (https://mxr.mozilla.org), was taken offline on June 13, 2016, to
> investigate a potential security issue. After careful review of the
> codebase, we have decided to accelerate the planned transition from

> MXR to its more modern equivalent, DXR (https://dxr.mozilla.org),

> instead of bringing MXR back online. As far as we know there was never
> a security compromise, but the unsupported legacy codebase (forked
> from an old version of LXR) would require significant time and effort
> to rewrite and bring up to spec.
>
>
> Our transition plan is as follows:
>
>

> *

>
> Add an interstitial web page at https://mxr.mozilla.orgthat
> displays a best-guess URL for the equivalent

> https://dxr.mozilla.orgfile data. This will help interactive users

> retrieve data from historical links in applications like Bugzilla.
>

> *

>
> Redirect certdata.txt and effective_tld_names.dat to their
> canonical source code repositories instead of DXR. All other
> search queries and automatic pulling of raw files by third parties
> will no longer be supported at the https://mxr.mozilla.orgURL.
>

> *

>
> Index the remaining repos listed in MXR in DXR for data parity,
> using bug 1281443 to track progress. Repos will be indexed in the
> order listed unless otherwise specified. If you need to prioritize
> the indexing of specific repos, please open a bug and block
> against bug 1281443.
>
>
> Our expectation is that the interstitial page will be in place and the
> following remaining high-priority repos will be indexed by June 24th,
> 2016:
>

> *
>
> add-ons
>
> *
>
> servo
>
> *

>
> l10n
>
>
> If you have concerns, questions, or requests, please open a new bug
> and mark it as blocking bug 1281443 or add a comment to one of its
> existing dependent bugs. Additional status updates will also be posted
> to bug 1281443 and its dependent bugs.
>
>
> Lawrence
>
>
>

> _______________________________________________
> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev

[Mats Palmgren]

On 06/28/16 01:01, Tanvi Vyas wrote:
> Is it possible to safely redirect mxr to dxr?

This would be most welcome. There are lots of pasted MXR links
in Bugzilla comments which now requires tedious editing to
follow.

/Mats

[Dão Gottwald]

Can we please automatically redirect from
https://mxr.mozilla.org/mozilla-central/source/x/y.z to
https://dxr.mozilla.org/mozilla-central/source/x/y.z? My browsing history
is littered with mxr URLs which used to make it very easy to find a file by
typing part of the name. As it stands all those URLs are broken.

2016-06-22 20:30 GMT+02:00 Lawrence Mandel <lma...@mozilla.com>:

> Mozilla Cross-Reference, better known as MXR (https://mxr.mozilla.org),
> was taken offline on June 13, 2016, to investigate a potential security
> issue. After careful review of the codebase, we have decided to accelerate
> the planned transition from MXR to its more modern equivalent, DXR (
> https://dxr.mozilla.org), instead of bringing MXR back online. As far as
> we know there was never a security compromise, but the unsupported legacy
> codebase (forked from an old version of LXR) would require significant time
> and effort to rewrite and bring up to spec.
>
> Our transition plan is as follows:
>
>

> -
>
> Add an interstitial web page at https://mxr.mozilla.org that displays
> a best-guess URL for the equivalent https://dxr.mozilla.org file data.

> This will help interactive users retrieve data from historical links in
> applications like Bugzilla.

> -

>
> Redirect certdata.txt and effective_tld_names.dat to their canonical
> source code repositories instead of DXR. All other search queries and
> automatic pulling of raw files by third parties will no longer be supported
> at the https://mxr.mozilla.org URL.

> -

>
> Index the remaining repos listed in MXR in DXR for data parity, using
> bug 1281443 to track progress. Repos will be indexed in the order listed
> unless otherwise specified. If you need to prioritize the indexing of
> specific repos, please open a bug and block against bug 1281443.
>
>
> Our expectation is that the interstitial page will be in place and the
> following remaining high-priority repos will be indexed by June 24th, 2016:
>

> -
>
> add-ons
> -
>
> servo
> -
>

[Mats Palmgren]

On 06/28/16 01:01, Tanvi Vyas wrote:

> Is it possible to safely redirect mxr to dxr?

[Erik Rose]

Hi, Kendall. As a pain mitigation strategy for MXR URLs embedded immutably in Bugzilla and in people's Awesomebar histories, can we redirect MXR requests as Dão suggests? Some won't work, but many will, and those people will be less sad.

[Gregory Szorc]

On Thu, Jun 30, 2016 at 7:20 AM, Dão Gottwald <dgot...@mozilla.com> wrote:

> Can we please automatically redirect from
> https://mxr.mozilla.org/mozilla-central/source/x/y.z to
> https://dxr.mozilla.org/mozilla-central/source/x/y.z? My browsing history
> is littered with mxr URLs which used to make it very easy to find a file by
> typing part of the name. As it stands all those URLs are broken.
>

Tangent: can we get a web feature that allows site operators to publish
"redirect rules" so user agents update references to URLs that now HTTP
301? It makes me sad as an end user every time a domain redirects and my
awesomebar becomes less awesome because the presence of multiple domains
throws off the algorithm. And my bookmarks aren't current. And history
search yields unexpected results. And ... As a site operator, it makes me
sad that I have to keep old domains living forever. Sure, this is the point
of URLs. But sometimes I just want to pull the plug and not have to deal
with the domain registration, x509 certificates, rules in my HTTP server,
etc.

[Robert O'Callahan]

In theory responses 301 and 308 mean "permanent redirect" so the browser
could do that for those responses.

In practice you'd need a lot of data to convince yourself that Web
developers haven't screwed this up too badly. Maybe 308, being newer, is
not compromised...

Rob
--
lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf
toD
selthor stor edna siewaoeodm or v sstvr esBa kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea
lurpr
.a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

[Martin Thomson]

On Fri, Jul 1, 2016 at 9:55 AM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> In theory responses 301 and 308 mean "permanent redirect" so the browser
> could do that for those responses.

Those would only work for as long as the 3xx is remembered, and it
wouldn't work for /x if you have only seen /y redirect.

To gps' question, yes, such a feature would be awesome, but it's
hairy. I might be working on something that would do that, though for
almost unrelated reasons.

[Mike Hommey]

How would user agents get "redirect rules" if you pull the plug on the
domain registration?

Mike

[Gregory Szorc]

On Thu, Jun 30, 2016 at 4:55 PM, Robert O'Callahan <rob...@ocallahan.org>

wrote:

>
> In theory responses 301 and 308 mean "permanent redirect" so the browser
> could do that for those responses.
>

> In practice you'd need a lot of data to convince yourself that Web
> developers haven't screwed this up too badly. Maybe 308, being newer, is
> not compromised...
>
>

To be explicit, I want a tangential feature to HTTP redirects. HTTP
redirects only work on single URLs. User agents need to hit/crawl every URL
and update accordingly. I want the site to publish a "URL translation map"
for URL patterns so whole URL namespaces can be bulk updated. Basically
exposing the URL rewrite rules from the HTTP server. Perhaps search engines
already support such a feature to aid spidering?

To address glandium's point, obviously this would likely only work if the
old domain is still alive, as there are obvious issues with domain X trying
to update URLs from domain Y! Although maybe there are tricks to establish
a trusted link from old domain to new. (I'm far from an expert here.)

[Justin Dolske]

This reminds me of a password manager bug we fixed 9 years ago (379997!),
where password manager would "helpfully" delete a saved HTTP login if it
got a 403 response upon using it. Unsurprisingly, this was a a terrible
idea that caused your saved logins to disappear when a site was glitchy.

Seem like it would be tough to find a solution to rewrite history/bookmarks
that works when servers are nice, but also ignores servers that are
naughty. Maybe this is addon-fodder for advanced users who want to
mass-edit bookmarks and history.

Justin

On Thu, Jun 30, 2016 at 4:55 PM, Robert O'Callahan <rob...@ocallahan.org>
wrote:

> In theory responses 301 and 308 mean "permanent redirect" so the browser
> could do that for those responses.
>
> In practice you'd need a lot of data to convince yourself that Web
> developers haven't screwed this up too badly. Maybe 308, being newer, is
> not compromised...
>

> Rob
> --
> lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf
> toD
> selthor stor edna siewaoeodm or v sstvr esBa kbvted,t
> rdsme,aoreseoouoto
> o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea
> lurpr
> .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr
> esn

[Panos Astithas]

It seems like the awesomebar could at least help you by boosting the
frecency weight of the new URL compared to the old one, so it can gradually
(or even not so gradually) be replaced. We are going to fix this in bug 737836.


Panos

[Chris H-C]

For now, can we get https://mxr.mozilla.org/ to point to something other
than the "Repairs in Progress" hardhat? A redirect to dxr would not be
amiss, methinks.

[Philip Chee]

On 28/06/2016 01:38, Erik Rose wrote:
> Thanks for pointing this out! We've increased the contrast in
> https://github.com/mozilla/dxr/commit/a0b7afeb82bc9903d8c80494fb487b93ef280b70.
> Do feel free to file bugs in the future.

ThankyouThankyouThankyouThankyouThankyouThankyouThankyouThankyou!

> Cheers, Erik

My hero!

>> On Jun 25, 2016, at 0:45 , Philip Chee <phili...@gmail.com>
>> wrote:
>>
>> MXR uses black text (#000000) on a white background (#FFFFFF). DXR
>> uses grey goop that pretends to be text and gives me eye-strain.

[Philip Chee]

On 25/06/2016 13:00, Douglas Turner wrote:
> Also, the point that people are making here is that MXR wasn't safe to run
> and we are fixing that by decommissioning. You're more than welcome to
> deploy this old software but you're going to get shell pop'ed and have a
> really bad day. And, trust me, those days really bad days suck.

I have no idea what "shell pop'ed" means and Google tells me that the
last MXR security bug was fixed in 24.06.2015.

[Boris Zbarsky]

On 6/30/16 3:56 PM, Erik Rose wrote:
> Hi, Kendall. As a pain mitigation strategy for MXR URLs embedded immutably in Bugzilla and in people's Awesomebar histories, can we redirect MXR requests as Dão suggests?

This really needs to happen. I've hit broken mxr links from bugzilla
comments at least 7 or 8 times so far, and I was on vacation for most of
the time mxr has been down. It's a pretty serious productivity drag.

-Boris

[Peter Elmers]

If it helps anyone, I happen to know that there exists an addon which
visits all anchor tags on a page and rewrites mxr.mozilla.org to
dxr.mozilla.org. https://addons.mozilla.org/en-US/firefox/addon/mxr-to-dxr/

Disclaimer: I'm the author.

[Lawrence Mandel]

dev-platform was not included on my response below. Looping back in to this
fork of the thread.

On Fri, Jul 8, 2016 at 10:55 AM, Lawrence Mandel <lma...@mozilla.com>
wrote:

> Sorry Dao. I have seen some responses. Maybe they were off list. We're
> working on details now. I'm going to get someone to put the redirects in
> place, likely with an interstitial page advising that MXR has been
> decommissioned, by next week.
>
> Lawrence
>
>
> On Friday, 8 July 2016, Dão Gottwald <dgot...@mozilla.com> wrote:
>
>> Why has nobody responded to the requests for a short-term fix for MXR
>> URLs for more than a week? Are the people responsible for MXR not in this
>> list?
>>
>> 2016-07-07 18:23 GMT+02:00 Eric Shepherd <eshe...@mozilla.com>:
>>
>>> We have tons of mxr links all through MDN, fwiw. I am updating the
>>> macros that generate them, but odds are very good these links will be
>>> around for a good while.
>>>
>>>
>>> That would be perfectly fine for my purposes, I expect, as long as it
>>> dealt with the relevant mxr features. What I want is for links to possibly
>>> specific lines of possibly specific revisions of specific files to work.
>>> Ideally with the highlighting bits too.
>>>
>>>
>>> --
>>>
>>> Eric Shepherd
>>> Senior Technical Writer
>>> Mozilla Developer Network <https://developer.mozilla.org/>
>>> Blog: https://www.bitstampede.com/
>>> Twitter: http://twitter.com/sheppy
>>> Doodle: http://doodle.com/the.sheppy

[Bobby Holley]

Can we skip the interstitial page and make the notice (if any) more
unobtrusive somehow? There are tons of mxr links all over the place, and
many of them are immutable. We don't gain anything by informing the viewer
about their obsolescence instead of showing them the content they want.

On Fri, Jul 8, 2016 at 12:20 PM, Lawrence Mandel <lma...@mozilla.com>
wrote:

[Lawrence Mandel]

We do in the case of 3rd party software referencing files from MXR (and I'm
told there is a lot of this). We also can't guarantee that MXR URLs will
direct to the right place in DXR. There is likely a balance to be struck
here with highly referenced files from 3rd party software getting an
interstitial page and other files not getting the page. Let's start with
getting the page with the redirect in place and then iterate from there as
required.

Lawrence

[Gijs Kruitbosch]

The problem is that if you're on a bmo page with these "perma"links,
really the ideal case is that they are indeed permalinks and continue to
work, especially where they point to specific lines in specific
revisions. The interstitial just makes it harder for people to get to
that info. A 404 on a working DXR, if the file has really disappeared or
something, is still better than the 'hard hat' page from which the path
to the data you want is much longer.

In case this is useful for folks: I wrote a webextension that rewrites
these links and that obeys the "rev" and "mark" query params from MXR
links and rewrites them to the equivalent DXR URL syntax:
https://addons.mozilla.org/en-US/firefox/addon/mxr-to-dxr-webextension/ .

~ Gijs

On 08/07/2016 20:27, Lawrence Mandel wrote:
> We do in the case of 3rd party software referencing files from MXR
> (and I'm told there is a lot of this). We also can't guarantee that
> MXR URLs will direct to the right place in DXR. There is likely a
> balance to be struck here with highly referenced files from 3rd party
> software getting an interstitial page and other files not getting the
> page. Let's start with getting the page with the redirect in place and
> then iterate from there as required.
>
> Lawrence
>
> On Fri, Jul 8, 2016 at 3:24 PM, Bobby Holley <bobby...@gmail.com
> <mailto:bobby...@gmail.com>> wrote:
>
> Can we skip the interstitial page and make the notice (if any)
> more unobtrusive somehow? There are tons of mxr links all over the
> place, and many of them are immutable. We don't gain anything by
> informing the viewer about their obsolescence instead of showing
> them the content they want.
>
> On Fri, Jul 8, 2016 at 12:20 PM, Lawrence Mandel
> <lma...@mozilla.com <mailto:lma...@mozilla.com>> wrote:
>
> dev-platform was not included on my response below. Looping
> back in to this
> fork of the thread.
>
> On Fri, Jul 8, 2016 at 10:55 AM, Lawrence Mandel

> <lma...@mozilla.com <mailto:lma...@mozilla.com>>

> wrote:
>
> > Sorry Dao. I have seen some responses. Maybe they were off
> list. We're
> > working on details now. I'm going to get someone to put the
> redirects in
> > place, likely with an interstitial page advising that MXR
> has been
> > decommissioned, by next week.
> >
> > Lawrence
> >
> >
> > On Friday, 8 July 2016, Dão Gottwald <dgot...@mozilla.com

> <mailto:dgot...@mozilla.com>> wrote:
> >
> >> Why has nobody responded to the requests for a short-term
> fix for MXR
> >> URLs for more than a week? Are the people responsible for
> MXR not in this
> >> list?
> >>
> >> 2016-07-07 18:23 GMT+02:00 Eric Shepherd

> <eshe...@mozilla.com <mailto:eshe...@mozilla.com>>:

> >>
> >>> We have tons of mxr links all through MDN, fwiw. I am
> updating the
> >>> macros that generate them, but odds are very good these
> links will be
> >>> around for a good while.
> >>>
> >>>
> >>> That would be perfectly fine for my purposes, I expect, as
> long as it
> >>> dealt with the relevant mxr features. What I want is for
> links to possibly
> >>> specific lines of possibly specific revisions of specific
> files to work.
> >>> Ideally with the highlighting bits too.
> >>>
> >>>
> >>> --
> >>>
> >>> Eric Shepherd
> >>> Senior Technical Writer
> >>> Mozilla Developer Network <https://developer.mozilla.org/>
> >>> Blog: https://www.bitstampede.com/
> >>> Twitter: http://twitter.com/sheppy
> >>> Doodle: http://doodle.com/the.sheppy
> >>>
> >>>

> >>> _______________________________________________
> >>> firefox-dev mailing list
> >>> firef...@mozilla.org <mailto:firef...@mozilla.org>

> >>> https://mail.mozilla.org/listinfo/firefox-dev
> >>>
> >>>
> >>
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org

> <mailto:dev-pl...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/dev-platform

[Steve Fink]

Are either mxr or dxr really the right thing for canonical links to our
source code? As long as we're updating links, through one means or
another, temporarily or permanently, couldn't we come up with a set of
urls that would be better in the long term? One that clearly states the
purpose of a link (eg a link to the latest version of a whole file vs a
link to a range of lines in a specific version vs a link to a named
function definition or whatever), and could be redirected when the next
great thing comes along?

I think of dxr as a code searching site, not an archival one.
hg.mozilla.org would seem to be better for versioned links, although I
would optimize for different performance characteristics and feature
sets than if I were setting up an archival service, so in my opinion
it's not the best choice either.

I'm not saying we should block everything on some hypothetical new
archival server, just that an additional layer of indirection might be
well-placed here. The archival urls can do some simple forwarding for
now. (And it would be nice for dxr/searchfox to offer up archival links
with these urls.) It'd even be fine if the archival versions pointed
back into dxr and/or hg.m.o or whatever.

Just a thought.

On 07/08/2016 12:27 PM, Lawrence Mandel wrote:
> We do in the case of 3rd party software referencing files from MXR (and I'm
> told there is a lot of this). We also can't guarantee that MXR URLs will
> direct to the right place in DXR. There is likely a balance to be struck
> here with highly referenced files from 3rd party software getting an
> interstitial page and other files not getting the page. Let's start with
> getting the page with the redirect in place and then iterate from there as
> required.
>
> Lawrence
>

> On Fri, Jul 8, 2016 at 3:24 PM, Bobby Holley <bobby...@gmail.com> wrote:
>
>> Can we skip the interstitial page and make the notice (if any) more
>> unobtrusive somehow? There are tons of mxr links all over the place, and
>> many of them are immutable. We don't gain anything by informing the viewer
>> about their obsolescence instead of showing them the content they want.
>>
>> On Fri, Jul 8, 2016 at 12:20 PM, Lawrence Mandel <lma...@mozilla.com>

>> wrote:
>>
>>> dev-platform was not included on my response below. Looping back in to
>>> this
>>> fork of the thread.
>>>
>>> On Fri, Jul 8, 2016 at 10:55 AM, Lawrence Mandel <lma...@mozilla.com>

[Boris Zbarsky]

On 7/8/16 3:23 PM, Lawrence Mandel wrote:
> On Fri, Jul 8, 2016 at 11:10 AM, Boris Zbarsky <bzba...@mit.edu
> <mailto:bzba...@mit.edu>> wrote:
> Why do we need the interstitial page for direct links to files?
> What action is expected to be taken by the person who followed that
> link, other than having to do extra clicks?
>
> Few things:
> - Want to encourage people to update their links to DXR.

This won't help with existing links from Bugzilla, obviously.

> - Lots of 3rd party software is directly linking to files in MXR. We
> want to provide a path to fixing these links now that MXR is offline.

OK. Sounds like you may want to skip the interstitial based on referrer
or something. Because having this interstitial for bugzilla links just
sounds like a PITA.

> - Not all links will work.

Which ones won't? Why won't they work?

-Boris

[Lawrence Mandel]

I filed bug 1285657 to rewrite MXR links in Bugzilla to DXR. Are there
other sites (MDN) that have a lot of links that we should look to rewrite?

Lawrence

On Fri, Jul 8, 2016 at 3:49 PM, Gijs Kruitbosch <gijskru...@gmail.com>
wrote:

> The problem is that if you're on a bmo page with these "perma"links,
> really the ideal case is that they are indeed permalinks and continue to
> work, especially where they point to specific lines in specific revisions.
> The interstitial just makes it harder for people to get to that info. A 404
> on a working DXR, if the file has really disappeared or something, is still
> better than the 'hard hat' page from which the path to the data you want is
> much longer.
>
> In case this is useful for folks: I wrote a webextension that rewrites
> these links and that obeys the "rev" and "mark" query params from MXR links
> and rewrites them to the equivalent DXR URL syntax:
> https://addons.mozilla.org/en-US/firefox/addon/mxr-to-dxr-webextension/ .
>
> ~ Gijs
>
>

>>> >>> Blog: <https://www.bitstampede.com/>https://www.bitstampede.com/
>>> >>> Twitter: <http://twitter.com/sheppy>http://twitter.com/sheppy
>>> >>> Doodle: <http://doodle.com/the.sheppy>http://doodle.com/the.sheppy

>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> firefox-dev mailing list
>>> >>> firef...@mozilla.org
>>> >>> https://mail.mozilla.org/listinfo/firefox-dev
>>> >>>
>>> >>>
>>> >>
>>> _______________________________________________
>>> dev-platform mailing list
>>> dev-pl...@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-platform
>>>
>>
>>
>
>

> _______________________________________________
> firefox-dev mailing listfirefox-dev@mozilla.orghttps://mail.mozilla.org/listinfo/firefox-dev
>
>
>

[Boris Zbarsky]

On 7/8/16 4:18 PM, Steve Fink wrote:
> Are either mxr or dxr really the right thing for canonical links to our
> source code? As long as we're updating links, through one means or
> another, temporarily or permanently, couldn't we come up with a set of
> urls that would be better in the long term? One that clearly states the
> purpose of a link (eg a link to the latest version of a whole file vs a
> link to a range of lines in a specific version vs a link to a named
> function definition or whatever), and could be redirected when the next
> great thing comes along?

Yes, please!

Especially because DXR doesn't allow independent specification of "lines
to highlight" and "line to scroll to", so any conversion from mxr links
to dxr ones will be somewhat lossy...

There's still the question of what to back the links with of course
(that is, what runs on the server).

> hg.mozilla.org would seem to be better for versioned links

Can't highlight lines there, last I checked.

-Boris

[Tanvi Vyas]

On 7/8/16 12:49 PM, Gijs Kruitbosch wrote:
> In case this is useful for folks: I wrote a webextension that rewrites
> these links and that obeys the "rev" and "mark" query params from MXR
> links and rewrites them to the equivalent DXR URL syntax:
> https://addons.mozilla.org/en-US/firefox/addon/mxr-to-dxr-webextension/ .
>
> ~ Gijs

Thank you Gijs!

>
> On 08/07/2016 20:27, Lawrence Mandel wrote:
>> We do in the case of 3rd party software referencing files from MXR
>> (and I'm told there is a lot of this). We also can't guarantee that
>> MXR URLs will direct to the right place in DXR. There is likely a
>> balance to be struck here with highly referenced files from 3rd party
>> software getting an interstitial page and other files not getting the
>> page. Let's start with getting the page with the redirect in place
>> and then iterate from there as required.
>>
>> Lawrence
>>
>> On Fri, Jul 8, 2016 at 3:24 PM, Bobby Holley <bobby...@gmail.com>

>> wrote:
>>
>> Can we skip the interstitial page and make the notice (if any)
>> more unobtrusive somehow? There are tons of mxr links all over
>> the place, and many of them are immutable. We don't gain anything
>> by informing the viewer about their obsolescence instead of
>> showing them the content they want.
>>
>> On Fri, Jul 8, 2016 at 12:20 PM, Lawrence Mandel
>> <lma...@mozilla.com> wrote:
>>
>> dev-platform was not included on my response below. Looping
>> back in to this
>> fork of the thread.
>>
>> On Fri, Jul 8, 2016 at 10:55 AM, Lawrence Mandel

>> <lma...@mozilla.com <mailto:lma...@mozilla.com>>

>> wrote:
>>
>> > Sorry Dao. I have seen some responses. Maybe they were off
>> list. We're
>> > working on details now. I'm going to get someone to put the
>> redirects in
>> > place, likely with an interstitial page advising that MXR
>> has been
>> > decommissioned, by next week.
>> >
>> > Lawrence
>> >
>> >
>> > On Friday, 8 July 2016, Dão Gottwald <dgot...@mozilla.com

>> <mailto:dgot...@mozilla.com>> wrote:
>> >
>> >> Why has nobody responded to the requests for a short-term
>> fix for MXR
>> >> URLs for more than a week? Are the people responsible for
>> MXR not in this
>> >> list?
>> >>
>> >> 2016-07-07 18:23 GMT+02:00 Eric Shepherd

>> <eshe...@mozilla.com <mailto:eshe...@mozilla.com>>:

>> >>
>> >>> We have tons of mxr links all through MDN, fwiw. I am
>> updating the
>> >>> macros that generate them, but odds are very good these
>> links will be
>> >>> around for a good while.
>> >>>
>> >>>
>> >>> That would be perfectly fine for my purposes, I expect,
>> as long as it
>> >>> dealt with the relevant mxr features. What I want is for
>> links to possibly
>> >>> specific lines of possibly specific revisions of specific
>> files to work.
>> >>> Ideally with the highlighting bits too.
>> >>>
>> >>>
>> >>> --
>> >>>
>> >>> Eric Shepherd
>> >>> Senior Technical Writer
>> >>> Mozilla Developer Network <https://developer.mozilla.org/>

>> >>> Blog: https://www.bitstampede.com/
>> >>> Twitter: http://twitter.com/sheppy
>> >>> Doodle: http://doodle.com/the.sheppy
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> firefox-dev mailing list
>> >>> firef...@mozilla.org <mailto:firef...@mozilla.org>

>> >>> https://mail.mozilla.org/listinfo/firefox-dev
>> >>>
>> >>>
>> >>
>> _______________________________________________
>> dev-platform mailing list
>> dev-pl...@lists.mozilla.org

>> <mailto:dev-pl...@lists.mozilla.org>
>> https://lists.mozilla.org/listinfo/dev-platform

[Gerald Squelart]

The annotate (aka blame) functionality of hg.mozilla.org can point at one line, e.g.:
https://hg.mozilla.org/mozilla-central/annotate/c2da34d96746288b5fee27bf6542a12c9f410988/dom/media/platforms/PDMFactory.cpp#l126
(Hash, lowercase 'L', line number)
I've opened https://bugzil.la/1282329 requesting that DXR generates these as recommended permalinks -- Hope you like it. :-)

[Boris Zbarsky]

On 7/8/16 7:25 PM, Gerald Squelart wrote:
> The annotate (aka blame) functionality of hg.mozilla.org can point at one line

Yes, I know. What it can't do is highlight some set of lines containing
more than one line. Think things like:

http://mxr.mozilla.org/whatever-file?mark=10-20,17,25#8

which highlights lines 10-20, 17, and 25, and scrolls to line 8. This
is very useful when pointing people at code.

-Boris

[Eric Rahm]

While mxr being retired is unfortunate, at this point would it not make
more sense to just file bugs against dxr for features you would like to be
added?

It's not clear to me after this lengthy discussion if anything actionable
has been accomplished.

-e

[Boris Zbarsky]

On 7/8/16 11:37 PM, Eric Rahm wrote:
> While mxr being retired is unfortunate, at this point would it not make
> more sense to just file bugs against dxr for features you would like to
> be added?

1) We're talking about hg.mozilla.org, not dxr. There are existing bug
reports on adding line highlighting to hgweb, I believe; they've been
around for a while now.

2) When I last suggested (back when dxr usage was much lower than now)
that dxr should stop using the fragment identifier for the set of lines
to highlight, so you could combine highlighting with selection of a line
to scroll to, I was flat out told that wasn't going to happen because
dxr wanted to use the fragment identifier for highlighting. Of course
at this point changing the behavior would also break existing dxr links,
even if there has been a change of heart...

> It's not clear to me after this lengthy discussion if anything
> actionable has been accomplished.

You're right, nothing has.

-Boris

[Gerald Squelart]

Ooh, didn't know about that one, nifty!
It would be great to get it into hgweb (as you mention in your reply to Eric).

g.

[Axel Grude]

That will be really useful, thanks!

All I need now is an open search engine for c-c to plug into Firefox search
box. ;)

Axel

On 11 Jul 2016 03:09, "Peter Elmers" <pel...@mozilla.com> wrote:

> If it helps anyone, I happen to know that there exists an addon which
> visits all anchor tags on a page and rewrites mxr.mozilla.org to
> dxr.mozilla.org.
> https://addons.mozilla.org/en-US/firefox/addon/mxr-to-dxr/
>
> Disclaimer: I'm the author.
>

>> _______________________________________________
>> dev-platform mailing list
>> dev-pl...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>>
>
>

[Matthew N.]

On Fri, Jul 8, 2016 at 12:27 PM, Lawrence Mandel <lma...@mozilla.com>
wrote:

> We do in the case of 3rd party software referencing files from MXR (and
> I'm told there is a lot of this).
>

​That's an existing problem which is orthogonal to the MXR decommissioning
so that doesn't need to be solved now or block better solutions to the
problem at hand.

IMO we should redirect mxr.mozilla.org + lxr.mozilla.org to ​dxr.mozilla.org's
server and have some rewrite rules to fix the cases where the URL needs to
change to retrieve the equivalent content.

​Rewriting inbound links is addressing the problem at the wrong place IMO
since there are so many links you won't be able to fix and there's not a
good reason to break them.

> We also can't guarantee that MXR URLs will direct to the right place in
> DXR. There is likely a balance to be struck here with highly referenced
> files from 3rd party software getting an interstitial page and other files
> not getting the page. Let's start with getting the page with the redirect
> in place and then iterate from there as required.
>
> Lawrence
>
>
> On Fri, Jul 8, 2016 at 3:24 PM, Bobby Holley <bobby...@gmail.com>
> wrote:
>
>> Can we skip the interstitial page and make the notice (if any) more
>> unobtrusive somehow? There are tons of mxr links all over the place, and
>> many of them are immutable. We don't gain anything by informing the viewer
>> about their obsolescence instead of showing them the content they want.
>>
>> On Fri, Jul 8, 2016 at 12:20 PM, Lawrence Mandel <lma...@mozilla.com>
>> wrote:
>>
>>> dev-platform was not included on my response below. Looping back in to
>>> this
>>> fork of the thread.
>>>
>>> On Fri, Jul 8, 2016 at 10:55 AM, Lawrence Mandel <lma...@mozilla.com>

[Eric Rahm]

Alright I filed bug 1286076 [1] for tracking the "redirect MXR" discussion and bug 1286079 [2] for the "adding multi-line highlight to hgweb" discussion. I don't think I saw anything else actionable in the thread (the text contrast issue has already been fixed), but if you have specific requests I'd suggest filing a bug [3,4] in addition to any discussion here.

-e

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1286076
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1286079
[3] https://bugzilla.mozilla.org/enter_bug.cgi?product=Webtools&component=DXR
[4] https://bugzilla.mozilla.org/enter_bug.cgi?product=Developer%20Services&component=Mercurial%3A%20hg.mozilla.org

[Erik Rose]

> All I need now is an open search engine for c-c to plug into Firefox search
> box. ;)

There's actually a bug open about this already, with some hints on how to get started: https://bugzilla.mozilla.org/show_bug.cgi?id=813521

[Jörg Knobloch]

Sorry for joining this thread after 77 previous posts, which,
admittedly, I haven't all read.

Also excuse me if I'm asking something that has been asked/answered
before. I tried looking for keywords in the messages (update, index,
refresh).

Simple question: How often is DXR refreshed from, say, mozilla-central.
In the past I had the impression that MXR was more up-to-date than DXR.
It would also be good to have the answer for comm-central.

Jörg K (Thunderbird developer).

[Erik Rose]

> Simple question: How often is DXR refreshed from, say, mozilla-central.

Every 6 hours. However, if it doesn't build successfully, it doesn't update (since DXR uses information exfiltrated from the compiler to give more accurate search results than MXR). That's what's happening now: https://jenkins-dxr.mozilla.org/job/mozilla-central/lastFailedBuild/console. I spoke with padenot yesterday, who reported landing a patch that should fix mach's installation of dependencies on Ubuntu, fixing the problem. When he shows up for the day, I'll see if I can get the commit hash and see if it's landed.

> It would also be good to have the answer for comm-central.

Also every 6 hours. comm-central isn't building due to an obsolete version of clang. fubar is upgrading it.

Cheers,
Erik

[Andreas Tolfsen]

Erik Rose <er...@mozilla.com> writes:

> > Simple question: How often is DXR refreshed from, say,
> > mozilla-central.
>
> Every 6 hours. However, if it doesn't build successfully, it doesn't
> update (since DXR uses information exfiltrated from the compiler to
> give more accurate search results than MXR).

Is this also the case for non-C++ code in the tree? One of the reasons
I don’t use dxr is because I don’t feel I can trust that the sources
are up-to-date.

For example, http://hg.mozilla.org/mozilla-central/rev/53d2d214a699 has
a push date of 2016-07-19 14:02, which is now more than 24 hours ago,
but the change is not in effect at
https://dxr.mozilla.org/mozilla-central/source/testing/marionette/error.js.

[Erik Rose]

> Is this also the case for non-C++ code in the tree?

The non-C++ code in a C++ project updates atomically with the C++ code in order to show a coherent view (and search DB) of the project. Projects that have no build step, like webtools, obviously don't run into such build issues.

[Philip Chee]

I ran into this trying to figure out why comm-central was failing to
build in Bug 1296664. A DXR search for SetHostAndPort turned up nil nada
zilch hits. I had to dig through my local mozilla-central clone to find
out what needed to be done to fix comm-central.

[whinge]I never had that problem with MXR[/whinge]