Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

TOC Decision Time

9 views
Skip to first unread message

David Walsh

unread,
Dec 6, 2013, 1:16:36 PM12/6/13
to Ali Spivak, Eric Shepherd, Luke Crouch, Holly Habstritt, John Karahalis, Maris Fogels, mdn-drivers
Hello All,

One major point of contention has been the table of contents and how we
handle "deep" nesting. At present, the TOC's counter can overlap TOC
anchor links and we all agree that's ugly. What we haven't agreed on is
how to fix it. Before I do any work on it, what should we do? Keep the
numbers and indent? Remove the numbers and indent? Use bullets?

Let's figure this out quickly. Thank you!

David

Will Bamberg

unread,
Dec 6, 2013, 1:15:25 PM12/6/13
to David Walsh, Ali Spivak, Eric Shepherd, Luke Crouch, Holly Habstritt, John Karahalis, Maris Fogels, mdn-drivers

Remove the numbers and indent, please!
> _______________________________________________
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/mdn-drivers

Luke Crouch

unread,
Dec 6, 2013, 5:08:40 PM12/6/13
to Will Bamberg, David Walsh, Ali Spivak, Eric Shepherd, Holly Habstritt, John Karahalis, Maris Fogels, mdn-drivers
I hear many calls to remove them, but if someone can give us a
hypothesis to test an expected value of keeping them, I'd be okay with
having an A/B test in-place at the time of launch and then quickly make
the call post-launch.

If no-one cares enough to come up with an hypothesis, test, and value to
measure we can go by simple consensus or majority here.

-L

--
Q: Why is this email five sentences or less?
A: http://five.sentenc.es

Daniel Buchner

unread,
Dec 6, 2013, 5:11:37 PM12/6/13
to Luke Crouch, David Walsh, Eric Shepherd, Maris Fogels, Will Bamberg, mdn-drivers, John Karahalis, Holly Habstritt, Ali Spivak
+1 for "remove the numbers and indent"

John Karahalis

unread,
Dec 6, 2013, 5:16:39 PM12/6/13
to Luke Crouch, David Walsh, Eric Shepherd, Maris Fogels, Will Bamberg, mdn-drivers, Holly Habstritt, Ali Spivak
+1 for evidence-based software engineering!

The hypothesis here would need to be "indented tables of content are more understandable". To be honest, I'm not sure an A/B test would be the best way of evaluating that. Qualitative evidence would be better, and in my opinion, we already have that in the number of bugs users have submitted supporting that position.

--
John Karahalis
Mozilla
openjck.com

----- Original Message -----

Will Bamberg

unread,
Dec 6, 2013, 5:17:03 PM12/6/13
to Luke Crouch, David Walsh, Ali Spivak, Eric Shepherd, Holly Habstritt, John Karahalis, Maris Fogels, mdn-drivers
I guess the assertion is that an indented ToC is more scannable, so it's
easier to find the element you're looking for. I'm not sure how to build
an A/B test for that. Maybe just, how often people click links in the
ToC, since they find the link they are looking for there rather than
scrolling through the page or giving up? How fast people click on the
ToC after loading the page?

Will

Eric Shepherd

unread,
Dec 6, 2013, 5:21:38 PM12/6/13
to Luke Crouch, mdn-drivers, David Walsh, Maris Fogels, Ali Spivak, John Karahalis, Eric Shepherd, Will Bamberg, Holly Habstritt
Sometimes it feels like people propose A/B testing just to do A/B testing. Given that indenting tables of contents is an incredibly common practice, it's not exactly like we're pulling this idea out of a dark hole. :)

-- 
Eric Shepherd
Developer Documentation Lead
Mozilla
Blog: http://www.bitstampede.com/
Twitter: http://twitter.com/sheppy

David Walsh

unread,
Dec 6, 2013, 5:26:55 PM12/6/13
to Will Bamberg, Luke Crouch, Ali Spivak, Eric Shepherd, Holly Habstritt, John Karahalis, Maris Fogels, mdn-drivers
I think A/B testing will be a waste of time since everyone hates the
current implementation and the use of numbers.

David


On 12/6/13, 4:17 PM, Will Bamberg wrote:
> I guess the assertion is that an indented ToC is more scannable, so
> it's easier to find the element you're looking for. I'm not sure how
> to build an A/B test for that. Maybe just, how often people click
> links in the ToC, since they find the link they are looking for there
> rather than scrolling through the page or giving up? How fast people
> click on the ToC after loading the page?
>

John Karahalis

unread,
Dec 6, 2013, 5:23:35 PM12/6/13
to Eric Shepherd, David Walsh, Eric Shepherd, mdn-drivers, Will Bamberg, Luke Crouch, Ali Spivak, Holly Habstritt, Maris Fogels
http://www.nizkor.org/features/fallacies/appeal-to-tradition.html ;-)

--
John Karahalis
Mozilla
openjck.com

----- Original Message -----

> From: "Eric Shepherd" <eshe...@mozilla.com>
> To: "Luke Crouch" <lcr...@mozilla.com>, "mdn-drivers"
> <mdn-d...@lists.mozilla.org>, "David Walsh" <dwa...@mozilla.com>, "Maris
> Fogels" <mfo...@mozilla.com>, "Ali Spivak" <asp...@mozilla.com>, "John
> Karahalis" <jkara...@mozilla.com>, "Eric Shepherd" <she...@mozilla.com>,
> "Will Bamberg" <wbam...@mozilla.com>, "Holly Habstritt"
> <hhabs...@mozilla.com>
> Sent: Friday, December 6, 2013 5:21:38 PM
> Subject: Re: TOC Decision Time

> Sometimes it feels like people propose A/B testing just to do A/B testing.
> Given that indenting tables of contents is an incredibly common practice,
> it's not exactly like we're pulling this idea out of a dark hole. :)

> --
> Eric Shepherd
> Developer Documentation Lead
> Mozilla
> Blog: http://www.bitstampede.com/
> Twitter: http://twitter.com/sheppy

> On December 6, 2013 at 5:17:04 PM, Will Bamberg ( wbam...@mozilla.com )

Eric Shepherd

unread,
Dec 6, 2013, 5:25:39 PM12/6/13
to Luke Crouch, mdn-drivers, David Walsh, Maris Fogels, Ali Spivak, John Karahalis, Eric Shepherd, Will Bamberg, Holly Habstritt
This. No sense A/B testing when one of the choices is something that absolutely everyone hates.  If you want to A/B test, you need to pick two totally different designs from the current one first.

-- 
Eric Shepherd
Developer Documentation Lead
Mozilla
Blog: http://www.bitstampede.com/
Twitter: http://twitter.com/sheppy

Daniel Buchner

unread,
Dec 6, 2013, 5:31:27 PM12/6/13
to Will Bamberg, David Walsh, Eric Shepherd, Maris Fogels, mdn-drivers, John Karahalis, Holly Habstritt, Ali Spivak, Luke Crouch
To really instrument and test this, we'd need a few measures (IMO) - you
make the call if it is worth it:

1. Setup an A/B test with and without the TOCs
2. Track clicks of TOC links
3. Track clicks of sub-links within TOC'd content sections
4. Track time-on-page before navigation to a sub-link within a section

There's two things to assess once you have the data:

1. Is the TOC even engaged by users that often? If not, the point is
moot and it should go.
2. If the TOC is engaged often, you can then compare the time it takes
for a user to reach a content block and navigate to a new sub-linked
document without the TOC (using scroll and scan), to the time it takes for
users who engage the TOC to navigate to the same sub-linked document. If
the time it takes a user to complete the goal step is reduced noticeably in
the presence of a TOC, there's your user benefit - weigh that in your
decision accordingly.

Thoughts?
- Daniel

On Fri, Dec 6, 2013 at 2:17 PM, Will Bamberg <wbam...@mozilla.com> wrote:

> I guess the assertion is that an indented ToC is more scannable, so it's
> easier to find the element you're looking for. I'm not sure how to build an
> A/B test for that. Maybe just, how often people click links in the ToC,
> since they find the link they are looking for there rather than scrolling
> through the page or giving up? How fast people click on the ToC after
> loading the page?
>
> Will
>
>
>
> On 2013-12-06 2:08 PM, Luke Crouch wrote:
>
>> I hear many calls to remove them, but if someone can give us a hypothesis
>> to test an expected value of keeping them, I'd be okay with having an A/B
>> test in-place at the time of launch and then quickly make the call
>> post-launch.
>>
>> If no-one cares enough to come up with an hypothesis, test, and value to
>> measure we can go by simple consensus or majority here.
>>
>> -L
>>
>>

David Walsh

unread,
Dec 6, 2013, 5:43:23 PM12/6/13
to Daniel Buchner, Will Bamberg, Eric Shepherd, Maris Fogels, mdn-drivers, John Karahalis, Holly Habstritt, Ali Spivak, Luke Crouch
Still pointless IMO. We have the technical issue of number and text
overlap. And as has been said, those numbers don't appear in the page
so they (1) are causing a layout issue and (2) don't have value in the
document.

Unless we want to *add* those numbers in the document...

David
> Mdn-d...@lists.mozilla.org <mailto:Mdn-d...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/mdn-drivers
>
>

Eric Shepherd

unread,
Dec 6, 2013, 5:40:47 PM12/6/13
to Daniel Buchner, Will Bamberg, David Walsh, Eric Shepherd, Maris Fogels, Ali Spivak, John Karahalis, Luke Crouch, mdn-drivers, Holly Habstritt
On December 6, 2013 at 5:39:40 PM, David Walsh (dwa...@mozilla.com) wrote:
Still pointless IMO.  We have the technical issue of number and text overlap.  And as has been said, those numbers don't appear in the page so they (1) are causing a layout issue and (2) don't have value in the document.
Right.

Unless we want to *add* those numbers in the document...
I don't... haven't heard anyone else say they want them, but they might be out there, I suppose.

Jeremie Patonnier

unread,
Dec 6, 2013, 5:51:04 PM12/6/13
to Eric Shepherd, David Walsh, Eric Shepherd, Maris Fogels, Daniel Buchner, mdn-drivers, John Karahalis, Luke Crouch, Ali Spivak, Holly Habstritt, Will Bamberg
Who's decision maker here?
Ali? Luke? Someone else?

That discussion is pointless. Everyone here has made his mind clear now or
in a previous thread/bug discussion. Can we go to a decision now. What we
seriously need now is identifying who take the decision to go one way or
another.

Best
Jérémie


2013/12/6 Eric Shepherd <eshe...@mozilla.com>
> _______________________________________________
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/mdn-drivers
>



--
Jeremie
.............................
Web : http://jeremie.patonnier.net
Twitter : @JeremiePat <http://twitter.com/JeremiePat>

David Walsh

unread,
Dec 6, 2013, 6:00:37 PM12/6/13
to Jeremie Patonnier, Eric Shepherd, Eric Shepherd, Maris Fogels, Daniel Buchner, mdn-drivers, John Karahalis, Luke Crouch, Ali Spivak, Holly Habstritt, Will Bamberg
Everyone has said to (1) get rid of numbers and (2) indent. Let's roll
with that.

David

On 12/6/13, 4:51 PM, Jeremie Patonnier wrote:
> Who's decision maker here?
> Ali? Luke? Someone else?
>
> That discussion is pointless. Everyone here has made his mind clear
> now or in a previous thread/bug discussion. Can we go to a decision
> now. What we seriously need now is identifying who take the decision
> to go one way or another.
>
> Best
> J�r�mie
>
>
> 2013/12/6 Eric Shepherd <eshe...@mozilla.com
> <mailto:eshe...@mozilla.com>>
>
> On December 6, 2013 at 5:39:40 PM, David Walsh (dwa...@mozilla.com
> <mailto:dwa...@mozilla.com>) wrote:
> Still pointless IMO. We have the technical issue of number and
> text overlap. And as has been said, those numbers don't appear in
> the page so they (1) are causing a layout issue and (2) don't have
> value in the document.
> Right.
>
> Unless we want to *add* those numbers in the document...
> I don't... haven't heard anyone else say they want them, but they
> might be out there, I suppose.
>
> --
> Eric Shepherd
> Developer Documentation Lead
> Mozilla
> Blog: http://www.bitstampede.com/
> Twitter: http://twitter.com/sheppy
> _______________________________________________
> Mdn-drivers mailing list
> Mdn-d...@lists.mozilla.org <mailto:Mdn-d...@lists.mozilla.org>
> https://lists.mozilla.org/listinfo/mdn-drivers
>
>
>
>
> --
> Jeremie
> .............................
> Web : http://jeremie.patonnier.net <http://jeremie.patonnier.net/>
> Twitter : @JeremiePat <http://twitter.com/JeremiePat>

John Karahalis

unread,
Dec 6, 2013, 6:10:14 PM12/6/13
to David Walsh, Jeremie Patonnier, Eric Shepherd, Maris Fogels, Daniel Buchner, mdn-drivers, Holly Habstritt, Eric Shepherd, Ali Spivak, Luke Crouch, Will Bamberg
In my opinion, evidence rather than people should drive the decisions we make. As I mention in an earlier message, I believe we already have that evidence: numerous bugs from real users all reporting the same confusion with the design.

--
John Karahalis
Mozilla
openjck.com

----- Original Message -----

> From: "David Walsh" <dwa...@mozilla.com>
> To: "Jeremie Patonnier" <jeremie....@gmail.com>, "Eric Shepherd"
> <eshe...@mozilla.com>
> Cc: "Daniel Buchner" <dan...@mozilla.com>, "Will Bamberg"
> <wbam...@mozilla.com>, "Eric Shepherd" <she...@mozilla.com>, "Maris Fogels"
> <mfo...@mozilla.com>, "Ali Spivak" <asp...@mozilla.com>, "John Karahalis"
> <jkara...@mozilla.com>, "Luke Crouch" <lcr...@mozilla.com>, "mdn-drivers"
> <mdn-d...@lists.mozilla.org>, "Holly Habstritt" <hhabs...@mozilla.com>
> Sent: Friday, December 6, 2013 6:00:37 PM
> Subject: Re: TOC Decision Time

> Everyone has said to (1) get rid of numbers and (2) indent. Let's roll with
> that.

> David

> On 12/6/13, 4:51 PM, Jeremie Patonnier wrote:

> > Who's decision maker here?
>
> > Ali? Luke? Someone else?
>

> > That discussion is pointless. Everyone here has made his mind clear now or
> > in
> > a previous thread/bug discussion. Can we go to a decision now. What we
> > seriously need now is identifying who take the decision to go one way or
> > another.
>

> > Best
>
> > Jérémie
>

> > 2013/12/6 Eric Shepherd < eshe...@mozilla.com >
>

> > > On December 6, 2013 at 5:39:40 PM, David Walsh ( dwa...@mozilla.com )
> > > wrote:
> >
>
> > > Still pointless IMO. We have the technical issue of number and text
> > > overlap.
> > > And as has been said, those numbers don't appear in the page so they (1)
> > > are
> > > causing a layout issue and (2) don't have value in the document.
> >
>
> > > Right.
> >
>

> > > Unless we want to *add* those numbers in the document...
> >
>
> > > I don't... haven't heard anyone else say they want them, but they might
> > > be
> > > out there, I suppose.
> >
>

> > > --
> >
>
> > > Eric Shepherd
> >
>
> > > Developer Documentation Lead
> >
>
> > > Mozilla
> >
>
> > > Blog: http://www.bitstampede.com/
> >
>
> > > Twitter: http://twitter.com/sheppy
> >
>
> > > _______________________________________________
> >
>
> > > Mdn-drivers mailing list
> >
>
> > > Mdn-d...@lists.mozilla.org
> >
>
> > > https://lists.mozilla.org/listinfo/mdn-drivers
> >
>

> > --
>
> > Jeremie
>
> > .............................
>
> > Web : http://jeremie.patonnier.net
>
> > Twitter : @ JeremiePat
>

Janet Swisher

unread,
Dec 6, 2013, 7:15:37 PM12/6/13
to mdn-d...@lists.mozilla.org

On 12/6/13 5:10 PM, John Karahalis wrote:
> In my opinion, evidence rather than people should drive the decisions we make. As I mention in an earlier message, I believe we already have that evidence: numerous bugs from real users all reporting the same confusion with the design.
I agree that numerous bugs make a strong argument.

But I disagree that you shouldn't listen to people with expertise, like
professional technical communicators with years of experience in online
document design. It can save you a lot of time and effort reinventing
wheels. In cases where there isn't precedent (nobody's done this before)
or consensus (experts disagree, or users overwhelmingly disagree with
experts), then go look for more evidence, such as through testing.

--
Janet Swisher <mailto:jREMOVE...@mozilla.com>
Mozilla Developer Network <https://developer.mozilla.org>
Developer Engagement Community Organizer

John Karahalis

unread,
Dec 7, 2013, 1:20:42 PM12/7/13
to Janet Swisher, mdn-d...@lists.mozilla.org
I agree, we should also listen to the experts. Bug count is certainly not the only evidence worth considering.

--
John Karahalis
Mozilla
openjck.com

----- Original Message -----

> From: "Janet Swisher" <jswi...@mozilla.com>
> To: mdn-d...@lists.mozilla.org
> Sent: Friday, December 6, 2013 7:15:37 PM
> Subject: Re: TOC Decision Time

Luke Crouch

unread,
Dec 9, 2013, 10:55:37 AM12/9/13
to John Karahalis, Janet Swisher, mdn-d...@lists.mozilla.org
Not the only evidence, but it's the best available evidence we have
right now. This thread has some great discussion and generated great
ideas for experiments we could run to re-assess this later if we want to.

So, I'll make the call - remove the numbers and indent.

I've updated the bug [1] with a comment pointing back to this thread in
case we ever re-open we can get back here.

Thanks everyone.
-L

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=944560#c7

Ali Spivak

unread,
Dec 9, 2013, 11:09:43 AM12/9/13
to Luke Crouch, mdn-d...@lists.mozilla.org, Janet Swisher, John Karahalis
I also agree - remove the numbers and indent.




ali spivak
408-859-8260
asp...@mozilla.com

----- Original Message -----
From: "Luke Crouch" <lcr...@mozilla.com>
To: "John Karahalis" <jkara...@mozilla.com>, "Janet Swisher" <jswi...@mozilla.com>
Cc: mdn-d...@lists.mozilla.org
Sent: Monday, December 9, 2013 7:55:37 AM
Subject: Re: TOC Decision Time

0 new messages