Master thesis about Talos noise finished

Showing 1-3 of 3 messages
Master thesis about Talos noise finished Jan Larres 11/21/11 6:15 PM
Hi all,

I recently finished my Master's thesis about noise/variance in the Talos
tests. Some of you may remember that I gave a talk about my work at the
All Hands in September, so here now is the finished write-up:

http://majutsushi.net/stuff/thesis.pdf

Short summary of the important points:

  - Memory randomization seems to have a significant impact on the
    variance, other "external" factors like concurrently running,
    unnecessary programs (like a full desktop environment) are also
    measurable but not quite as pronounced.
  - There is some variance in the number and order of events that are
    used during the tests, but they don't seem to correlate with the
    test results.
  - I also have a suggestion for a statistical technique that should
    give more reliable outlier detection than the t-test that is
    currently being used by the graph server.

I hope it's somewhat useful. If there's anything unclear feel free to
ask!

Cheers,
Jan
Re: Master thesis about Talos noise finished Chris AtLee 11/30/11 11:50 AM
Hi Jan,

Very interesting, thank you for sharing this.

I wrote the current system for detecting outliers, and am very eager to
give your suggestions a try.

Cheers,
Chris
Re: Master thesis about Talos noise finished Johnathan Nightingale 12/1/11 11:11 AM
Yeah, this work is promising - filed https://bugzilla.mozilla.org/show_bug.cgi?id=706912

J
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning

---
Johnathan Nightingale
Director of Firefox Engineering
joh...@mozilla.com