Re: Simple Tool for measuring page load times over simulated networks
Johnathan Nightingale
I haven't gotten a chance to set this up and use it yet, but if it
behaves as promised, it looks like a helpful tool, and I wanted to
thank you for putting it together.
Cheers,
Johnathan
On 11-Jun-08, at 3:58 PM, Patrick McManus wrote:
> Hello All,
>
> As I mentioned yesterday on the list, I have put together a standalone
> linux based test driver that sets up an shaped network environment and
> uses the -tp option of fennec to gather some page load times over it
> so
> we can experiment with the effect of different network environments.
>
> The network configurations and test loads are separately
> configurable. I
> have a simple load and a couple example configs available as part of
> this announcement, but it will be a (near) future step to identify
> realistic configs for them. The tool itself is ready for its first
> "release early, release often" spin - so I am sharing that code with
> this email and I am soliciting feedback on network profiles or data
> sets.
>
> For the moment I have it living in my hg under
> testing/performance/moz-shaper available at:
>
> hg pull static-http://www.ducksong.com/hg/mcmanus-mozilla-central/
>
> I also have it in a convenient bundle here:
>
> http://www.ducksong.com/hg/bundles/moz-shaper.r2
>
> I hope this format is ok - I must be the only person that finds hg
> more
> mysterious than git.
>
> I encourage anybody interested to take a look, try it out and provide
> feedback. I will inline the README here.
>
> ::::
>
> This is a test driver for fennec that sets up a simulated network to
> measure the effects of different kinds of networks on firefox mobile.
>
> The driver takes two inputs: The network configuration, and a manifest
> of pages to test.
>
> It can work either over localhost, or across a real interface. Right
> now it is dependent on a Linux environment to setup the network
> emulation.
>
> The sample-config directory contains two sample XML files that
> represent different networks. The XML defines the interface, latency,
> jitter, drop rate, and bandwidth of the network to be emulated.
>
> The sample-manifest directory contains sample workloads, which are
> just pages listed as one URL per line. The test driver automatically
> sets up a web server on localhost port 8912 which has a documentroot
> of src/testdata - the manifest can reference URLs there for a self
> contained convenience test.
>
> Future versions of the test will separate the network emulation
> portion from the client driver. That will allow the two parts to be
> run on different platforms - enabling testing of the client under OSes
> other than Linux, and it will also allow the network emulation portion
> to act as a network bridge where it will be able to give more precise
> emulation than a normal desktop will allow.
>
> Dependencies:
>
> /sbin/modprobe
> /sbin/tc
> /sbin/ip
>
> standard 2.6 kernel modules (or linked in): ifb and sch_netem
>
> These are found on most Linux distributions, but if you don't have tc
> or ip, check for the iproute or iproute2 package.
>
> Altering the network configuration requires root permissions and the
> test driver will exit if it is not run with that security (ahem)
> level.
> The test seeks to return the network to a normal state when exiting.
>
> The easiest way to run it is
> cd src
> sudo python moz-shaper.py ../sample-config/ref.xml [manifest ]
>
>
> _______________________________________________
> dev-platforms-mobile mailing list
> dev-platfo...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platforms-mobile
---
Johnathan Nightingale
Human Shield
joh...@mozilla.com
Patrick McManus
First - a hearty congratulations to all on the general release of FF3.
Way hip.
Second - I have a couple major updates to the tool I wrote to you about
a week ago (quoted below), and one question I am seeking input on.
The updates:
- The XML network profiles can now specify separate upstream and
downstream bandwidths and moz-shaper breaks the shaping queues into two
separate queues even if they are defined symmetrically. Mobile networks
are often highly asymmetrical. This has minimal impact for most HTTP
GETs, but could be very interesting for some larger interactive
sequences. And more importantly it separates the http response data
packets and the mobile ack packets into different queues which gives a
more accurate model under congestion even if the profile is symmetric.
- I've done some literature research and based on that in the
sample-config directory I have formed 7 "real-life" profiles for typical
performance on various wireless networks. Obviously real conditions do
vary, but these are meant to be pretty typical of what you would see. I
have also included one "degraded" profile to represent a wireless
network having a bad day. The profiles are for GPRS, Edge, EVDO-Rev-A,
UMTS, UMTS+HSPDA, bluetooth, and a modest 802.11g wlan.
An hg repo with this continues to be available at
static-http://www.ducksong.com/hg/mcmanus-mozilla-central/
An updated bundle is available at
http://www.ducksong.com/hg/bundles/moz-shaper.r3
My next step will be to gather and publish a bunch of data. I will start
with page load times, which is the simplest metric. I am soliciting
input for what pages folks would like to see. We get the most reliable
results by including copies of the pages in the test suite, so (c) is an
issue. I currently am working with snapshots of a tinderbox report, a
spread firefox page, and a results capture from MXR. There are largely
placeholders.
-Patrick
Daniel Brooks
> My next step will be to gather and publish a bunch of data. I will start
> with page load times, which is the simplest metric. I am soliciting
> input for what pages folks would like to see. We get the most reliable
> results by including copies of the pages in the test suite, so (c) is an
> issue. I currently am working with snapshots of a tinderbox report, a
> spread firefox page, and a results capture from MXR. There are largely
> placeholders.
>
> -Patrick
Indeed. The current talos comes with a proxy server that caches all of
the documents locally. You run the test once to create the cache, then
you can run it for real with the proxy set to only return documents from
the cache. This makes it a lot easier to set up.
db48x
