Proposed Hyperspace stress/performance test

[Sanjit Jhala]

I'm thinking of writing a stress/performance test for hyperspace so we can keep track of how various recent and upcoming changes affect the performance of the system. I'm thinking of a couple of different workloads:
1. R/W load: consisting of checking file existence, creation, reading and writing attributes
2. Comprehensive load: more varied load comprising of locks, releases, deletes along with R/W load.

For the R/W load test, a bunch of clients will run the test in parallel simultaneously.
Each test client has a bunch of configurable parameters:
-Ratio of reads/writes (r)
-Total #files (N)
-#files opened by a client (k)
-Size of individual write
-Total #bytes to be read + written (T)
-random seed

The filesystem hierarchy is a complete binary tree of N nodes. Each test client randomly picks k out of these N nodes to operate on and sorts these k nodes topologically. It then proceeds through the list of nodes (in sorted order) doing the following till it has executed T bytes of reads/writes (wrapping around the list of nodes if required):
1. Check if file exists (if not open) and open if it does. If not then go up the hierarchy till you hit a file that exists and create the appropriate child node for it.
2. Read/Write to the node.
The purpose of going through the nodes in order is to stagger the node creation requests (which would be overwhelmingly at the beginning in case we went through the nodes in random order).

By running for different  N, k  and r it should be possible to simulate varied workloads. For example, suppose you have 30 test clients, a total of 50 files in the directory tree and each client operates on 40 random files - this should simulate a high contention workload.

It should be possible to extend this idea to the comprehensive workload test with some tweaks (% breakdown of each request type etc).

-Sanjit

[Luke]

Simulating thousands for servers starting and trying to establish a
session would be a good test. In normal operation mode, hyperspace
shouldn't be hit too hard.