Advice on zipkin storage

[Adrian Cole]

We've had a couple ask about larger scale usage of mysql, and this made me realize that we've never formally captured advice on storage options in Zipkin.

What zipkin supports is very much a function of time, energy and expertise. As zipkin is run mostly on volunteer time, certain storage options absolutely get more support than others. Here's a convenience paste of zipkin's README on the topic of storage:

===

Zipkin includes a StorageComponent, used to store and query spans and dependency links. This is used by the server and those making custom servers, collectors, or span reporters. For this reason, storage components have minimal dependencies; many run on Java 7.

Ex.

// this won't create network connections
storage = CassandraStorage.builder()
                          .contactPoints("my-cassandra-host").build();

// but this will
trace = storage.spanStore().getTrace(traceId);

// clean up any sessions, etc
storage.close();

In-Memory

The InMemoryStorage component is packaged in zipkin's core library. It is not persistent, nor viable for realistic work loads. Its purpose is for testing, for example starting a server on your laptop without any database needed.

MySQL

The JDBCStorage component currently is only tested with MySQL 5.6-7. It is designed to be easy to understand, and get started with. However, it has known performance issues at larger scales. For example, queries will eventually take seconds to return if you put a lot of data into it.

Cassandra

The CassandraStorage component is tested against Cassandra 2.2+. It is designed for larger scales of data. For example, it has manually implemented indexes to make querying larger data more performant.

Elasticsearch

The ElasticsearchStorage component is tested against Elasticsearch 2.3. It is designed for larger scales of data, and works on json directly. The Elasticsearch component is the newest option.