Read-only schema registry

[Dmitry Dzema]

Hi,

At my company we use jenkins to run automated tests and package software. During the test phase we would like to run a check that avro event schemas packaged with application and used to send data to Kafka are registered in the registry. For security reasons though we do not have access to production systems from our jenkins and can not execute API calls to registry directly.

One of possible solutions I came up with is to run Kafka on jenkins and use mirror maker to replicate `_schemas` topic into Kafka running on jenkins and then run schema registry on jenkins backed by replicated topic. This should give us the same state as on production.  But it will also allow somebody to register schemas using registry running in jenkins which will cause troubles.

A solution to avoid this will be an ability to run schema registry in read-only mode allowing only GET requests and forbidding any requests performing changes in underlying storage.

Would that be something which could be interested to a wider community and could be adopted into platform if I do all code changes since I don't think we will have resources to maintain a fork of schema registry for our own purposes?

Or maybe somebody faced the same problem and came up with a different solution?

Cheers,

Dmitry.

[Erik Forsberg]

On Wednesday, March 30, 2016 at 9:38:34 PM UTC+2, Dmitry Dzema wrote:

Would that be something which could be interested to a wider community and could be adopted into platform if I do all code changes since I don't think we will have resources to maintain a fork of schema registry for our own purposes?

The ability to have a read-only schema registry is definitely of interest. 

Or maybe somebody faced the same problem and came up with a different solution?

Two ideas out of my head:

1. A short term solution could be that you put a schema registry instance just for jenkins behind a proxy, i.e. nginx or apache, and limit the request methods allowed in the proxy to only allow GET requests.

2. When schema registry gets support for Kafka security, it might become possible to setup a schema registry that is prohibited by Kafka access control do do any changes to the topic. But now I'm just speculating :-)

That would be my two cents.

Regards,

\EF

[gerard...@dizzit.com]

You can use the option master.eligibility=false to prevent a certain schema registry from writing. It will still forward request to a master schema registry, if one is registered in zookeeper.

In our setup we don't want clients to be able to update the schema's. So we have a couple of schema registry's which have master.eligibility=false, and one schema registry with master.eligibility=true. We run this master in a docker, and don't have it's port mapped, so the slaves will not be able to send the request for an update to the master. This will cause an error, which we catch in 'our' client library. Updates on schema's are done using a docker container.