POSTing new schemas is annoying! :p

[Andrew Otto]

Ok ok, this email is going to make me sound like whiner, but maybe now is the right time to whine?

I find it fairly difficult to successfully post a new schema to the Schema Registry.  I think this might be because I have to take an already defined JSON .avsc schema, escape all double quote characters, and then include it as a string in an a JSON object like:

  '{ “schema”: “{\”namespace\”: … }” }’

Maybe I’ll just get better at doing this, but thus far I’ve had to slog through the following error messages:

  {"error_code":42201,"message":"Input schema is an invalid Avro schema”}

And if I don’t remove all newlines in the schema too:

  Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in string value

Since the only top level key in the JSON object expected by http://confluent.io/docs/current/schema-registry/docs/api.html#post--subjects-(string- subject)-versions is “schema”, perhaps we could do away with that and just allow the actual JSON avsc schema to be the full POST body? This would allow us to pass the .avsc file itself directly to curl (or whatever) to POST a new schema, like:

  curl -X POST -i -H "Content-Type: application/vnd.schemaregistry.v1+json" --data @MyAvroSchema.avsc http://localhost:8081/subjects/MyAvroSchema/versions

Eh eh?  Just a thought :)

-AO

[Ewen Cheslack-Postava]

We have this issue filed on kafka-rest: https://github.com/confluentinc/kafka-rest/issues/45 which is addressing a lot of the issues you're talking about. We don't have the equivalent on schema-registry yet, but we probably should.

One reason for wrapping the schema in a JSON envelope and not just posting the schema directly is API evolution. If we ever want to add additional parameters to the request we get stuck having to rev the API version or add another endpoint. I think the /versions endpoint should probably be left generalizable, but maybe a POST /subjects/<subject>/versions/raw or something like that could accept the raw schema for convenience.

In any event, I'm surprised this is causing so much trouble -- is there anything wrong with doing something like the following pseudocode?

payload = JSON.stringify({"schema": read_file("MyAvroSchema.avsc")})

Presumably the JSON serializer would take care of any escaping that's required.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/941B38D0-52CD-4869-919C-DD25D475A493%40wikimedia.org.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen

[Andrew Otto]

payload = JSON.stringify({"schema": read_file("MyAvroSchema.avsc")})

This totally helped, thanks for the idea.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAE1jLMM6hEWrOXzU%2BN_njtQ5hiCaprDVPQ2PN%3DNz5kG2_mkSxg%40mail.gmail.com.