Serving Protobuf

[hbf]

Dear all,

Using spray-json's DefaultJsonProtocol one can easily serve case class instances as JSON, which is great. I am wondering whether there are any way to output/serializing using Google Protocol Buffers (protobuf)?

Many thanks for any pointers!

Kaspar

[Martin Grigorov]

Hi,

Spray decides what (Un)Marshaller to use depending on the request/response content type.

I have no experience with Protobuf so I cannot tell you which content type exactly you need to use.

--
You received this message because you are subscribed to the Google Groups "spray-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spray-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Johannes Rudolph]

Hi Kaspar,

there are some old post on the mailing list on this topic. There's
currently no built-in support for protobuf but you can write custom
Marshalling/Unmarshalling infrastructure to support any binary format.
See

http://spray.io/documentation/1.1-SNAPSHOT/spray-httpx/marshalling/

Johannes

> --
> You received this message because you are subscribed to the Google Groups
> "spray-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to spray-user+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

[Age Mooij]

That is incorrect. Type class application is done at compile time and simply depends on which Unmarshaller is in implicit scope for the type you are trying to unMarshall using something like entity(as[Mytype]) or marshal using complete(instanceOfMyType)

If you want runtime content type negotiation you will need to write your own custom higher-level (un)marshallers that inspect the request for information what to produce

See Mathias's answer to a very similar question in the following discussion:

https://groups.google.com/d/topic/spray-user/xtawBl2Ytx4/discussion

To get back to the original question, (un)marshalling to/from protobuf requires an (un)marshaller that implements this to be available for your type(s) in implicit scope. I'm not aware of any pre-existing general-purpose implementation but they should not be very hard to write using something like Scalabuff (https://github.com/SandroGrzicic/ScalaBuff). 

The documentation for how to write your own (un)marshallers is here:

http://spray.io/documentation/1.1-SNAPSHOT/spray-httpx/

Hope this helps

Age

[Martin Grigorov]

Age,

On Fri, Sep 20, 2013 at 11:30 AM, Age Mooij <age....@gmail.com> wrote:

That is incorrect. Type class application is done at compile time and simply depends on which Unmarshaller is in implicit scope for the type you are trying to unMarshall using something like entity(as[Mytype]) or marshal using complete(instanceOfMyType)

If you want runtime content type negotiation you will need to write your own custom higher-level (un)marshallers that inspect the request for information what to produce

See Mathias's answer to a very similar question in the following discussion:

https://groups.google.com/d/topic/spray-user/xtawBl2Ytx4/discussion

Thanks! I'm aware of this.

But do you say that if there are more than one marchallers in the implicit scope, e.g. JSON and Protobuf and I use 

respondWithContentType(ContentType.`application-json`) {...} then Spray (or more correctly Scala) will fail the build when trying to decide which of the available Marchallers to use ?

[Age Mooij]

On Sep 20, 2013, at 11:05, Martin Grigorov <martin....@gmail.com> wrote:

> Thanks! I'm aware of this.
>
> But do you say that if there are more than one marchallers in the implicit scope, e.g. JSON and Protobuf and I use
> respondWithContentType(ContentType.`application-json`) {...} then Spray (or more correctly Scala) will fail the build when trying to decide which of the available Marchallers to use ?

Yes, that's what I'm saying. Scala's type classes depend on implicit resolution and when there are two resolutions for an implicit at the same resolution level, the compiler will fail with a message explaining which ambiguous implementations it has found.

That means that in order to decide which content type to produce in your response, you will have to provide a higher level Marshaller implicitly which wraps explicitly referenced other marshallers and makes its decision based on the request Accept header. Mathias gives a nice example of this in the post I referenced before.

Age

[hbf]

Age, Martin, Johannes, thanks for the responses and explanations – and sorry for the late reply, I have only just gotten back to this.

In the meantime, I have written the marshalling code to handle content negation, and have posted it in the thread that Age mentioned. The missing bits for the Protobuff-specific marshalling are these:

object ProtobufProtocols {

  val ProtobufMediaType = MediaTypes.register(MediaType.custom("application", "protobuf", compressible = false, binary = true))

  val Protobuf = ContentType(ProtobufMediaType)

  implicit def marshaller[T <: com.google.protobuf.MessageLite]() =

    Marshaller.of[T](Protobuf) {

      (value, contentType, ctx) ⇒

        ctx.marshalTo(HttpEntity(contentType, value.toByteArray))

    }

  implicit def unmarshaller[T <: net.sandrogrzicic.scalabuff.Message]() =

    Unmarshaller[T](ProtobufMediaType) {

      case HttpBody(contentType, buffer) =>

        // TODO

    }

}

However, I have run into a bit of a problem when trying to write some generic unmarshalling code: for each Protobuf message M that you compile to a Scala case class using ScalaBuff, there are three ways to deserialize from a buffer: either you call the method parseFrom(buffer) on M's object, you create a new instance m of M (without constructor arguments) and call m.parseFrom(buffer), or you use the defaultInstance in M's object and call parseFrom(buffer) on it.

Does anybody know how to achieve this within an Unmarshaller?

Any help most welcome!

Cheers,

Kaspar