Schema Mismatch Between Client and Server

[Matt Stern]

Hi Capnp folks,

I have a producer that writes events to a queue. The schema looks like:

struct Event {

  union {

    foo @0 : Foo;

    bar @1 : Bar;

    internal @2 : Internal;

  }

}

There are downstream consumers of this queue that are meant to ignore the "internal" field -- in fact, we don't even provide the Internal struct definition to them.

The downstream consumers have had a lot of log spam lately because they use this stripped down schema to generate code:

struct Event {

  union {

    foo @0 : Foo;

    bar @1 : Bar;

  }

}

and then handle the union with a switch/case like so:

switch (reader.which()) {

  case FOO: return handle(reader.getFoo());

  case BAR: return handle(reader.getBar());

  default: return logError("A new message type exists that we don't know about!");

}

The default case in the switch makes sense -- if a new message type appears from the producer, we should surface that in some way so the consumers know to update their schemas and handle the new message. However, we don't want to trigger this case for the known Internal type, which consumers don't need to handle.

I am thinking of giving the consumers a new schema that looks like:

struct Event {

  union {

    foo @0 : Foo;

    bar @1 : Bar;

    internal @2: Void;

  }

}

so they can explicitly ignore the internal messages but still log errors for other new messages that may appear.

My question boils down to: Is it safe for the producer to assign one type (struct Internal) to field 2 but the consumer to assign another type (Void)? Or would this cause issues?

Thanks!

[Kenton Varda]

Hi Matt,

Your idea would work initially, but if an @3 field were ever added, it could end up incompatible.

Instead, assuming `Internal` is a struct type, you can instead declare `internal` to have type `AnyPointer`:

struct Event {
  union {
    foo @0 : Foo;
    bar @1 : Bar;

    internal @2: AnyPointer;
  }
}

Since `Internal` is a struct, a field of type `Internal` is represented as a pointer. Since `AnyPointer` is also a pointer, it'll produce the same layout.

At the risk of over-engineering, another option would be to use generics:

struct Event(InternalType) {

  union {
    foo @0 : Foo;
    bar @1 : Bar;

    internal @2: InternalType;
  }
}

This way, you and your consumers can actually use exactly the same definition of `Event`. Your consumers would use `Event<>` (which is equivalent to `Event<capnp::AnyPointer>`). In your internal code with knowledge of the internal type, you'd use `Event<Internal>`.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/f94fe404-b080-4654-bfc6-c578c411cf55n%40googlegroups.com.

[Matt Stern]

Awesome, thanks so much!