C++ Server Send capnp segments to java_client by rpc_call, In java_client how can I get the message in Correct

[张小]

C++ server send Capn Object to JavaClient like this

capnp::MallocMessageBuilder message;

Info::Builder info = message.initRoot<Info>();

info.setId(123);

kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments = message.getSegmentsForOutput();

then I convert the  segments  object to char* send to java_client by proto rpc_call (bytes field)

Then In java_client, I do like this to read the capn object

com.google.protobuf.ByteString capn_object_bytes =   response.getCapnObject()

SegmentReader segment = new SegmentReader(capn_object_bytes.asReadOnlyByteBuffer(), null);

AnyPointer.Reader any = new AnyPointer.Reader(segment, 0, 64*1024*1024);

Info.Reader info = any.getAs(Info.factory);

System.out.println("Id:" + info.getId());

run java_cliet ,happend error like this :

Exception in thread "main" java.lang.NullPointerException

at org.capnproto.WireHelpers.readStructPointer(WireHelpers.java:918)

at org.capnproto.StructFactory.fromPointerReaderRefDefault(StructFactory.java:34)

at org.capnproto.StructFactory.fromPointerReader(StructFactory.java:41)

at org.capnproto.StructFactory.fromPointerReader(StructFactory.java:24)

at org.capnproto.AnyPointer$Reader.getAs(AnyPointer.java:56)

[David Renshaw]

On the Java side, you need to first read the bytes into a `MessageReader`. That's typically done via one of the `Serialize.read()` methods.

The `AnyPointer.Reader()` is not intended for external use. Probably we should make it private.

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/081ab20b-5eae-43d5-a4fc-8230ef39d4a5%40googlegroups.com.

[张小]

Yes, I use like this :

MessageReader message = org.capnproto.Serialize.read(capn_object_bytes.asReadOnlyByteBuffer());, 

Info.Reader adInfo = message.getRoot(Info.factory);

but error like this :

Exception in thread "main" java.lang.IllegalArgumentException

at java.nio.Buffer.limit(Buffer.java:275)

at org.capnproto.Serialize.read(Serialize.java:140)

at org.capnproto.Serialize.read(Serialize.java:111)

 

在 2019年9月30日星期一 UTC+8下午8:29:41，David Renshaw写道：

To unsubscribe from this group and stop receiving emails from it, send an email to capn...@googlegroups.com.

[张小]

First Time:

 In C++ server,I Use capnp::DynamicStruct, code like this:

capnp::MallocMessageBuilder msg;

capnp::DynamicStruct::Builder fullinfo_builder = msg.initRoot<capnp::DynamicStruct>(g_schema);

capnp::DynamicStruct::Builder fullInfo =  msg.initRoot<capnp::DynamicStruct>(g_schema);

capnp::messageToFlatArray(msg);  ---->convert it to char* send to java_cliet

Java Client ,code like this:

MessageReader message = org.capnproto.Serialize.read(capn_object_bytes.asReadOnlyByteBuffer());,

Info.Reader adInfo = message.getRoot(Info.factory);

Above ， My First Time do work , but The Second Time, I do like this ,it do not work:

 In C++ server, code like this:

capnp::MallocMessageBuilder message;

Info::Builder info = message.initRoot<Info>();

info.setId(123);

kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments = message.getSegmentsForOutput();

then I convert the  segments  object to char* send to java_client by proto rpc_call (bytes field)

Java Client ,code like this:

MessageReader message = org.capnproto.Serialize.read(capn_object_bytes.asReadOnlyByteBuffer());,

Info.Reader adInfo = message.getRoot(Info.factory);

In Second Time, happen Error like this:

Exception in thread "main" java.lang.IllegalArgumentException

at java.nio.Buffer.limit(Buffer.java:275)

at org.capnproto.Serialize.read(Serialize.java:140)

at org.capnproto.Serialize.read(Serialize.java:111)

How Can I do , In Second Time ,thanks 

在 2019年9月30日星期一 UTC+8下午8:29:41，David Renshaw写道：

On the Java side, you need to first read the bytes into a `MessageReader`. That's typically done via one of the `Serialize.read()` methods.

To unsubscribe from this group and stop receiving emails from it, send an email to capn...@googlegroups.com.

[Kenton Varda]

Are you transmitting one big buffer containing all the segments, or are you transmitting each segment separately?

It looks like you're trying to do the latter, but in that case you cannot use Serialize.read() to read it. You need to use `new MessageReader(segments)`.

-Kenton

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/816c3227-c2b2-4b72-8ead-e707759a257c%40googlegroups.com.

[张小]

hi Kenton：

  I need your help,This problem has been bothering me for many days.

In c++ server ,code like this 

capnp::MallocMessageBuilder message;

FullInfo::Builder fullInfo = message.initRoot<FullInfo>();

fullInfo.setGoodsId(919731)

kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments = message.getSegmentsForOutput();

kj::ArrayPtr<const capnp::word> segment = segments[0];

kj::ArrayPtr<const char> chars = segment.asChars();

const char* mem_buf = chars.begin();

int32_t mem_buf_len = chars.size();

and c++ server send the mem_buf to java_client by rpc call(bytes field)

Client and server communicate through protobuf rpc

In java_client code like this:() 

com.google.protobuf.ByteString capn_object_bytes =   response.getCapnObject();

ByteBuffer[] bytes_buffer = new ByteBuffer[1];

bytes_buffer[0] = capn_object_bytes.asReadOnlyByteBuffer();

MessageReader message = new MessageReader(bytes_buffer, ReaderOptions.DEFAULT_READER_OPTIONS);

Display.FullInfo.Reader fullInfo  = message.getRoot(Display.FullInfo.factory);

System.out.println("goodsId:" + fullInfo.getGoodsId());

but in java_client ，print goodsId is 0, however in c++ server, I  setGoodsId(919731) ,What's wrong with my method of use? @Kenton

the fullinfo.capnp code is:

@0xcb70f505c89d1634;

using Java = import "./compiler/src/main/schema/capnp/java.capnp";

$Java.package("org.capnproto.examples");

$Java.outerClassname("Display");

struct FullInfo{

    goodsId @1 :Int64;

 }

在 2019年10月1日星期二 UTC+8上午3:23:37，Kenton Varda写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/816c3227-c2b2-4b72-8ead-e707759a257c%40googlegroups.com.

[Kenton Varda]

Your code only handles the case of a single segment. I recommend using the methods in capnp/serialize.h and org.capnproto.Serialize. I do not recommend using getSegmentsForOutput() nor constructing a MessageReader directly from a segment array, as these are advanced functions that can more easily go wrong.

That said, in your example case, only a single segment should be needed, so something else is wrong too.

I don't see any other obvious problems with the code you provided. I suspect a bug exists in the code you didn't show. I recommend you try to verify that the bytes on the receiving end are actually exactly the bytes from the sending end. Try logging the byte values and the size of the segment at each end to make sure everything matches.

I think the most likely problem is that somewhere your `char*` is being interpreted as a NUL-terminated string, and is therefore being truncated at the first zero-valued byte. Most likely, the very first byte in the segment is a zero, so probably your ByteBuffer on the Java end ends up being zero-length. This probably leads to the struct appearing to contain only default values, hence goodsId is 0.

-Kenton

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/1e177ff5-e49f-4d4a-b5f3-1c096604bb29%40googlegroups.com.

[张小]

 hi Kenton：

    getSegmentsForOutput() performance better than messageToFlatArray right?  These two interfaces  What scenarios are applied to?

  best wish

  whutbd

在 2019年10月7日星期一 UTC+8上午3:41:23，Kenton Varda写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/1e177ff5-e49f-4d4a-b5f3-1c096604bb29%40googlegroups.com.

[Kenton Varda]

messageToFlatArray() involves an extra copy compared to getSegmentsForOutput().

However, you are putting the bytes into a protobuf anyway. Protobuf will make many copies. So I don't think you should worry too much about this one extra copy.

If you really want to use getSegmentsForOutput(), then you need to use a `repeated bytes` field in protobuf and you need to add each segment to the repeated bytes.

-Kenton

To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/222eed4a-8f6a-48e7-828e-09235b228253%40googlegroups.com.

[Kenton Varda]

Your message may have more than one segment if you put enough data into it. You shouldn't assume that messages will always have one segment, unless you know that your message will always be less than a certain overall size, and you tell the MessageBuilder to allocate the first segment to be larger than that.

If you are embedding the message into a protobuf, you could use a `repeated bytes` field to allow for more than one segment. In that case it could make sense to use `getSegmentsForOutput()`.

-Kenton

On Fri, Nov 15, 2019 at 2:58 PM whutbd <7745...@qq.com> wrote:

hi Kenton Varda

  when I do this:

   kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments = message.getSegmentsForOutput();

  the segments size Always 1，so in protobuf I use bytes field to send the segments[0].asChars();

Is there any problem in using it like this

   
在 2019年10月8日星期二 UTC+8上午12:31:10，Kenton Varda写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/222eed4a-8f6a-48e7-828e-09235b228253%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/fa2451ad-4441-4c3a-8af0-39a9b45cb7c9%40googlegroups.com.

[Kenton Varda]

In this case you would want to construct `org.capnproto.MessageReader` directly from the array of `ByteBuffer`s; you would not use `Serialize`.

-Kenton

On Fri, Nov 15, 2019 at 5:04 PM 张钊 <7745...@qq.com> wrote:

Hi Kenton:

 If use repeated bytes In java client ,read capn object like this? 

com.google.protobuf.ByteString capn_object_bytes =   response.getCapnObject();

ByteBuffer[] bytes_buffer = new ByteBuffer[1];

bytes_buffer[0] = capn_object_bytes.asReadOnlyByteBuffer();

MessageReader message = new MessageReader(bytes_buffer, ReaderOptions.DEFAULT_READER_OPTIONS);

Display.FullInfo.Reader fullInfo  = message.getRoot(Display.FullInfo.factory);

System.out.println("goodsId:" + fullInfo.getGoodsId());

or use org.capnproto.Serialize.read(byteBuffer)?

thanks

whutbd

------------------ 原始邮件 ------------------

发件人: "Kenton Varda"<ken...@cloudflare.com>;

发送时间: 2019年11月16日(星期六) 凌晨5:53

收件人: "张钊"<7745...@qq.com>;

抄送: "Cap'n Proto"<capn...@googlegroups.com>;

主题: Re: [capnproto] C++ Server Send capnp segments to java_client by rpc_call, In java_client how can I get the message in Correct