Protobuf-net Serialize problem

[test...@gmail.com]

Does someone have a Serialize sample for protobuf-net? I'm using VS
2005/C#.

When I try Encoding.ASCII.GetBytes() and Encoding.ASCII.GetString(), I
get an exception "invalid wire type - 7". If I
Convert.FromBase64String, serialize/deserialize works but the message
is rejected by the server which is running on Linux and is expecting a
valid protobuf as a string.

private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.Serialize<OrderProto>(stream, proto);
byte[] data = new byte[stream.Length];
stream.Seek(0, SeekOrigin.Begin);
stream.Read(data, 0, data.Length);
return Convert.ToBase64String(data);
}
}

private OrderProto TestDeserialize(string str)
{
byte[] data = Convert.FromBase64String(str);
MemoryStream stream = new MemoryStream(data);
OrderProto proto = Serializer.Deserialize<OrderProto>
(stream);
return proto;
}

OrderProto proto = p.GetOrderProto(order.OrderID);
string protoStr = this.Serialize(proto); // but as Base64, this
protobuf is rejected by the server
OrderProto test = TestDeserialize(protoStr);

Another thing I noticed is that if I re-serialize a protobuf I
received from the server, the resulting byte array is roughly 10 bytes
shorter than when came in. But the deserialized class instance looks
correct.

OrderProto order =
Serializer.Deserialize<OrderProto>(new MemoryStream(contentBytes));
MemoryStream ms = new MemoryStream();
Serializer.Serialize<OrderProto>(ms, order);
ms.Seek(0, SeekOrigin.Begin);
byte[] buffer = ms.ToArray();
OrderProto test =
Serializer.Deserialize<OrderProto>(ms);

Thanks

[Marc Gravell]

Hi; I'm the author of protobuf-net, and I hope I can help you find
what is going on here...

The binary format should be compatible between platforms (that is the
point, after all). Are you able to share your OrderProto class with
me? Either here, or at marc.g...@gmail.com. The base-64 should be
unrelated as long as it is being packed/unpacked correctly. I know of
many people using protobuf-net to communicate between both client/
server and windows/linux (mono), so it should work...

If you are transporting over a network - then one common issue is that
the protocol buffers format is not terminated; messages will bleed
into each-other unless you separate messages. You can do this using
SerializeWithLengthPrefix and DeserializeWithLengthPrefix, which
include the length of the message and only read that much data from
the stream.

Marc Gravell

[test...@gmail.com]

Marc,

Thanks for responding.

As I'd mentioned, the server is expecting the proto serialized as
binary and passed as a string param. It's doing the following.

Order::ProtoBufType sourceBuff;

if(!sourceBuff.ParseFromString(params["Order"]->string_value())) {
// exception

I'm able to gen the proto and use it elsewhere successfully without
having to handle encoding. This is how the server sends out the msg.

e->order.SerializeToString(&sOrder);

This is my first attempt to *send* a proto back to the server. I
tried encoding it as Unicode and while I didn't get the "invalid wire
type" error, the package was still rejected by the server.

private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.Serialize<OrderProto>(stream, proto);
byte[] data = new byte[stream.Length];
stream.Seek(0, SeekOrigin.Begin);
stream.Read(data, 0, data.Length);

return Encoding.Unicode.GetString(data);
}
}

private OrderProto TestDeserialize(string str)
{
byte[] data = Encoding.Unicode.GetBytes(str);

MemoryStream stream = new MemoryStream(data);
OrderProto proto = Serializer.Deserialize<OrderProto>
(stream);
return proto;
}

Also, do you know why buffer.Length would be slightly smaller than
contentBytes.Length in the following:

byte[] contentBytes = contents.Value;

OrderProto order = Serializer.Deserialize<OrderProto>(new MemoryStream
(contentBytes));
MemoryStream ms = new MemoryStream();
Serializer.Serialize<OrderProto>(ms, order);
ms.Seek(0, SeekOrigin.Begin);
byte[] buffer = ms.ToArray();

Here's the OrderProto class.

Thanks

[System.Serializable, ProtoBuf.ProtoContract(Name = @"Order")]

public partial class OrderProto : ProtoBuf.IExtensible
{
public OrderProto() { }


private string _ID0EW = "";

[ProtoBuf.ProtoMember(1, IsRequired = false, Name =
@"clearingid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string clearingid
{
get { return _ID0EW; }
set { _ID0EW = value; }
}

private string _ID0E6 = "";

[ProtoBuf.ProtoMember(2, IsRequired = false, Name =
@"clientid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string clientid
{
get { return _ID0E6; }
set { _ID0E6 = value; }
}

private string _ID0EIB = "";

[ProtoBuf.ProtoMember(3, IsRequired = false, Name =
@"ordersource", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string ordersource
{
get { return _ID0EIB; }
set { _ID0EIB = value; }
}

private string _ID0ERB = "";

[ProtoBuf.ProtoMember(4, IsRequired = false, Name =
@"orderid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string orderid
{
get { return _ID0ERB; }
set { _ID0ERB = value; }
}

private ulong _ID0E1B = default(ulong);

[ProtoBuf.ProtoMember(5, IsRequired = false, Name =
@"ordercreationtime", DataFormat =
ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(ulong))]

public ulong ordercreationtime
{
get { return _ID0E1B; }
set { _ID0E1B = value; }
}

private string _ID0EDC = "";

[ProtoBuf.ProtoMember(6, IsRequired = false, Name =
@"parentorderid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string parentorderid
{
get { return _ID0EDC; }
set { _ID0EDC = value; }
}

private string _ID0EMC = "";

[ProtoBuf.ProtoMember(7, IsRequired = false, Name =
@"rootorderid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string rootorderid
{
get { return _ID0EMC; }
set { _ID0EMC = value; }
}

private string _ID0EVC = "";

[ProtoBuf.ProtoMember(8, IsRequired = false, Name =
@"subsystemid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string subsystemid
{
get { return _ID0EVC; }
set { _ID0EVC = value; }
}

private ulong _ID0E5C = default(ulong);

[ProtoBuf.ProtoMember(9, IsRequired = false, Name = @"symbol",
DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(ulong))]

public ulong symbol
{
get { return _ID0E5C; }
set { _ID0E5C = value; }
}

private readonly
System.Collections.Generic.List<OrderProto.OrderStateProto> _ID0EHD =
new System.Collections.Generic.List<OrderProto.OrderStateProto>();

[ProtoBuf.ProtoMember(1776, Name = @"orderstate", DataFormat =
ProtoBuf.DataFormat.Default)]

public
System.Collections.Generic.List<OrderProto.OrderStateProto> orderstate
{
get { return _ID0EHD; }

}

[System.Serializable, ProtoBuf.ProtoContract(Name =
@"OrderState")]

public partial class OrderStateProto : ProtoBuf.IExtensible
{
public OrderStateProto() { }


private int _ID0E6D = default(int);

[ProtoBuf.ProtoMember(1, IsRequired = false, Name =
@"orderversion", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int orderversion
{
get { return _ID0E6D; }
set { _ID0E6D = value; }
}

private int _ID0EIE = default(int);

[ProtoBuf.ProtoMember(2, IsRequired = false, Name =
@"state", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int state
{
get { return _ID0EIE; }
set { _ID0EIE = value; }
}

private int _ID0ERE = default(int);

[ProtoBuf.ProtoMember(3, IsRequired = false, Name =
@"ordertype", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int ordertype
{
get { return _ID0ERE; }
set { _ID0ERE = value; }
}

private int _ID0E1E = default(int);

[ProtoBuf.ProtoMember(4, IsRequired = false, Name =
@"execat", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int execat
{
get { return _ID0E1E; }
set { _ID0E1E = value; }
}

private uint _ID0EDF = default(uint);

[ProtoBuf.ProtoMember(5, IsRequired = false, Name =
@"execflags", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(uint))]

public uint execflags
{
get { return _ID0EDF; }
set { _ID0EDF = value; }
}

private int _ID0EMF = default(int);

[ProtoBuf.ProtoMember(6, IsRequired = false, Name =
@"side", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int side
{
get { return _ID0EMF; }
set { _ID0EMF = value; }
}

private int _ID0EVF = default(int);

[ProtoBuf.ProtoMember(7, IsRequired = false, Name =
@"orderqty", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int orderqty
{
get { return _ID0EVF; }
set { _ID0EVF = value; }
}

private int _ID0E5F = default(int);

[ProtoBuf.ProtoMember(8, IsRequired = false, Name =
@"cumqty", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int cumqty
{
get { return _ID0E5F; }
set { _ID0E5F = value; }
}

private int _ID0EHG = default(int);

[ProtoBuf.ProtoMember(9, IsRequired = false, Name =
@"leavesqty", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int leavesqty
{
get { return _ID0EHG; }
set { _ID0EHG = value; }
}

private long _ID0EQG = default(long);

[ProtoBuf.ProtoMember(10, IsRequired = false, Name =
@"avgpx", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long avgpx
{
get { return _ID0EQG; }
set { _ID0EQG = value; }
}

private long _ID0EZG = default(long);

[ProtoBuf.ProtoMember(11, IsRequired = false, Name =
@"price", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long price
{
get { return _ID0EZG; }
set { _ID0EZG = value; }
}

private long _ID0ECH = default(long);

[ProtoBuf.ProtoMember(12, IsRequired = false, Name =
@"stoppx", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long stoppx
{
get { return _ID0ECH; }
set { _ID0ECH = value; }
}

private string _ID0ELH = "";

[ProtoBuf.ProtoMember(13, IsRequired = false, Name =
@"clordid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string clordid
{
get { return _ID0ELH; }
set { _ID0ELH = value; }
}

private uint _ID0EUH = default(uint);

[ProtoBuf.ProtoMember(14, IsRequired = false, Name =
@"ordstatus", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(uint))]

public uint ordstatus
{
get { return _ID0EUH; }
set { _ID0EUH = value; }
}

private string _ID0E4H = "";

[ProtoBuf.ProtoMember(15, IsRequired = false, Name =
@"exdestination", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string exdestination
{
get { return _ID0E4H; }
set { _ID0E4H = value; }
}

private ulong _ID0EHAAC = default(ulong);

[ProtoBuf.ProtoMember(16, IsRequired = false, Name =
@"expirationtime", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(ulong))]

public ulong expirationtime
{
get { return _ID0EHAAC; }
set { _ID0EHAAC = value; }
}

private ulong _ID0EQAAC = default(ulong);

[ProtoBuf.ProtoMember(17, IsRequired = false, Name =
@"transactiontime", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(ulong))]

public ulong transactiontime
{
get { return _ID0EQAAC; }
set { _ID0EQAAC = value; }
}

private int _ID0EZAAC = default(int);

[ProtoBuf.ProtoMember(18, IsRequired = false, Name =
@"reasontype", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int reasontype
{
get { return _ID0EZAAC; }
set { _ID0EZAAC = value; }
}

private string _ID0ECBAC = "";

[ProtoBuf.ProtoMember(19, IsRequired = false, Name =
@"reasontext", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string reasontext
{
get { return _ID0ECBAC; }
set { _ID0ECBAC = value; }
}

private long _ID0ELBAC = default(long);

[ProtoBuf.ProtoMember(20, IsRequired = false, Name =
@"marketopen", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long marketopen
{
get { return _ID0ELBAC; }
set { _ID0ELBAC = value; }
}

private long _ID0EUBAC = default(long);

[ProtoBuf.ProtoMember(21, IsRequired = false, Name =
@"bid", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long bid
{
get { return _ID0EUBAC; }
set { _ID0EUBAC = value; }
}

private int _ID0E4BAC = default(int);

[ProtoBuf.ProtoMember(22, IsRequired = false, Name =
@"bidsize", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int bidsize
{
get { return _ID0E4BAC; }
set { _ID0E4BAC = value; }
}

private string _ID0EGCAC = "";

[ProtoBuf.ProtoMember(23, IsRequired = false, Name =
@"bidmarket", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string bidmarket
{
get { return _ID0EGCAC; }
set { _ID0EGCAC = value; }
}

private long _ID0EPCAC = default(long);

[ProtoBuf.ProtoMember(24, IsRequired = false, Name =
@"ask", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long ask
{
get { return _ID0EPCAC; }
set { _ID0EPCAC = value; }
}

private int _ID0EYCAC = default(int);

[ProtoBuf.ProtoMember(25, IsRequired = false, Name =
@"asksize", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int asksize
{
get { return _ID0EYCAC; }
set { _ID0EYCAC = value; }
}

private string _ID0EBDAC = "";

[ProtoBuf.ProtoMember(26, IsRequired = false, Name =
@"askmarket", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string askmarket
{
get { return _ID0EBDAC; }
set { _ID0EBDAC = value; }
}

private ulong _ID0EKDAC = default(ulong);

[ProtoBuf.ProtoMember(27, IsRequired = false, Name =
@"nbbotime", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(ulong))]

public ulong nbbotime
{
get { return _ID0EKDAC; }
set { _ID0EKDAC = value; }
}

private int _ID0ETDAC = default(int);

[ProtoBuf.ProtoMember(28, IsRequired = false, Name =
@"exectype", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int exectype
{
get { return _ID0ETDAC; }
set { _ID0ETDAC = value; }
}

private int _ID0E3DAC = default(int);

[ProtoBuf.ProtoMember(29, IsRequired = false, Name =
@"execsource", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int execsource
{
get { return _ID0E3DAC; }
set { _ID0E3DAC = value; }
}

private string _ID0EFEAC = "";

[ProtoBuf.ProtoMember(30, IsRequired = false, Name =
@"execid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string execid
{
get { return _ID0EFEAC; }
set { _ID0EFEAC = value; }
}

private long _ID0EOEAC = default(long);

[ProtoBuf.ProtoMember(31, IsRequired = false, Name =
@"lastpx", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(long))]

public long lastpx
{
get { return _ID0EOEAC; }
set { _ID0EOEAC = value; }
}

private int _ID0EXEAC = default(int);

[ProtoBuf.ProtoMember(32, IsRequired = false, Name =
@"lastqty", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(int))]

public int lastqty
{
get { return _ID0EXEAC; }
set { _ID0EXEAC = value; }
}

private string _ID0EAFAC = "";

[ProtoBuf.ProtoMember(33, IsRequired = false, Name =
@"execrefid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string execrefid
{
get { return _ID0EAFAC; }
set { _ID0EAFAC = value; }
}

private string _ID0EJFAC = "";

[ProtoBuf.ProtoMember(34, IsRequired = false, Name =
@"liquidityflag", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string liquidityflag
{
get { return _ID0EJFAC; }
set { _ID0EJFAC = value; }
}

private string _ID0ESFAC = "";

[ProtoBuf.ProtoMember(35, IsRequired = false, Name =
@"fixpropbag", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string fixpropbag
{
get { return _ID0ESFAC; }
set { _ID0ESFAC = value; }
}

private uint _ID0E2FAC = default(uint);

[ProtoBuf.ProtoMember(36, IsRequired = false, Name =
@"ordercapacity", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(uint))]

public uint ordercapacity
{
get { return _ID0E2FAC; }
set { _ID0E2FAC = value; }
}

private uint _ID0EEGAC = default(uint);

[ProtoBuf.ProtoMember(37, IsRequired = false, Name =
@"lastcapacity", DataFormat = ProtoBuf.DataFormat.TwosComplement)]
[System.ComponentModel.DefaultValue(default(uint))]

public uint lastcapacity
{
get { return _ID0EEGAC; }
set { _ID0EEGAC = value; }
}

private string _ID0ENGAC = "";

[ProtoBuf.ProtoMember(38, IsRequired = false, Name =
@"traderid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]

public string traderid
{
get { return _ID0ENGAC; }
set { _ID0ENGAC = value; }
}

private ProtoBuf.IExtension extensionObject;
ProtoBuf.IExtension ProtoBuf.IExtensible.GetExtensionObject
(bool createIfMissing)
{ return ProtoBuf.Extensible.GetExtensionObject(ref
extensionObject, createIfMissing); }
}

private ProtoBuf.IExtension extensionObject;
ProtoBuf.IExtension ProtoBuf.IExtensible.GetExtensionObject
(bool createIfMissing)
{ return ProtoBuf.Extensible.GetExtensionObject(ref
extensionObject, createIfMissing); }

}
On Apr 2, 6:13 am, Marc Gravell <marc.grav...@gmail.com> wrote:
> Hi; I'm the author of protobuf-net, and I hope I can help you find
> what is going on here...
>
> The binary format should be compatible between platforms (that is the
> point, after all). Are you able to share your OrderProto class with

> me? Either here, or at marc.grav...@gmail.com. The base-64 should be

[test...@gmail.com]

It seems I was wrong in trying to use ASCII or UTF-8 encoding. This
works.

private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.Serialize<OrderProto>(stream, proto);

using (StreamReader reader = new StreamReader(stream))
return reader.ReadToEnd();
}
}

Thanks

[test...@gmail.com]

Actually, I was wrong, this code produces an empty string. The
following code serializes *something* which fails on the server side
on ParseFromString. Any ideas what I have to do to correctly serialize
this?

private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.Serialize<OrderProto>(stream, proto);

stream.Seek(0, SeekOrigin.Begin);
using (BinaryReader reader = new BinaryReader(stream,
Encoding.ASCII))
{
char[] chars = new char[stream.Length];
for (int i = 0; i < stream.Length; i++)
chars[i] = reader.ReadChar();
return new string(chars);
}
}
}

Thanks

On Apr 2, 1:04 pm, "test.f...@nomail.please" <test.f...@gmail.com>
wrote:

[test...@gmail.com]

Well, this code certainly passes the unit test of comparing before and
after protobufs. But the string still fails to be recognized as a
valid protobuf object on the server.

Any ideas?

Thanks

private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.Serialize<OrderProto>(stream, proto);

byte[] data = stream.ToArray();

char[] chars = new char[stream.Length];

for (int i = 0; i < data.Length; i++)
chars[i] = (char) data[i];
return new string(chars);
}
}

private OrderProto Deserialize(string protoStr)
{
char[] chars = protoStr.ToCharArray();
byte[] data = new byte[chars.Length];
for (int i = 0; i < chars.Length; i++)
data[i] = (byte)chars[i];
using (MemoryStream stream = new MemoryStream(data))
return Serializer.Deserialize<OrderProto>(stream);
}

[Marc Gravell]

Quick question - which build are you using? (at client and server)?

Based on the field names, the code-generation dates back to November;
if the dlls are aged similarly, it is possible that this is an old
bug. If possible, can you retry with the current build?

I'm just trying the code you posted, to see if I can spot anything...

Marc

[Marc Gravell]

You can't use unicode to transport arbitrary binary data - it won't be
a valid unicode string. If you have to use strings, then base-64 is
the safest option:

private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.Serialize<OrderProto>(stream, proto);

return Convert.ToBase64String(stream.GetBuffer(), 0,
(int)stream.Length);
}
}
private OrderProto Deserialize(string str)
{
byte[] data = Convert.FromBase64String(str);

using (MemoryStream stream = new MemoryStream(data)) {
return Serializer.Deserialize<OrderProto>(stream);
}
}

However, if possible, raw binary would be even more efficient.

I've tried the above (with the current build), and it round-trips
fine, giving the same length when repeated.

Does the above fix it at all?

Marc

[Marc Gravell]

(re last post)

As with unicode, you can't use ASCII to handle raw binary; either use
base-64 (previous post), or send binary.

Marc

[test...@gmail.com]

Marc,

The server is using ProtoBuf 2.0.1. I had to upgrade my .net version
to the latest protobuf-net in order to have protogen work with
optional fields correctly.

It seems that the server only processes bytes up to the 1st NULL char
when calling ParseFromString. Will this problem be avoided with
Base64?

Thanks

[Marc Gravell]

OK; can I clafy then; the server is using the standard C++ version of
ProtoBuf 2.0.1, and the client is protobuf-net (recent version), and
you are having difficulty using ParseFromString in the C++ version.

Is that correct?

I will need to check (Kenton?) what format ParseFromString is
expecting, as I simply haven't had need to use this. I will launch a
new thread to try to find out...

Marc

[Marc Gravell]

I've just been looking more - and ParseFromString should do the job.
I /suspect/ the problem is the definition a "string"... you aren't
going to be able to treat raw binary in .NET as a String; you are
going to have to treat it as a Stream or a byte[], or you can encode
it to base-64 (as above) to get something that *can* be treated as a
String.

Either as binary or a base-64, you would then transfer that to the
server. At the server, ProtoBuf again expects raw binary, but in a C++
string (if you have used base-64 as a transfer mechanism, you would
have to decode the base-64 back to binary). You should then be able to
mount that data into a C++ string (or an istream).

My best guess is that at the moment you are losing data during
transfer (as above).

What mechanism are you using to transfer the data? If it is a network
socket (for example) you should be able to just send binary, without
ever touching a .NET string; likewise HTTP can be used without
creating a .NET string.

Marc

[test...@gmail.com]

Marc,

Yes, we suspect the problem is in the transmit from .Net as well due
to the truncation at 1st NULL char in the Unicode string. I'm sending
the data back as a TIBCO message which also adds the field as a
string. Do you think if I use the AddField(string, byte[]) overload,
it'll work on the server?

Thanks

[Marc Gravell]

I'm not an expect on TIBCO messages; and I'm not sure what AddField
(string,byte[]) method you are talking about, sorry.

But if that is a Tibco method for packing a (named) byte[] into a
Tibco message, then it sounds ideal. Serialize to a MemoryStream and
use .ToArray() to get the byte[] - for example:

using(var ms = new MemoryStream()) {
Serializer.Serialize(ms, obj);
message.AddField("pb", ms.ToArray());
}

This is based purely on speculation about the AddField message! But it
sounds promising.

Marc

[test...@gmail.com]

Marc,

We figured out the problem. I had to send it over TIBCO as byte[] and
on the server side, it had to be pulled out as tibrv_u8array. After
that, the protobuf string conversion worked.

However, I noticed couple of odd things in this latest release of
protobuf-net.

1) It seems that an Optional Int field generated with detectMissing
flag does not get serialized if it was set to 0 (meaning *Specified
field was true but the field didn't reach the server).

2) optional string fields are being handled incorrectly.
public string execid
{
get { return _execid ?? ""; }
set { _execid = value; }
}
If execId was null, it will get serialized as an empty string which
means it ends up getting on the wire.

I'm going to revert back to gen'ing the class without the
detectMissing flag.

Thanks

[test...@gmail.com]

Marc,

ACtually, not using detectMissing will fix the string serialization
but not the primitive types which would default to default(T). How
will protobuf-net serialize the field if I munged the signature as

public int? Foo { get; set; }

Thanks

[test...@gmail.com]

Marc,

another issue is that for optional string fields, protogen defaults to
"". Shouldn't it default to null?

optional string clientid = 2;

private string _clientid = "";

[ProtoBuf.ProtoMember(2, IsRequired = false, Name =
@"clientid", DataFormat = ProtoBuf.DataFormat.Default)]
[System.ComponentModel.DefaultValue("")]
public string clientid
{

get { return _clientid; }
set { _clientid = value; }
}

Thanks