Upgrading float fields to double

[James Philbin]

Hi,

Is it safe to upgrade float fields to doubles for serialized protos? If not, why not?

Thanks,

James

[Marc Gravell]

Not really, no. They take different amounts of space on the wire, and have a different declared wire type (header). Some libraries may choose to be gracious and apply the conversion silently, but other libraries could just say "unexpected wire type" and stop processing.

You could perhaps do it as a `oneof`, and have the client check both?

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.
To post to this group, send email to prot...@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.

[James Philbin]

But the type stored on the wire is known - so it seems like it could be supported (simply static_cast<double>(float_val)). By 'some libraries': does the C++ ParseFromString method do this or not?

Thanks!

James

On Wed, Nov 15, 2017 at 3:57 PM Marc Gravell <marc.g...@gmail.com> wrote:

Not really, no. They take different amounts of space on the wire, and have a different declared wire type (header). Some libraries may choose to be gracious and apply the conversion silently, but other libraries could just say "unexpected wire type" and stop processing.

You could perhaps do it as a `oneof`, and have the client check both?

On 15 Nov 2017 11:51 p.m., "James Philbin" <phil...@gmail.com> wrote:

Hi,

Is it safe to upgrade float fields to doubles for serialized protos? If not, why not?

Thanks,

James

--

You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.

[Bo Yang]

float is 4 bytes on wire, while double is 8 bytes on wire.

If you change float to double, in the case of old data and new parser, the parser will take the additional 4 bytes as double (these additional 4 bytes should belong to the next field in wire),

in the case of new data and old parser, the parser will only take 4 bytes (which should be 8 bytes.)

[liujisi]

The spec doesn't allow such conversion. If some language implementation supports the conversion, it is considered a bug by not following spec.

Protobuf is designed to interop across different languages/platforms. Implicit conversion in "some libraries" allows creating non portable payload which leads to issues/surprises when you change the language/implementation in part of your pipeline, or communicate to a different party.

That being said, you can propose to add the conversion to the spec to promote float->double promotion. That would require to change all the language implementations and we need to evaluate the cost and benefit.

On Wednesday, November 15, 2017 at 10:18:15 PM UTC-8, James Philbin wrote:

But the type stored on the wire is known - so it seems like it could be supported (simply static_cast<double>(float_val)). By 'some libraries': does the C++ ParseFromString method do this or not?

Thanks!

James

On Wed, Nov 15, 2017 at 3:57 PM Marc Gravell <marc.g...@gmail.com> wrote:

Not really, no. They take different amounts of space on the wire, and have a different declared wire type (header). Some libraries may choose to be gracious and apply the conversion silently, but other libraries could just say "unexpected wire type" and stop processing.

You could perhaps do it as a `oneof`, and have the client check both?

On 15 Nov 2017 11:51 p.m., "James Philbin" <phil...@gmail.com> wrote:

Hi,

Is it safe to upgrade float fields to doubles for serialized protos? If not, why not?

Thanks,

James

--

You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.

[James Philbin]

OK, thanks. Is there a recommended way of upgrading a float field to a double field? There is no discussion of this path right now in the protobuf docs.

James

On Thursday, November 16, 2017 at 11:16:58 AM UTC-8, liujisi wrote:

The spec doesn't allow such conversion. If some language implementation supports the conversion, it is considered a bug by not following spec.

Protobuf is designed to interop across different languages/platforms. Implicit conversion in "some libraries" allows creating non portable payload which leads to issues/surprises when you change the language/implementation in part of your pipeline, or communicate to a different party.

That being said, you can propose to add the conversion to the spec to promote float->double promotion. That would require to change all the language implementations and we need to evaluate the cost and benefit.

On Wednesday, November 15, 2017 at 10:18:15 PM UTC-8, James Philbin wrote:

But the type stored on the wire is known - so it seems like it could be supported (simply static_cast<double>(float_val)). By 'some libraries': does the C++ ParseFromString method do this or not?

Thanks!

James

On Wed, Nov 15, 2017 at 3:57 PM Marc Gravell <marc.g...@gmail.com> wrote:

Not really, no. They take different amounts of space on the wire, and have a different declared wire type (header). Some libraries may choose to be gracious and apply the conversion silently, but other libraries could just say "unexpected wire type" and stop processing.

You could perhaps do it as a `oneof`, and have the client check both?

On 15 Nov 2017 11:51 p.m., "James Philbin" <phil...@gmail.com> wrote:

Hi,

Is it safe to upgrade float fields to doubles for serialized protos? If not, why not?

Thanks,

James

--

You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+u...@googlegroups.com.

[Adam Cozzette]

I think the way to do it is just to create a new, separate double field and migrate over to that new field. Depending on how your message is used, you might have to temporarily populate both fields until all readers have been updated to prefer the new field.

To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscribe@googlegroups.com.