Firestore compatibility of protobuf object across languages

[Gil Salomon]

Storing protobuf objects to Firestore is not compatible across languages.

When you store protobuf object from go you will get uppercase type field keys.

When you store protobuf object from dart you will get numeric field keys

Other languages like java are even more messy not providing even out of the box methods for storing protobuf documents.

My question is how is it possible to store (and later read) protobuf objects to Firestore from different languages, hence storing the same protobuf object will end up in the same firestore document format across different languages.

[Sam Stern]

Hi Gil,

Could you explain a bit more why/how you're storing protobufs in Firestore? That's not a common use case.

- Sam

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/b6f3b256-3caf-4aef-b17c-2b3b1bbd0276n%40googlegroups.com.

[Gil Salomon]

I can see it's not so common for the support but it is very useful to be able to use protobuf messages as data model instead of managing both and manually setting data from one to the other.
Also seems like google has made the trouble of supporting it well for few languages but inconsistently.
As for my use case, I can say that I have the need to write code in a few languages (go, dart, java at least) to interact with the same data stored in Firestore.

You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/0y8nihK3XxE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAHafJBpms%2BF40D4rJNgJsnDWmFcKS%2B4HiSC%2Bbma_f4GF2mq8mg%40mail.gmail.com.

[Sam Stern]

These are the supported data types in Firestore:
https://firebase.google.com/docs/firestore/manage-data/data-types

If you want to store a ProtoBuf rather than a native firestore object containing those types, you'll have to convert it to a string or an array of bytes and decode it on the client.

> As for my use case, I can say that I have the need to write code in a few languages (go, dart, java at least) to interact with the same data stored in Firestore.

That's what the Firestore SDKs were designed to do, no protobufs necessary.

- Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CA%2B69R69oqm8U_K5W%2BW1j-EEeDwd62-iF021_sMxe%3Dh%2BN%2BtGM8A%40mail.gmail.com.

[Gil Salomon]

Thanks Sam,
I'm well aware of the basic data types, however a Firestore document is a hierarchical data structure and as it happens in go for example any object (interface) including protobuf is accepted and automatically converted so it was confusing to find out there is no standard convention of how the conversion is done and it is done in a different way in different languages (or not done at all like in Dart and Java).

[Alan deLespinasse]

I've thought about using protobufs with Firestore for cross-language type safety. I'd serialize to JSON to store them, so that indexing can work. That way you could avoid having to write a separate abstraction and validation layer in each language that accesses the database.

I kind of wish Firestore used protobufs natively instead of its schemaless pseudo-JSON system, but I guess that was probably ruled out because it would be a barrier to adoption. (I think maybe the old App Engine datastore used protobufs internally, like the Google-proprietary database it was based on?)

[Anastasios Andronidis]

I'm not sure if this helps, but you could use your existing protobuf rules to validate your data model through security rules like this: https://github.com/FirebaseExtended/protobuf-rules-gen, instead of serialise/deserialise them to store in firebase.

I agree with you though that firebase would significantly benefit from a direct API to use protobuf. I'm also interesting on how to synchronise/enforce a common data schema in a polyglot environment.