Golang with Real time database

[Agrim Gupta]

Hey,

I am trying to build a trading app with firebase and want to use Golang for matching orders.

I am learning Golang for this specific use case. I just read in the documents (https://firebase.google.com/docs/admin/setup) this:

Note: The Realtime Database API in the Go Admin SDK currently does not support realtime event listeners. This means there is no provision for adding event listeners to a database reference in order to automatically receive realtime update notifications. Instead, in Go, updates should be proactively fetched by explicitly invoking read operations.

Can someone explain what this means and suggest if this will affect me? I am going to need real-time listeners for getting the orders placed by clients and then match the buy orders with sell orders and vice versa.

Is this a deal-breaker? Should I use python instead (i already know python and learning Golang for its performance)?

Also, this issue aside is it worth learning Golang for this use case?

Thank you in advance.

[Kato Richardson]

Hello Agrim,

I don't believe Python does realtime event listeners either. What's the goal here? Is this a server environment or are you attempting to use Firebase in some sort of desktop app? The simple answer is that you should use an SDK that supports realtime events.  For what it's worth, I believe Python streams using the REST API, and it's probably possible to rig up something similar in Go if you want to reinvent a few wheels (connectivity, local caching, offline operations, authentication, et al).

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/fb42e101-2c56-461c-b079-4c272c487d28n%40googlegroups.com.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398

[Agrim Gupta]

Hey, thanks for the response. 

The link I sent Golang specifically had an asterisk which had the Note I sent in the previous email saying that it doesn’t support real-time event listeners. So I assumed that the rest of the languages do. 

Also, I found this in the documentation that says that python does have a real-time event listener in the Admin SDK: https://firebase.google.com/docs/reference/admin/python/firebase_admin.db#listenerregistration

Yes, it's a server environment where the goal is to have a WebSocket sort of thing where I can listen to the orders placed from the client-side and then process the orders server-side and then broadcast the order update to all clients as the order-book and the last traded price (current market price) should be broadcasted to all clients in real-time.

I didn't understand as to why I will have to reinvent these (connectivity, local caching, offline operations, authentication, et al)? Can you explain? Are you saying that with Admin SDK all these features come right out of the box but if I use REST API then will have to implement all these manually?

Are you aware of the reason why Go Admin SDK doesn't have real-time listener support or when can we expect that?

Also if you are aware of any third-party library that implements those features, that would be a great help.

I am still inclined to use golang for its performance but I am a startup and working alone and don't have a team to reinvent the wheel as you said and kind of using firebase for faster development.

Can you guide me as to what can be the possible solution?

Should I just use Python? (That is actually convenient as I will not have to learn a new language.)

The link I sent in the last mail says "Instead, in Go, updates should be proactively fetched by explicitly invoking read operations." 

How does this work? Periodically (say every sec) perform read operations to kind of emulate real-time feature?

But what if there are no updates say for an hour. Then all those read operations go to waste?

Thank you.

Regards.

You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/FlaIRABEwuE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CADypTEZAkw33LEUSwQ9PYe5e%3Dtkr%3DmnHGOiGvNWax%2BUjzBGmkw%40mail.gmail.com.

[Sam Stern]

Hi Agrim,

It sounds like you want your server to respond each time there is a new write to the database. Rather than use a server-side listener, which doesn't generally scale well, have you considered using Cloud Functions?

Here's a video that explains the concept. It's pretty old so some of the code might be a bit out of date, but the architecture and concepts remain the same:
https://www.youtube.com/watch?v=DglTSNEdl0U

- Sam

On Sat, Jul 10, 2021 at 5:42 AM Agrim Gupta <agri...@gmail.com> wrote:

Hey, thanks for the response. 

The link I sent Golang specifically had an asterisk which had the Note I sent in the previous email saying that it doesn’t support real-time event listeners. So I assumed that the rest of the languages do. 

Also, I found this in the documentation that says that python does have a real-time event listener in the Admin SDK: https://firebase.google.com/docs/reference/admin/python/firebase_admin.db#listenerregistration

Yes, it's a server environment where the goal is to have a WebSocket sort of thing where I can listen to the orders placed from the client-side and then process the orders server-side and then broadcast the order update to all clients as the order-book and the last traded price (current market price) should be broadcasted to all clients in real-time.

I didn't understand as to why I will have to reinvent these (connectivity, local caching, offline operations, authentication, et al)? Can you explain? Are you saying that with Admin SDK all these features come right out of the box but if I use REST API then will have to implement all these manually?

Are you aware of the reason why Go Admin SDK doesn't have real-time listener support or when can we expect that?

Also if you are aware of any third-party library that implements those features, that would be a great help.

Also on second thought do I really need those features given the code is going to run server-side?

I am still inclined to use golang for its performance but I am a startup and working alone and don't have a team to reinvent the wheel as you said and kind of using firebase for faster development.

Can you guide me as to what can be the possible solution?

Should I just use Python? (That is actually convenient as I will not have to learn a new language.)

The link I sent in the last mail says "Instead, in Go, updates should be proactively fetched by explicitly invoking read operations." 

How does this work? Periodically (say every sec) perform read operations to kind of emulate real-time feature?

But what if there are no updates say for an hour. Then all those read operations go to waste?

Thank you.

Regards.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/095036cd-e5d8-4f21-992c-41fca3a0ea20n%40googlegroups.com.

[Kato Richardson]

> I didn't understand as to why I will have to reinvent these (connectivity, local caching, offline operations, authentication, et al)? Can you explain? Are you saying that with Admin SDK all these features come right out of the box but if I use REST API then will have to implement all these manually?

Correct. Admin SDKs, like other Firebase SDKs, do a lot of optimizations for connectivity and read ops.

> Also on second thought do I really need those features given the code is going to run server-side?

You would certainly need some strategy for handling connectivity issues -- error handling and retry logic. It's hard to say about the rest; you'll have to decide that. I guess the tradeoff you'll have to make here is language of choice vs getting these features out of the box with Java or Node. 

It sounds like the wasted read ops are also a concern based on your last email. So you'd probably want some caching/optimization strategy other than downloading all the data on a regular interval just to see if it changed. There are cheap strategies for this of course. For example, write an update timestamp and only download the data if it has changed since your last read.

Really just depends on how much time you want to spend optimizing these and what scale you're working at, and whether it's worth the energy to use your preferred language.

> Should I just use Python?

I think you'll need to decide that. I think it's still using the streaming REST API instead of a more sophisticated protocol, so it's probably still a little less optimized that the Node and Java SDKs, but depending on the scale you're working at, maybe that's fine.

> Are you aware of the reason why Go Admin SDK doesn't have real-time listener support or when can we expect that?

I'd guess it's just not in high demand. Not really sure what the tradeoffs or priorities were that went into this decision.

☼, Kato

On Fri, Jul 9, 2021 at 9:42 PM Agrim Gupta <agri...@gmail.com> wrote:

Also on second thought do I really need those features given the code is going to run server-side?

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAL5upcRPY4J2WpW0ymrxUR%2B3ipChi%2B2Reop-GMVGCp-i2Uk4Mg%40mail.gmail.com.