Real time server with google map

[Jothi Sankar]

Hi Everyone,

I am a beginner in Node.js. I have been working on some back-end server API that will listen for location information from our client devices and the API will drop a pin on google map that will be rendered in server ( without reloading the page ). We have been discussing about the server capabilities.In our case we are about to handle millions of server requested so we need a concurrent server like Node.js. We have been searching for the optimized way of implementing this in Node.js 

Need clarification on some basic area. Hope the community will clarifies our doubts. 

Should our server as HTTP server? 

  Coz on client devices does nothing with the responses from the server. It can be avoidable. 

What will be best suitable server for us like TCP?

Shoule we use web-sockets for this?

Share some tutorials that will really help us to understand the node server better. We are happy to learn but we need some one to guide us in right direction. So came here. 

Thanks and Regards,

Jothi sankar.

[Aria Stewart]

On Sunday, March 22, 2015 at 11:33:51 AM UTC-4, Jothi Sankar wrote:

Hi Everyone,

I am a beginner in Node.js. I have been working on some back-end server API that will listen for location information from our client devices and the API will drop a pin on google map that will be rendered in server ( without reloading the page ). We have been discussing about the server capabilities.In our case we are about to handle millions of server requested so we need a concurrent server like Node.js. We have been searching for the optimized way of implementing this in Node.js 

Need clarification on some basic area. Hope the community will clarifies our doubts. 

Should our server as HTTP server? 

  Coz on client devices does nothing with the responses from the server. It can be avoidable. 

What will be best suitable server for us like TCP?

Shoule we use web-sockets for this?

What protocol you use depends on the constraints -- if you are on unrestricted networks, and using a native app, TCP can be a great default communication protocol, and probably your lowest overhead. Web browsers, however, won't speak plain TCP.

You can use HTTP-based protocols -- long polling, and where there's no proxy to interfere, websockets. Websockets, once established, have less overhead than most HTTP-based communication since there's not headers going back and forth on each transaction, and the connection stays established.

Long polling HTTP is probably the most widely available, but also the most inefficient of the three protocols. (Could do worse... could always do short polling)

Your bottlenecks will be on the number of sockets a single server can maintain -- file descriptor limits in your OS, buffer memory allocation per socket, lots of tuning when you get over 10,000 simultaneous connections per server. It's completely doable, but it's going to take some effort to scale that.

Tools like engine.io and socket.io have fallbacks, so they try different protocols and try to use the most efficient one that works given a network. I personally tend to go somewhat low level, to get the control needed to scale easily and to have less to debug, but your mileage may vary.

You will want to look at which direction the information flows (in to server, out of server, both), how often (is it a continuous stream, or is it periodic updates?).

If the information flow is bidirectional, or if it's constant, streaming, TCP or websocket-like things are best.

If it's one-way, from client to server, and intermittent, then it may make the most sense just to use plain HTTP and PUT or POST some data periodically.
 

Share some tutorials that will really help us to understand the node server better. We are happy to learn but we need some one to guide us in right direction. So came here. 

I don't know of tutorials offhand, but it sounds like you've got something interesting going on. Building a couple prototypes to test APIs and protocols is probably your best bet -- something super light, just to understand how you're actually sending data.

Aria

[Jothi Sankar]

Thanks Aria,

Your response is "clean" and "understandable" and now i can get some clarity from your suggestions.

The communication and data flow will mostly be one-directional and we don't need to maintain the connections between client and server. It seems using plan HTTP will be the wise choice.

Am just curious about one thing. Hope you can help me with that.

I have created a http server that renders a page with google map in it with the help of express and jade modules. 

   

   app.get('/', function(req, res){
      res.render('mymap');
   });

It is working fine. I can able to view the map and interact with it when access it via (http://my-host-name:IP). Lets say when the server receives further request like ('/location') with location information is it possible to add a new marker/pin on the rendered map ( without re-render / reload the page on server )???

[Aria Stewart]

On Friday, March 27, 2015 at 8:04:14 AM UTC+13, Jothi Sankar wrote:

Thanks Aria,

Your response is "clean" and "understandable" and now i can get some clarity from your suggestions.

The communication and data flow will mostly be one-directional and we don't need to maintain the connections between client and server. It seems using plan HTTP will be the wise choice.

Am just curious about one thing. Hope you can help me with that.

I have created a http server that renders a page with google map in it with the help of express and jade modules. 

   

   app.get('/', function(req, res){
      res.render('mymap');
   });

It is working fine. I can able to view the map and interact with it when access it via (http://my-host-name:IP). Lets say when the server receives further request like ('/location') with location information is it possible to add a new marker/pin on the rendered map ( without re-render / reload the page on server )???

To make a request from the browser to the server without refreshing the page, you use XMLHttpRequest (though it has nothing to do with XML, thanks legacy naming!) or a wrapper for it like $.ajax from jquery or any of the other wrappers. That lets you fetch and interpret what the server sends -- you'd request JSON, usually, and the server can `res.send(jsondata)`

Or, you connect via a socket method, and the server sends the data unsolicited after that.

In either case, then you can interpret that incoming data in the script in the browser, and call the functions to add points to the map.

Aria

[Jothi Sankar]

Hi,

@thanks Aria Strwart

I have made it work socketClusters for server and client ( my map page ) location communication. Its working fine.

My server(VPS) configuration is as follows

Ubuntu 14.10 64bit

8GB RAM

4 core

I am using socketCluster on the server with the no of worker equivalent to the no of Core/CPU (i.e) 4 workers in my case
The CPU usage with the htop command in server terminal is 45%
Memory usage was around 4GB / 8GB & swap space not used
The ab command i used to load the server was

ab -n 20000 -c 20000 "http://IP_ADDRESS:8000/API_INFO"

when i load test my server with the above command i am getting the below error

apr_socket_recv: Connection reset by peer (104)  

i have increased the ulimit - n 999999 and

no of soft and hard files opening to 1000000

Thats neither fix the issue. 

What would be the cause? how can i achieve more concurrency. as i have mentioned in my first mail i need to scale the server that can handle 1M concurrent connection.

Thanks in advance,

josan

On Sunday, March 22, 2015 at 9:03:51 PM UTC+5:30, Jothi Sankar wrote: