Using node-red to monitor/chart the Quality of your incoming internet connection?

[Steve French]

Hello all!

Recently Dave, Julian and I kinda went off-topic in this thread talking about Ubiquiti networking equipment.  It triggered a great question that has been on my mind....how to best use node-red to report some kind of Broadband Quality report.  At my office I pay for business internet and it is fast....at home, however, things are often slow and painful.  I am not sure if my router is the problem or if the incoming internet service is questionable at times.  Of course, I could manually go to some broadband speed test like this...but...

What would be a good way for node-red to report/log internet/network throughput on a semi-regular basis?  Do I plot ping-times?  Do I automate file transfers and look at throughput rates?  This seems like good info for troubleshooting problems and perhaps ammunition for telling my ISP that they are under-spec @ home.  Any thoughts would be helpful.  Thx!

-frenchy (Steve French)

www.voltvision.com

[Julian Knight]

So there are a couple of things you want to track when looking at network performance.

1) Latency: Ping times can measure this, how long it takes for a packet to get from point a to point b. High latency makes gaming and interactive voice/video all but impossible.

2) Throughput: how much you can squeeze down and up the pipe in any given second.

Something like Speedtest.org will give you both figures and you should do some baselining by testing (always to the same server at the other end, I generally find that the London Vodafone node is good for UK measurements). Test of course at different times of day and days of the week. This gives you a good feel for the capability of your broadband.

But for an ongoing dashboard, Speedtest would be infeasible because it will suck the life out of your connection all the time. 

So pings can act as a proxy for latency measurements. Choose a reliable end point such as the BBC or Google. NR has a ping node you can use.

For throughput, a reasonable proxy measure is how fast you can reach and recover a web-page. You can do that using an http request node. 

You should be able to compare those proxy figures with your Speedtest runs to see how they compare and then you can report when the performance goes outside given criteria. All good and relatively simple.

While you are at it, you probably want to do some measurements against your router. At the least, you can ping it. If you see anything above a few milliseconds for a ping response, you know you have a network issue somewhere. But pings are not a very reliable measure since it is likely a network card itself may respond to the ICMP request which doesn't necessarily tell you if you have a higher level issue.

Depending on your router model, it may have other metrics that you can view. The best being something called SNMP. Bad news is that understanding SNMP is not always easy and many consumer router vendors don't publish their SNMP "MIBS" that you need to make sense of the data. Good news is that there is a node for SNMP! A simpler but less reliable method might be similar to your throughput proxy measure, try to grab the admin page from the router & see how long it takes. Since the web server that delivers the page runs as an application on the router, it can be a proxy measure for how busy the router is.

If you have other servers on your network such as a NAS or a media server, you may be able to do similar measures. Don't forget to also measure the performance and busyness of your Pi as well.

Now you have a set of figures and you will need to slowly become familiar with what they mean. You will be able to see whether any slow downs are due to the router or something beyond. The absolute numbers are not as meaningful as comparative ones. Are things better or worse than "normal". 

NR is great for experimenting with these things easily. But it isn't always the best tool so don't forget other tools to. There are plenty of tools around that record and graph all of the things we have spoken about. One of the easiest to work with I've found in a long time is Telegraf from the same people as InfluxDB. Telegraf has a number of built-in measures that make it easy to record network and machine metrics. Grafana makes it easy to visualise them. InfluxDB makes it easy to record them. All are available pre-packaged for the Pi and will run happily (on a Pi2 at least) along side a reasonably complex Node-Red installation.

One of my next jobs, now that the new router is working pretty well, is for me to turn on the SNMP monitoring and begin experimenting. Doubtless I will use NR first as it is easy to mess around but I will undoubtedly end up adding to the Telegraf configuration so that I can pull the Pi, the NAS, the Router and the external measures together and show them on an integrated dashboard. I've only done the Pi and some external measures so far.

Oh, and don't forget to record the performance of the DNS you are using. I use OpenDNS. It provides some family safety features and is generally fast but sometimes it can be slower than Google DNS. Both are generally faster than your ISP's DNS.