My struggle with mongoDB and the solutions I found.

[Sjoerd van Staveren]

Hello Node-RED users,

I'm new to node-red and I'm really enthusiastic about it. I have been trying all kinds of things over the last weeks. The biggest struggle I had was to find information on how to use this mongoDB node. Not as much as how to enter data in a collection as well how to get the data out in the way I want. It took me several days to find things out and I would like to share my results so that others don't have to search as long as I did. The biggest problem is that the mogoDB syntax you find overhere (https://docs.mongodb.com/) is not exactly the same as the syntax Node-RED uses. One difference is the use of quote signs (" and ') but there are other subtle differences. 

The collection in my database contains records that are coming directly from the wunderground node. This node gives the actual weather data for  a given location. It is great node to make local weather dashboard. like this:

 This is an example of one record from wundergound as it is stored in a mongDB collection:

0: object 

_msgid: "ce7a3da9.d1435"

topic: ""

payload: object

location: object

data: object

response: object

current_observation: object

image: object

display_location: object

observation_location: object

estimated: object

station_id: "ILIMBURG77"

observation_time: "Last Updated on July 19, 12:46 PM CEST"

observation_time_rfc822: "Wed, 19 Jul 2017 12:46:40 +0200"

observation_epoch: "1500461200"

local_time_rfc822: "Wed, 19 Jul 2017 12:46:42 +0200"

local_epoch: "1500461202"

local_tz_short: "CEST"

local_tz_long: "Europe/Amsterdam"

local_tz_offset: "+0200"

weather: "Clear"

temperature_string: "88.2 F (31.2 C)"

temp_f: 88.2

temp_c: 31.2

relative_humidity: "37%"

wind_string: "From the WSW at 2.0 MPH Gusting to 2.5 MPH"

wind_dir: "WSW"

wind_degrees: 249

wind_mph: 2

wind_gust_mph: "2.5"

wind_kph: 3.2

wind_gust_kph: "4.0"

pressure_mb: "1010"

pressure_in: "29.83"

pressure_trend: "0"

dewpoint_string: "59 F (15 C)"

dewpoint_f: 59

dewpoint_c: 15

heat_index_string: "87 F (31 C)"

heat_index_f: 87

heat_index_c: 31

windchill_string: "NA"

windchill_f: "NA"

windchill_c: "NA"

feelslike_string: "87 F (31 C)"

feelslike_f: "87"

feelslike_c: "31"

visibility_mi: "N/A"

visibility_km: "N/A"

solarradiation: "665"

UV: "4.0"

precip_1hr_string: "0.00 in ( 0 mm)"

precip_1hr_in: "0.00"

precip_1hr_metric: " 0"

precip_today_string: "0.00 in (0 mm)"

precip_today_in: "0.00"

precip_today_metric: "0"

icon: "clear"

icon_url: "http://icons.wxug.com/i/c/k/clear.gif"

forecast_url: "http://www.wunderground.com/global/stations/06391.html"

history_url: "http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=ILIMBURG77"

ob_url: "http://www.wunderground.com/cgi-bin/findweather/getForecast?query=51.492737,6.120970"

nowcast: ""

forecast: object

time: "2017-07-19T10:46:40.000Z"

title: "Data supplied by The Weather Underground."

description: "Current weather information at coordinates: 51.49, 6.11"

_id: "596f38924bdbfc08ed3297b5"

This is the query I use to return only the temperature (F en C)  between two given dates. In the comments I show which SQL statement compares to the Node-RED statements.

/*

Data manipulation to get the correct datatype for the msg.payload.

*/

var start = Date.parse("2017-07-20 00:00:00").toString(); // Date "from"

var stop = Date.parse("2017-07-20 12:00:00").toString();  // Date "to"

//-------------------------- "SQL" part ------------------------------

//SELECT statement

msg.projection = {  'data.current_observation.temp_c' : 1, // 1 returns the field, 0 suppresses the field

                    'data.current_observation.temp_f' : 1

};

//SELECT TOP statement

msg.limit = 10;  // returns the first 10 records

//WHERE statement

msg.payload = {'data.current_observation.local_epoch' :

        {

            "$gte" : start, // Larger or equal to

            "$lte" : stop   // Smaller or equal to

        }

};

//ORDER BY statement

msg.sort = {'data.current_observation.local_epoch' : 1}; // 1 = ASC, -1 = DESC

//-----------------------------------------------------------------------

return msg;

I hope this is of any use for the group.

Regards,.

Sjoerd van Staveren

[David Dempster]

Hi Sjoerd,

I've been trying to do a similar task - like most people I want to log temperature and humidity to a database, and then plot sections of data to a graph (to see what my house is doing).

So far I've tried using mySQL and now for the last 3 days I've been learning about MongoDB.

Your write-up will certainly help me to get to grips with MongoDB.

The other "frustation" I've had is with the node "google chart" - i.e. Chart Request and Chart Response.

On the face of it this looks like a really neat way to communicate from Node-Red to Google Chart API and back to Node-Red.

Have you used this to plot graphs?

Cheers from David.

[Julian Knight]

You might like to note that MongoDB does not play nicely with 32bit OS's including the Raspberry Pi. It has a nasty tendency to crash and corrupt the db well before the theoretical file size limit is reached.

For this reason, I switched to InfluxDB making use of its ability to auto-summarise data. So I keep detailed (1 minute) data for 7 days and accumulate average hourly data to keep forever.

The other advantage to InfluxDB is the ability to create really nice dashboards directly from it by using Grafana. I am able to run Node-RED, Mosquitto, InfluxDB, Telegraf and Grafana all on a Pi2.

[Sjoerd van Staveren]

Hi David,

No I haven't managed to do that yet. I can't find the proper way to feed data into the graph node. I just posted a request for help on that.

Regards,

Sjoerd

[GlennK]

I didn't realize there was a Weather Underground node, so I had written my own interface to monitor a weather station located on our farm/ranch from anywhere.  Weather Underground has a website which gives you current conditions with some historical graphing.  But, one key piece of data that I was needing a graph of the UV light instrument.  But Weather Underground does not provide that, so time to roll your own solution.

I basically use the "http in" node to return a parsed JSON object containing current conditions for the specific weather station.  Pass it to the ''json" node to convert to a Javascript object.  Then a "function" node can pick out the specific values, so for example the current temperature in Farenheit is returned from Weather Underground as "temp_f"..

var response = msg.payload;
var responseObj = JSON.parse(response);
msg.payload=responseObj.current_observation.temp_f;

Then pass the value on to dashboard nodes and you have the following simple dashboard with some 10 minute data points graphed for 12 hours.  All within Node-Red.

[Jan Van den Audenaerde]

I agree with Julian's comment regarding mongodb on pi. It also got corrupted in my case after several months logging 5 integers with timestamp every 10 sec in a collection. So if you plan to store quite some data then I would not use a mongodb.
Kr jan