Time Series and Sensor Data

[Guy Dillen]

Hello,

I would like using starting with MongoDB as my datastore for an IoT application. I'm new to MongoDB and already watched some webcasts on this topic but still have some questions about the way to store time series data in MongoDB.

1. The examples given on e.g. http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb show storing data for every second. Is the same approach/principle valid when the requirements are to store sensor data e.g. every 15 minutes or every hour? So e.g. in these cases storing data for 24 hours in 1 document?

2. Some sensors can sense at the same time different 'things' e.g. temperature and humidity. Is it best to store the temperature and humidity data in 2 separate documents in the  same collection (or both readings/values in the same document)? Additionally, it is possible the time intervals for reading the values of temperature and humidity can be different. Flexibility is also important: to have the possibility to easily add other sensor types (e.g. for pH, etc).

Many thanks.

Guy

[Kevin Adistambha]

Hi Guy,

1. The examples given on e.g. http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb show storing data for every second. Is the same approach/principle valid when the requirements are to store sensor data e.g. every 15 minutes or every hour? So e.g. in these cases storing data for 24 hours in 1 document?

Please note that the blog post was written two years ago, and so may contain outdated technical details. For example, the in-place update mentioned in the blog post (under “More Updates than Inserts” heading) is only valid for the MMAPv1 storage engine, and less relevant to the WiredTiger engine.

If your use case closely follows the case in the blog post, then the approach is still valid even if you need to store different time span. The time span involved in designing a time-series solution is highly dependent on what you need to do with the data you collected. The blog post solution assumes that you need to see a summary of data for every minute, but also need to store the detailed information every second.

1. Some sensors can sense at the same time different ‘things’ e.g. temperature and humidity. Is it best to store the temperature and humidity data in 2 separate documents in the same collection (or both readings/values in the same document)? Additionally, it is possible the time intervals for reading the values of temperature and humidity can be different.

It depends on how you want to use the data. If the temperature and humidity measurements are:

• frequently used together, and
• collected at the same time interval (e.g. every second), and
• need to be summarized within the same timeframe (e.g. every minute)

then you could combine the two in a single document, for example:

{
  timestamp: <timestamp>,
  temperature: {0:23, 1:23.5, 2:23.6, ...}
  humidity:    {0:70, 1:70,   2:70,   ...}
}

However, if temperature and humidity are:

• not frequently used together, or
• collected at different time intervals (e.g. every second for temperature, every minute for humidity), or
• need to be summarized at different timeframe (e.g. every hour for temperature, every day for humidity)

then storing temperature and humidity in two different documents may be a better fit.

Flexibility is also important: to have the possibility to easily add other sensor types (e.g. for pH, etc).

MongoDB stores data in a flexible schema, so you can easily add more measurements in your documents.

Other resources regarding schema design that you may find useful are:

• Data Models
• 6 Rules of Thumb for MongoDB Schema Design.

Best regards,
Kevin

​