rflog_db.py no humidity readings in the database.

49 views
Skip to first unread message

shawn logan

unread,
Feb 14, 2020, 8:40:02 AM2/14/20
to PrivateEyePi

I followed this http://projects.privateeyepi.com/home/create-a-database-logger
I am running 10 rf temp & temp w/humidity sensors from jemrf. 
I am getting battery and temp data to my database but no humidity readings (type 4).
Anyone else see this issue, how do I correct?
Below is a snippet from the database.:

  -> |    292 | 60        |    3 | 70.61  | 2020-02-14 06:31:38 | F               |
    -> |    293 | 15        |    3 | 71.6   | 2020-02-14 06:35:29 | F               |
    -> |    294 | 60        |    3 | 70.61  | 2020-02-14 06:36:50 | F               |
    -> |    295 | 18        |    3 | 72.23  | 2020-02-14 06:39:12 | F               |
    -> |    296 | 18        |    2 | 2.40   | 2020-02-14 06:39:12 |                 |
    -> |    297 | 21        |    3 | -16.42 | 2020-02-14 06:41:26 | F               |
    -> |    298 | 21        |    2 | 2.29   | 2020-02-14 06:41:26 |                 |
    -> |    299 | 10        |    3 | 70.83  | 2020-02-14 06:43:41 | F               |
    -> |    300 | 18        |    3 | 72.36  | 2020-02-14 06:44:25 | F               |
    -> |    301 | 21        |    3 | -16.42 | 2020-02-14 06:46:39 | F               |
    -> |    302 | 60        |    3 | 70.61  | 2020-02-14 06:47:16 | F               |
    -> |    303 | 11        |    3 | 70.23  | 2020-02-14 06:47:45 | F               |
    -> |    304 | 10        |    3 | 70.88  | 2020-02-14 06:48:54 | F               |
    -> |    305 | 18        |    3 | 72.63  | 2020-02-14 06:49:38 | F               |
    -> |    306 | 60        |    3 | 70.61  | 2020-02-14 06:52:28 | F               |
    -> |    307 | 11        |    3 | 70.27  | 2020-02-14 06:52:57 | F               |
    -> |    308 | 62        |    3 | 70.05  | 2020-02-14 06:53:25 | F               |
    -> |    309 | 18        |    3 | 72.99  | 2020-02-14 06:54:50 | F               |
    -> |    310 | 11        |    3 | 70.5   | 2020-02-14 06:58:10 | F               |
    -> |    311 | 11        |    3 | 70.5   | 2020-02-14 06:58:10 | F               |
    -> |    312 | 62        |    3 | 70.34  | 2020-02-14 06:58:37 | F               |
    -> |    313 | 10        |    3 | 70.63  | 2020-02-14 06:59:20 | F               |
    -> |    314 | 10        |    3 | 70.63  | 2020-02-14 06:59:20 | F               |
    -> |    315 | 23        |    3 | 83.26  | 2020-02-14 07:00:44 | F               |
    -> |    316 | 15        |    3 | 71.65  | 2020-02-14 07:01:33 | F               |
    -> |    317 | 21        |    3 | -16.42 | 2020-02-14 07:02:17 | F               |
    -> |    318 | 60        |    3 | 70.88  | 2020-02-14 07:02:54 | F               |
    -> |    319 | 10        |    3 | 70.57  | 2020-02-14 07:04:32 | F               |
    -> |    320 | 11        |    3 | 70.27  | 2020-02-14 07:13:49 | F               |
    -> |    321 | 62        |    3 | 70.34  | 2020-02-14 07:14:15 | F               |
    -> |    322 | 62        |    2 | 3.00   | 2020-02-14 07:14:15 |                 |
    -> |    323 | 10        |    3 | 70.47  | 2020-02-14 07:14:58 | F               |
    -> |    324 | 21        |    3 | -16.42 | 2020-02-14 07:17:55 | F               |
    -> |    325 | 60        |    3 | 70.88  | 2020-02-14 07:18:32 | F               |
    -> |    326 | 18        |    3 | 73.08  | 2020-02-14 07:26:07 | F               |
    -> |    327 | 21        |    3 | -16.42 | 2020-02-14 07:28:20 | F               |
    -> |    328 | 60        |    3 | 71.15  | 2020-02-14 07:28:57 | F               |
    -> |    329 | 11        |    3 | 70.05  | 2020-02-14 07:29:27 | F               |
    -> |    330 | 62        |    3 | 70.34  | 2020-02-14 07:29:53 | F               |
    -> |    331 | 21        |    3 | -16.42 | 2020-02-14 07:33:33 | F               |
    -> |    332 | 21        |    2 | 2.29   | 2020-02-14 07:33:33 |                 |
    -> |    333 | 60        |    3 | 70.88  | 2020-02-14 07:34:10 | F               |
    -> |    334 | 11        |    3 | 70.14  | 2020-02-14 07:34:40 | F               |
    -> |    335 | 62        |    3 | 70.05  | 2020-02-14 07:35:06 | F               |


Gadjet Nut

unread,
Feb 15, 2020, 11:46:15 PM2/15/20
to shawn logan, PrivateEyePi
The temperature and humidity sensor sends humidity first and then temperature. This code relies on temperature being sent first:

if tempbdata<>"" and sensordata<>"": 

Probably best to download the latest rfsensor.py and modify it to log to local database.  

--
You received this message because you are subscribed to the Google Groups "PrivateEyePi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to privateeyepi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/privateeyepi/f1b4a0da-7a34-4054-aed3-b4ae8c2eb7bf%40googlegroups.com.

shawn logan

unread,
Feb 17, 2020, 11:57:26 AM2/17/20
to PrivateEyePi
I need a bit more help then that. i purchased these sensors as directed by privateeyepi from jemrf (Are these your companys Gadjet Nut?). 
The fact that privateeyepi limits to 1500 reads(i paid for the unlimited database while it was offered.), i am hitting 1500read by 10am.   

I was following  http://projects.privateeyepi.com/home/create-a-database-logger. & http://projects.privateeyepi.com/home/home-alarm-system-project/wireless-projects/download-and-run-the-wireless-software

"You use a Python program called rflog_db.py to log the telemetry readings to the database. You can download the software using the following commands:

$ cd ~

git clone git://github.com/gadjetnut/Data-logging

$ cd Data-logging

Run the program:

$ python rflog_db.py"



Now you are saying not to follow the above instructions. and  do this "Probably best to download the latest rfsensor.py and modify it to log to local database.  "
I am going need a little more direction, or better yet just update the instructiions on : http://projects.privateeyepi.com/

pi@RfTempsPi:~/Data-logging $ cat  rfsensor.py
#!/usr/bin/env python
"""
rfsensor.py v21 PrivateEyePi RF Sensor Interface
---------------------------------------------------------------------------------
 Works conjunction with host at www.privateeyepi.com
 Visit projects.privateeyepi.com for full details

 J. Evans October 2013
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
 WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
 CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

 Revision History
 V1.00 - Release
 V2.00 - Incorporation of rules functionality
 V3.00 - Incorporated Button B logic
 V3.01 - High CPU utilization fixed
 V9    - Rule release
 V10   - Added support for the BETA single button power saving wireless switch
     - Functionality added for wireless temperature and humidity sensor
 V11   - Fixed a bug with negative readings from a DHT22 sensor
 V13   - Publish temperature to LCD
 V14   - Added auto sensor creation on the server, dropped support for obsolete two button sensors
 V15   - Added token based authentication
 V16   - Removed delay to speed up serial polling
 V17   - Added functionality for Light sensors
 V18   - Fixed bug wireles switch BUTTONON and BUTTONOFF sensing same state
 V20   - Changed STATEON STATEOFF to not trigger rules or log on the server
 V21   - Added support for BME280 Temperature, Humidity and Air Pressure
 -----------------------------------------------------------------------------------
"""

import globals
import time
import sys
from threading import Thread
from alarmfunctionsr import UpdateHostThread
from time import sleep
from bme280 import process_bme_reading
from rf2serial import rf2serial
import rfsettings

def dprint(message):
  if (globals.PrintToScreen):
    print message

def ProcessMessage(value, DevId, PEPFunction):
  global measure

  # Notify the host that there is new data from a sensor (e.g. door open)

  hostdata =[]
  hostdata.append(DevId)
  hostdata.append(value)
  if PEPFunction==22: #Battery
      MaxVoltage=3
      for z in range (0,len(globals.VoltageList)):
        if globals.VoltageList[z] == int(DevId):
          MaxVoltage=globals.MaxVoltage[z]
      hostdata.append(MaxVoltage) #MaxVoltage
  if PEPFunction==37: #Temperature or Analog
      hostdata.append(measure)
  rt=UpdateHostThread(PEPFunction,hostdata)

  return(0)

def DoFahrenheitConversion(value):
  global measure
  if globals.Farenheit:
    value = float(value)*1.8+32
    value = round(value,2)
    measure = '1'
  else:
    measure='0'
  return(value)

def remove_duplicates():
    x=0
    while x<len(rfsettings.message_queue)-1:
        if rfsettings.message_queue[x]==rfsettings.message_queue[x+1]:
            rfsettings.message_queue.pop(x)
        else:
            x=x+1

def queue_processing():
  try:
    sensordata=""
    bme_data=""
    bme_messages=0
    while (True):
        if len(rfsettings.message_queue)>0 and not rfsettings.rf_event.is_set():
            remove_duplicates()
            message = rfsettings.message_queue.pop()
            devID = message[0:2]
            data = message[2:12]
            dprint(time.strftime("%c")+ " " + message)
            if data.startswith('BUTTONON'):
                devID=globals.BUTTONPrefix+devID
                sensordata=0
                PEPFunction=26

            if data.startswith('STATEON'):
                devID=globals.BUTTONPrefix+devID
                sensordata=0
                PEPFunction=38

            if data.startswith('STATEOFF'):
                devID=globals.BUTTONPrefix+devID
                sensordata=1
                PEPFunction=38

            if data.startswith('BUTTONOFF'):
                sensordata=1
                PEPFunction=26

            if data.startswith('TMPA'):
                sensordata=DoFahrenheitConversion(str(data[4:].rstrip("-")))
                PEPFunction=37

            if data.startswith('ANAA'):
                sensordata=str(data[4:].rstrip("-"))
                sensordata=(float(sensordata)-1470)/16 #convert it to a reading between 1(light) and 48 (dark)
                sensordata=str(sensordata)
                PEPFunction=37
                measure='2'

            if data.startswith('ANAB'):
                devID=globals.ANABPrefix+devID
                sensordata=str(data[4:].rstrip("-"))
                sensordata=(float(sensordata)-1470)/16 #convert it to a reading between 1(light) and 48 (dark)
                sensordata=str(sensordata)
                measure='2'
                PEPFunction=37

            if data.startswith('TMPC'):
                devID=globals.TMPCPrefix+devID
                sensordata=DoFahrenheitConversion(str(data[4:].rstrip("-")))
                PEPFunction=37

            if data.startswith('TMPB'):
                devID=globals.TMPBPrefix+devID
                sensordata=DoFahrenheitConversion(str(data[4:].rstrip("-")))
                PEPFunction=37

            if data.startswith('HUM'):
                devID=globals.HUMPrefix+devID
                sensordata=str(data[3:].rstrip("-"))
                PEPFunction=37
                measure='2'

            if data.startswith('BATT'):
                sensordata=data[4:].strip('-')
                PEPFunction=22

            if data.startswith('B') and sensordata=='':
              bme_data=bme_data+data[1:9]
              bme_messages=bme_messages+1
              if bme_messages==5:
                bme280=process_bme_reading(bme_data, devID)
                if bme280.error <> "":
                  dprint(bme280.error)
                else:
                  if bme280.temp_rt == 1:
                    ProcessMessage(DoFahrenheitConversion(round(bme280.temp,2)), devID, 37)
                  if bme280.hum_rt == 1:
                    measure='2'
                    ProcessMessage(round(bme280.hum,2), globals.HUMPrefix+devID, 37)
                  if bme280.hum_rt == 1:
                    measure='2'
                    ProcessMessage(round(bme280.press/100,1), globals.PRESPrefix+devID, 37)
                bme_messages=0;
                bme_data=""
            if sensordata <> "":
                ProcessMessage(sensordata, devID, PEPFunction)
        sensordata=""

        if rfsettings.event.is_set():
            break

  except Exception as e:
      template = "An exception of type {0} occurred. Arguments:\n{1!r}"
      message = template.format(type(e).__name__, e.args)
      print message
      print e
      rfsettings.event.set()
      exit()

def main():
    globals.init()
    rfsettings.init()

    a=Thread(target=rf2serial, args=())
    a.start()

    b=Thread(target=queue_processing, args=())
    b.start()

    while not rfsettings.event.is_set():
      try:
          sleep(1)
      except KeyboardInterrupt:
          rfsettings.event.set()
          break

if __name__ == "__main__":
    try:
      main()
    except Exception as e:
      template = "An exception of type {0} occurred. Arguments:\n{1!r}"
      message = template.format(type(e).__name__, e.args)
      print message
      print e
      rfsettings.event.set()
    finally:
      rfsettings.event.set()
      exit()
To unsubscribe from this group and stop receiving emails from it, send an email to privat...@googlegroups.com.

Gadjet Nut

unread,
Feb 21, 2020, 1:04:16 PM2/21/20
to PrivateEyePi
Yes PrivateEyePI and JemRF are both mine. Two separate entities tho. JemRF is for profit and PrivateEyePi is free. JemRF isnt tied to PrivateEyePi, you can use the JemRF sensors with many different logging systems:


We can update the logging tutorial no problem. Sorry I assumed you were OK with Python and programming but understand my direction may have been too high level. 

Gadjet Nut

unread,
Feb 21, 2020, 4:44:35 PM2/21/20
to PrivateEyePi
We have updated the Github repository, but please note this is a tutorial. Its not a fully fledged logging and alerting system. It's intended to teach the basics database and website development. 

You can install by deleting your Data-logging directory or start a new directory with a fresh install:

git clone git://github.com/gadjetnut/Data-logging

The web page looks like this now. Note sensor 70 is a temperature and humidity sensor and both temp and humidity are showing up OK.

The last 10 telemetry logs are:
DateDevice IDTypeValueUOM
2020-02-21 16:18:51803-.82C
2020-02-21 16:18:3595120.9C
2020-02-21 16:18:359571155.5P
2020-02-21 16:18:3595522.45%
2020-02-21 16:18:0676521.99%
2020-02-21 16:18:0676322.08C
2020-02-21 16:17:4770523.11%
2020-02-21 16:17:4770322.62C
2020-02-21 16:17:477062.70V
2020-02-21 16:17:0807321.07C


shawn logan

unread,
Feb 22, 2020, 7:12:34 AM2/22/20
to PrivateEyePi
I truly thank you for all of your community support. I will test that new 
repository,
Reply all
Reply to author
Forward
0 new messages