Home energy monitors

[Thomas Keffer]

Has anyone done any home energy monitoring? I'd love to correlate the weather with electric and oil consumption at my house.

I've been intrigued with the work of the OpenEnergyMonitor project in the UK. They offer Arduino kits for monitoring energy use, and for interfacing with a PC. It looks like it would be very straightforward to interface with the weewx database. 

But, even better would be an RPi kit. 

Another route is a commercial unit, such as Current Costs.

Before I embark on a Mad Scientist experiment of my own, I was wondering if anyone else has covered this ground.

-tk

--

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

[Devonian]

I have seen some Arduino implementations using similar devices to this

http://www.freetronics.com/pages/power-monitoring-with-clipsal-cent-a-meter-and-arduino

and this

 http://www.practicalarduino.com/projects/weather-station-receiver

 

 Someone did a nice write up on intercepting the radio data signals from devices such as the cent-a-meter, darned if I can find it now!

 

I have a variation of the cent-a-meter called 'OWL' and have seen it under the 'electrisave' brand name as well.

 

http://board.homeseer.com/showthread.php?t=120301&highlight=electrisave
http://board.homeseer.com/

 

Too many ideas and not enough time...

 

Nigel.

[pterodaktil]

I do not like the idea of ​​using a clamp meter to assess potrebyalemoy power.

In my experience an error of 20%

I used this energy monitor.

Now i use official counter with plug to  computer via CAN interface.

понедельник, 29 октября 2012 г., 14:59:24 UTC+4 пользователь Devonian написал:

[Jason Rennie]

On Sat, Oct 27, 2012 at 8:03 PM, Thomas Keffer <tke...@gmail.com> wrote:

Has anyone done any home energy monitoring? I'd love to correlate the weather with electric and oil consumption at my house.

FWIW, I do this manually.  I use the monthly summaries to calculate HDD and/or CDD between refills/meter-checks and use a spreadsheet to calculate things like kWh/CDD and gallons/100-HDD.  I'll send you a link.  I'm interested to know what you end up with---I would certainly like to make my "system" more automated :)

Jason

[Thomas Keffer]

I do something similar, but unless I'm prepared to go out in the rain a couple times a day to measure the amount of oil in the tank, it's hard to correlate weather with consumption!

-tk

--
 
 

[pterodaktil]

I have data with energy and weather. For my flat (central heating) ther is no corellation. 

I made it  like in this article.

Sorry for russian link

понедельник, 29 октября 2012 г., 17:15:56 UTC+4 пользователь Tom Keffer написал:

[Thomas Keffer]

Google Translate actually did a pretty decent job of translating to English! It was easy to understand.

That is a very cool project!

Alas, my oil heater does not have a CAN interface, so I'll have to do something more basic. What I'm thinking of is just monitoring when the oil valve opens. If I know the flow rate (liters/hour), then I can calculate the amount of fuel burned.

I am surprised that you found no correlation between consumption and weather. Any theories why? In my house I suspect there will be a very strong correlation.

-tk

[pterodaktil]

Great that you can understand it.

China energy meter device with clamp meter have very big error.

So i recomend try to test oficialy used devices (but set it after conroll device).

Ther was some specific for russia fun with  energy  quality. when i  prove that qualitly of  energy is no  standart.

In my home  there is central hearting ( there are over 100 flat's in building).  I  use eleictrisity only with conditioner and vent. 

понедельник, 29 октября 2012 г., 18:02:25 UTC+4 пользователь Tom Keffer написал:

[Brandon Peterson]

I recently bought a TED 5000 (www.theenergydetective.com). It does have an API to pull data from it so that might work to integrate Weewx. See http://files.theenergydetective.com/docs/TED5000-API-R330.pdf. That's an older version but I couldn't find a newer one. The latest is R499.

The other projects mentioned here sound neat too... some of them appear to have a slightly more up to date web interface versus the TED 5000.

My initial thought is to add a few fields to the Weewx database and pull the data in through the API after receiving it from the weather station. Maybe kWH and voltage for the time period? I'm not sure what else would be useful to have in Weewx. It can calculate cost and CO2 emissions too so I suppose those might be candidates but I'm not sure if they are available in the API.

Another thing to think about is that you can have multiple measuring devices so that might let you customize what parts of your electric load were pulled in to Weewx - ie if you have a dedicated measuring device on your water heater or something like that.

Brandon

[Brandon Peterson]

For what it's worth I made a quick script to pull the data down. I haven't put it in the database yet but I was thinking of doing that and trying to get it onto a chart against the outside temperature. In my case I only have a single measuring device so there's only one record per minute - otherwise there would be one per measuring device.

Sample data:

<History>

<MINUTE><MTU>0</MTU><DATE>10/29/2012 14:27:00</DATE><POWER>958</POWER><COST>12</COST><VOLTAGE>1249</VOLTAGE></MINUTE>

<MINUTE><MTU>0</MTU><DATE>10/29/2012 14:26:00</DATE><POWER>933</POWER><COST>11</COST><VOLTAGE>1248</VOLTAGE></MINUTE>

<MINUTE><MTU>0</MTU><DATE>10/29/2012 14:25:00</DATE><POWER>932</POWER><COST>11</COST><VOLTAGE>1249</VOLTAGE></MINUTE>

<MINUTE><MTU>0</MTU><DATE>10/29/2012 14:24:00</DATE><POWER>931</POWER><COST>11</COST><VOLTAGE>1250</VOLTAGE></MINUTE>

</History>

Script output:

10/29/2012 14:27:00 MTU 0 0.02 kWh $0.00 124.9 volts

10/29/2012 14:26:00 MTU 0 0.02 kWh $0.00 124.8 volts

10/29/2012 14:25:00 MTU 0 0.02 kWh $0.00 124.9 volts

10/29/2012 14:24:00 MTU 0 0.02 kWh $0.00 125.0 volts

Script:

#!/opt/bin/python

import urllib2

import xml.dom.minidom as minidom

file = urllib2.urlopen('http://10.208.1.149/history/minutehistory.xml?COUNT=5')

data = file.read()

file.close()

doc = minidom.parseString(data)

node = doc.documentElement

minutes = doc.getElementsByTagName('MINUTE')

for minute in minutes:

    time = minute.getElementsByTagName('DATE')[0].childNodes[0].data

    deviceId = minute.getElementsByTagName('MTU')[0].childNodes[0].data

    kWh = float(minute.getElementsByTagName('POWER')[0].childNodes[0].data) / 1000 / 60

    cost = float(minute.getElementsByTagName('COST')[0].childNodes[0].data) / 100 / 60

    volts = float(minute.getElementsByTagName('VOLTAGE')[0].childNodes[0].data) / 10

    print time + ' MTU {0} {1:.2f} kWh ${2:.2f} {3:.1f} volts'.format(deviceId, kWh, cost, volts)

[Thomas Keffer]

Sweet! I'll have to take a look at TED --- it looks pretty simple. 

I would actually put the data in a separate database. This guards different sampling rates, as well as simplifying the data acquisition. Because weewx uses late binding for both plots and templates, you can actually specify the data source for individual plot lines or template tags, They don't all have to come from weewx.sdb.

I should qualify that. Unfortunately, weewx plots only support one scale, so it would not be meaningful to put both energy usage and, say, temperature in the same plot. Maybe that should go in the new feature list!

To use the late binding... Say you want a weekly temperature plot (from the normal database) and a weekly energy plot from a custom database, called energy_sqlite. (I'm using V2.0 database abstraction here.) In the example that follows, archive_database is the abstract name used by the plots. It is bound appropriately for each image. The skin configuration file skin.conf would look like:

[[week_images]]

    ... # the normal weekly options

    [[[weektemp]]]

      archive_database = archive_sqlite # This option isn't really necessary because 

                                        # archive_database is already bound to 

                                        # archive_sqlite in weewx.conf

      [[[[outTemp]]]]

    [[[week_energy]]]

      archive_database = energy_sqlite # Override to use the energy database

      [[[[kWh]]]]    # This assume the database has column 'kWh'

       
This would produce a plot week energy.png with a plot of kWh consumption over the week. Simple!

-tk

--
 
 

[Brandon Peterson]

After upgrading to 2.0 it seems to work, see http://www.thebandit.org/~weather/. I'm not sure if that chart type is the best for this... maybe something more like the rain chart instead. Dunno, I'll have to play with it.

So far I've just got the script doing it on demand and not automatically so the chart won't really update like normal.

Thanks for the pointers!

Brandon

[Thomas Keffer]

This is great! I'm glad it's working for you. You're inspiring me to try something similar.

-tk

[Bill]

I read the energy monitor thread with interest. it would be nice to see what
folks come up with for integration with weewx.

I know I have a 'smart meter' with Wifi, but I can't access the data other
than going to the power companies web site. I need to check to see what sort
of granularity they will supply, but I don't think it is very high.

This raises the whole topic of extra data sources for integration into weewx.
I recall Tom mentioned some time back about Network performance, and I was
thinking about collecting the data from my UPS (has anyone tried this).

Maybe that is a direction to consider; splitting weewx into a more general
thing where weather is just one instance, with others possible?

/bill

[vds]

I just got my Raspberry Pi (three day delivery from MakerShed) and have been looking into what it might be able to do and stumbled across http://www.raspberrypi.org/phpBB3/viewtopic.php?f=37&t=1989&start=25

The forums thread mentions http://openenergymonitor.org/emon/ which might be worth a look....

(and the Pi looks pretty fun - got the reference Debian running on it in 5 minutes, although from the benchmarks it 'seems' like my existing Seagate Dockstars are 3x or more faster as a basic low-power weewx host)

[Thomas Keffer]

When I did Version 2 I very much wanted to create a general purpose "instrument monitoring" platform, but it started to get more and more complicated. In the extreme, you've reinvented Twisted.

In the end, I chose a middle ground. The state model of V2 is hardwired, but I think it's general enough that data from other instruments can be captured. By hardwiring the model, the state machine stays simple and easy to understand.

The general model is: collect data from some data source as fast as you can until an exception of type BreakLoop is thrown. This is the opportunity to do "out of band" interactions with the data source (for example, download archive data, or set parameters). Then go back to collecting data.

Such a simple model does not allow the engine to act as, say, a webserver, but so far it covers most kinds of hardware.

-tk

--

[Bill]

Please don't reinvent Twisted, one of those is more than enough :-)

From your description is sounds like there is enough. I guess I will have to
wait and see once I have time to try some other device.

/bill

[Brandon Peterson]

Hey Tom,

One thing I've found out this past week that might be important to you is the TED 5000 outdoor device seems to stop working after it gets below about 20 F outside. The specs seem to only promise down to 32 F so depending on your climate it might be a problem. I'm planning to experiment with putting the outdoor unit inside a sock or something to see if that helps so I'll let you know. In my case it's located ~75 feet away on a power pole with my meter - if your main power panel is attached to your house you might be able to have it inside and just snake the wires out to the panel so that might not be a problem for you.

I've got a question about the html pages... is there a way to bring in the current or last 30 minutes (aggregated) usage into them versus just in a chart? The chart is great but I've been thinking of adding the current usage to the mobile page in particular.

My script for pulling the data has been pretty stable so far so if there's interest I can post it on my website. It's probably not ideal but until I have time to really spruce it up it seems to be doing the job.

Thanks!

Brandon

[Thomas Keffer]

If it's in the database, it can be brought into either the chart or the statistics. Assuming the current power usage is in the database under SQL key 'powerusage', you can access it as

  $current.powerusage

Other things are accessible in a similar manner:

  Average dailly power consumption for the month: $month.powerusage.avg

  Daily max usage for each day of the month:

  #for $day in $month.days

    $day.powerusage.max

etc.

Now, the formatting routines will have no idea how to format and label this, so you'll have to add the key (i.e., powerusage) to the appropriate sections in your skin.conf file.

-tk

Brandon

--
 
 

[Mark F]

I have and use a Wattnode (http://www.ccontrolsys.com/w/WattNode_Modbus), for which I've written a Ruby script to poll every 5 minutes and dump into a MySQL database.  It does use current transformers, but I have found it to be quite accurate vs. the city's meter.  I got the MODBUS one, and use a RS485 bus to talk back to my Mac Mini (which also is the host running weewx) in my basement.  I'd recommend it.  The company is great to work with, and the price point is pretty good.

I definitely recommend doing energy monitoring of some sort.  It's useful to pinpoint times when energy usage is higher than expected ("why did we use 8kW for 10 minutes at 4am?!").  It's also nice to sanity check the city's meter.   I have a Rails app that I wrote which acts as the front end to the MySQL database and provides pages with pretty charts (HighCharts) and nightly summary emails about the weather, energy, solar generation, water usage, rainwater collection, etc. in the house.  It's fun more than anything, but it does have some utility.

[Thomas Keffer]

That sounds like a very impressive setup, Mark! Is any of it online?

-tk

[Thomas Keffer]

By the way, how do you get the RS485 signal into the PC?

-tk

On Tue, Dec 4, 2012 at 7:56 AM, Mark F <mfl...@gmail.com> wrote:

[Mark F]

For the RS485 bus, I have the USB converter seen here.  It uses a CP2102 chip and is pretty well supported.  I don't recall having much trouble getting it working. 

I have emailed you directly with information on the site.. I'd prefer to keep it from being too publicly facing, mostly for security and system load reasons.

[Thomas Keffer]

Welcome, and glad you found us!

Great tips! I'll check out the brultech devices. I've more-or-less decided on an Arduino board to monitor the opening and closing of the valve to my oil furnace --- by far the biggest energy and $$ sink in the house --- but I'd like to monitor electrical usage as well. I was not aware of the Brultech devices. I'll take a look!

And, don't worry about weewx becoming a home energy monitor! That wasn't my plan. I was planning on using "something else" to get the data into a SQL database. At worse, I would run a weewx service to grab a data point via a RESTful service and stuff it into a (separate) sqlite datebase. Then I wanted to use the existing weewx reporting engine to graph it. 

As for grabbing data from weewx, right now that's done through the database. I have been working with a couple of users on a way to publish the database elsewhere using RESTful protocols. I'll also take a look at the protocols you mention for uploading elsewhere.

Thanks again for the EM tips!

-tk

On Tue, Dec 4, 2012 at 6:53 PM, mwall <goo...@lancet.mit.edu> wrote:

tom,

please do not make weewx try to be another thingspeak (or pachube or cosm or openenergymonitor or ...)

make it do weather, and do weather well.

provide an api so that another monitor/script can grab data from weewx and stuff it into oem or thingspeak or whatever.

i just discovered your weewx project (it was mentioned in the wview discussion group).  i have been frustrated with wview for the same reasons you enumerated, and i was just getting frustrated enough to start a weather monitor in python when i discovered weewx.  so thank you for saving me a lot of work :)

we use brultech devices (ecm-1240, gem) to monitor power.  they are very accurate and let you monitor many circuits (the ecm-1240 does 7, the gem does 32).  the gem also monitors 8 one-wire devices (e.g. temperature sensors).  they also monitor pulse counters (one on the ecm-1240, or 4 on the gem). brultech has the lowest per-circuit cost of any device on the market, including the build-it-yourself stuff from open energy monitor or smart energy groups.

btw, you might want to make an uploader for weewx that sends data to smart energy groups (www.smartenergygroups.com). sam at seg has done brilliant work in creating widgets to visualize, compare, and manage data. uploading is trivial (btmon.py and ecmread.py have some python code to do it, as well as code to upload to thingspeak, pachube, oem, etc).

we have a couple of ted5000 devices, but they are not as accurate as the brultech devices. the ted5000 also tends to be rather flaky - data is transmitted over the power lines, and if you use in it older buildings (ours were built in the mid-1800s and early 1900s) you will probably have issues.  and forget about fail-safety - you cannot put a ted on an ups, so forget about dealing well with power outages and switching to/from generators and mains.

http://lancet.mit.edu/mwall/projects/power

http://lancet.mit.edu/mwall/projects/weather

--
 
 

[Ralph Underwood]

I'm wondering if you have any advice regarding the Brultech GEM energy monitor. I have two WeeWX stations up and working now. I want to add energy monitoring (several circuits) and also add several more temperature sensors. I am considering the one wire for my additional temps.

Thanks, Ralph

[Mark F]

I use a Brultech GEM.  It's a solid device with a slightly questionable interface.  However, once configured, you just set it up to report to your server at the desired interval, and you never have to interact with it.  Mine has been up for a couple of months successfully, and I have a friend who has had his up for longer.  No complaints.  Installation was simple.  I have the WiFi GEM.  Any questions, let me know.

[Thomas Keffer]

I have a Brultech GEM as well and briefly had it running using weewx. It ran using a special driver and service. Here's a GitHub repository with the code. NB: it's a work in progress and, even at that, I have not looked at it for over a year. I just haven't had the time.

Meanwhile, Matthew is the expert on these systems. Take a look at this thread.

-tk

--
You received this message because you are subscribed to the Google Groups "Weewx user's group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ralph Underwood]

Thanks for the response and link.

[mwall]

ralph,

the problem with using a single instance of weewx to collect from many different types of hardware is the timing of data collection.  for example, in weewx, 'live' (loop) data are collected as fast as a weather station can produce them.  but if you collect from a weather station and a power monitor, you now have two different 'preferred' collection periods, for example 2.5 seconds for a vantage weather station and 10 seconds for a GEM.  if that is ok, then use weewx with a service for each additional type of hardware.  if not, then you can use a weewx instance for each hardware device.  and if that is too much overhead, then use weewx for the weather station, btmon for the GEM, etc.

another issue is the data storage.  it is probably a bad idea to extend your weewx database schema to include power data.  that might be ok for whole-house monitoring, but if you include all 32 channels from a GEM you'll want to put that into a separate table or database.

we use btmon to collect data from the GEM and weewx to collect weather data, then we feed everything into emoncms.  in some cases we also feed to SEG and Xively, and we typically use nagios to monitor the systems and send notifications when something goes down.

on larger systems (data coming from weather station, internet-capable thermostats, internet-capable alarm system, many one-wire sensors) we tend to use one collector per device.  for example, one instance of btmon for each GEM, rtmon for radio-thermostats, owmon for one-wire, weewx for weather.

on simple installations we go with weewx plus a service for each additional device, for example weewx for weather with owfss for one-wire data.

imho, there is not (yet) a one-size-fits-all approach to the data collection, storage, display, or analysis.  and since this stuff is changing rapidly (compare ecm1240 with ted500 with GEM, not to mention the industrial equivalents), having open hardware and toolchain is key.

here are some notes:

http://lancet.mit.edu/mwall/projects/power

http://lancet.mit.edu/mwall/projects/weather

trystan has made huge progress with emoncms in the past few years.  we use emoncms to display a floorplan of the premises overlaid with weather data, power data, camera feeds, and links to the historical data.  but we still use the output from weewx directly for weather-oriented dashboards.

m

[Ralph Underwood]

Thanks, that gives me a lot of info to research.  We have a 3500 square foot shop with a radiant heated concrete floor.  We are exploring adding a solar hot water system into the floor system.  Using WeeWx to monitor weather conditions and probably the various solar system temps. The electric boiler has two 60 amp 240V circuits. I want to be able to view reductions in electric over time. I also want to modify the habits sod some shop users who leave 1200 watts of lighting on when they are not needed.

I have been involved with electric utility SCADA and EMS systems for 20 years and so making a small scale system out of open source stuff is great fun.

Thanks again, Ralph

[Daniel Rich]

Ralph,

If you want good power monitoring, take a look at Brultech's GreenEye (http://brultech.com/). I installed one about a year ago and have been making quite a bit of progress in reducing my electricity use now that I now exactly what I'm using. It's a bit of a pita to install since you have to put cores on every wire in your breaker panel, but it gives you real-time electrical usage on a per-circuit level.

Integrating it with WeeWX might be an interesting project, I'll have to think about that one. Currently I'm forwarding all of my data to SmartEnergyGroups.com and letting them deal with turning it into pretty pictures. :-)

Admittedly, I've only been running WeeWX since Sunday (converted over from wview this weekend) so I'm still a novice as to what I can do with it!
  http://www.lapseofthought.com/Weather/

[Ralph Underwood]

Daniel,

I see two units the GreenEye and the ECM-1240. Is the GEM the one to use to interface via USB?

Ralph

[mwall]

ralph,

the ecm-1240 has a serial interface.  you can talk to it using a usb-serial adapter.

the gem has a serial interface (two, in fact).  it also has options for ethernet or wireless.  imho, the ethernet interface is the most reliable and easiest to use.

m

[Thomas Keffer]

My GEM has a WiFi interface and I've found it reliable enough.

However, the directions on how to set it up are terrible. In fact, all the documentation for the GEM is pretty bad. Lots of mysterious buttons and boxes in the setup utility, scant explanation on the difference between "server" mode and "client" mode and why you'd want to use one over the other, and so on.

Expect a steep learning curve.

OTOH, it's a very capable device, well made and flexible.

-tk

--

[Ralph Underwood]

Thanks, I think I will order one and start climbing that steep curve. I am thinking now of opting for the ethernet option.

[Thomas Keffer]

I don't know if they still offer it, but mine can switch hit: it has both an ethernet port, and WiFi. No need to choose, at least now!

-tk

[drich]

The GEM is the GreenEye -- sorry, I only speak in TLAs... :-)

Avoid the dual ethernet/WiFi module, it can be a nightmare to setup, especially if you don't have a Windows system to configure it with.

I don't have mine talking to WeeWx though, I send everything (including my WeeWX data) off to SEG (SmartEnergyGroups.com). At some point I plan on using their API to pull the data back down so I can potentially build some code into my home automation system (Insteon / ISY994i) to do things like turn on ceiling fans when it gets hot in the house, tweak my power usage on hot days, and/or adjust my lawn/garden watering schedule based on temperature.

 

---
Dan Rich <dr...@employees.org> |   http://www.employees.org/~drich/
                               |  "Step up to red alert!"  "Are you sure, sir?
                               |   It means changing the bulb in the sign..."
                               |          - Red Dwarf (BBC)

[tds]

Speaking of SmartMeter, our utility company offers the option to connect to the SmartMeter via ZigBee protocol and get "near real-time" data (15-60 sec delay, according to them).  Has anyone tried to integrate devices like the Rainforest Eagle (http://rainforestautomation.com/eagle) to weewx?

--tds

[Andrew Creahan]

Hi,

I've had some success with the OpenEnergyMonitor via Arduino.  Just monitoring the basic stuff and still work in progress, but here's the data presented so far:

 http://weather.blognz.org/emon3b.html

Happy to write up when I get some time if anyone's interested.

[Thomas Keffer]

You keep a cold home!

-tk

--

[Andrew Creahan]

Temp gauge is currently in the garage while I'm testing the system - Heat pumps kick in when the true indoor temp drops below 22C, so can see the power usage spike when they activate.

[Lloyd Adams]

I implemented a monitoring and control system a couple of years ago using arduinos and openenergy.org when we had our solar panels installed. Very versatile and well supported. The site is here http://www.vanillapig.co.uk/energy/, but I've never bothered to combine it with my weather station (which is also arduino based) (http://cople.linkpc.net/weewx)