P1 Smart meter - Dutch Slimme meter via Tasmota

[Martijn Hendriks]

P1 smart meter MQTT wifi gateway. 

Couple of weeks ago I start a small project to use a Wemos D1 mini as MQTT P1 gateway using Tasmota software. Initially started with EasyESP but I don't like this setup and moved to Tasmota.

At the moment the D1 Mini transmits every 5sec the most actual P1 values. Any suggestions for improvement are welcome.

See the add-on id91 in the attachment.

SETUP:

- DSMR 4+ smart meter (incl. CRC)

- OpenHab 2.0 server

- P1 signal inverter (like the P1-wifi gateway shield from http://www.esp8266thingies.nl/wp/); use a proto board and a transistor and 3 resistors).

More info:

- http://domoticx.com/p1-poort-slimme-meter-hardware/

Things to configure:

- Add xsns_91_p1gateway.ino to your project

- add user_config.h in row ~218 serial sensors  

#define USE_P1GATEWAY                              // Add support for P1 smart meter sensor (+extra code)

- en-GB.h in laguage folder below // sonoff_template.h

#define D_SENSOR_P1GATEWAY  "P1GATEWY"

- add line below in "enum UserSelectablePins" sonoff_template.h just row before GPIO_SENSOR_END };

  GPIO_P1GATEWAY,      // P1 Smartmeter interface (serial RX)

and in the same file in const char kSensorNames[] PROGMEM = before ending

"|" D_SENSOR_P1GATEWAY

- change in TasmotaSerial.h

#define TM_SERIAL_BAUDRATE           115200   // Max supported baudrate 

#define TM_SERIAL_BUFFER_SIZE        1250     // Receive buffer size default 64 ->MH:+ 2* buffersize of P1 datagram

Add new items and sitemap in OpenHab. Like examples below

home.sitemap

    Frame label="Power" {

    Text item=T1 label="KWh Teller 1: [%.1f KWh]"

    Text item=T2 label="KWh Teller 2: [%.1f KWh]"

    Text item=P label="Actueel vermogen: [%.3f KW]"

    Text item=I label="Actueel stroom: [%.1f A]"

    Text item=GAS label="Gasmeter: [%.1f m3]"

    }

        

home.items

//P1 interface

Number T1 "KWh Teller 1" {mqtt="<[mosquitto:tele/D1M_01/SENSOR:state:JSONPATH($.P1GATEWAY.T1)]"}

Number T2 "KWh Teller 2" {mqtt="<[mosquitto:tele/D1M_01/SENSOR:state:JSONPATH($.P1GATEWAY.T2)]"}

Number P "Actueel vermogen" {mqtt="<[mosquitto:tele/D1M_01/SENSOR:state:JSONPATH($.P1GATEWAY.P)]"}

Number I "Actueel stroom" {mqtt="<[mosquitto:tele/D1M_01/SENSOR:state:JSONPATH($.P1GATEWAY.L1_I)]"}

Number GAS "Gasmeter" {mqtt="<[mosquitto:tele/D1M_01/SENSOR:state:JSONPATH($.P1GATEWAY.GAS)]"}

[Menno Slingerland]

Am I the only one finding this very useful?

I did everything you said, except i don't see any values in my mqtt broker. I've flashed the modified firmware. Which module do I have to configure in the Configure Module section and what are the needed settings for that module?

I'm using the P1 smart meter wifi gateway

Thank you.

Op zondag 11 maart 2018 13:01:27 UTC+1 schreef Martijn Hendriks:

[Philip Knowles]

Have you configured the MQTT broker in Tasmota. IP address and port plus username and password and client? 5 secs is a short period I have mine set to 120 seconds (that's in logging). In Other you have the Friendly name too.

Regards

Phil

On Jun 28, 2018 2:23 PM, "Menno Slingerland" <msling...@gmail.com> wrote:

Am I the only one finding this very useful?

I did everything you said, except i don't see any values in my mqtt broker. I've flashed the modified firmware. Which module do I have to configure in the Configure Module section and what are the needed settings for that module?

--
You received this message because you are subscribed to the Google Groups "SonoffUsers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonoffusers+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/sonoffusers.
To view this discussion on the web, visit https://groups.google.com/d/msgid/sonoffusers/2a462758-5dbb-41e7-a0ae-84f4910a7101%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Menno Slingerland]

Hi Philip,

Thank you for your response.

Knipsel.png is what i'm seeing in tasmota logging.

Knipsel2.png is what i'm seeing on my incoming mqtt.

Any ideas?

Best regards

Op donderdag 28 juni 2018 18:31:29 UTC+2 schreef Philip Knowles:

To unsubscribe from this group and stop receiving emails from it, send an email to sonoffusers...@googlegroups.com.

[Menno Slingerland]

It's working now. 

For future reference: In Tasmota: Configuration -> Configure module -> GPIO3 Serial In -> Select P1GATEWY

I don't see the value for GAS.

The onlything left i don't get are the values for L1_U and Gas. What kind of values are they?

MQT: tele/sonoff/SENSOR = {"Time":"2018-06-28T19:51:06","P1GATEWAY":{"T1":002583.063,"T2":001908.910,"T1r":000000.000,"T2r":000000.000,"P":00.554,"Pr":00.000,"L1_U":G1009422819,"L1_I":003,"GAS":MX5LGBBFG10}}

Thank you Martijn for your time and effort to built this.

Op donderdag 28 juni 2018 20:14:53 UTC+2 schreef Menno Slingerland:

[Marco Verleun]

Hi Martijn,

I just noticed your addition to the Tasmota firmware and it looks nice. I've been looking for an alternative to ESPEasy as well.

Have you considered creating a pull request against the original Tasmota firmware? That way it would be integrated with future versions as well.

Regards,

Marco

Op zondag 11 maart 2018 13:01:27 UTC+1 schreef Martijn Hendriks:

P1 smart meter MQTT wifi gateway. 

[Martijn Hendriks]

You're welcome Menno.

The readouts  in L1_U (Voltage) and GAS are faulty. Same issue here.

change row 64 to:

  {"GAS","0-1:24.2.1(",26,"*m3)",}  

This fixed the issue for me.

Other strange readout is the L1_U value ... this is incorrect. Maybe another P1 datagram format.

@Marco: I will do a pull request in the near future.

Regards, Martijn

Op donderdag 28 juni 2018 20:44:34 UTC+2 schreef Menno Slingerland:

[Kees van Hoekelen]

Hi Martijn,

Like others I want to get rid of Espeasy and came across your Tasmota module. Problem is that Tasmota seems to have gotten a different structure and I can't get your module to work. When I pull an older Tasmota image, I get compile errors. 

Which version of Tasmota did you use to build? And perhaps, any clue how to get it working in the new Tasmota structure?

Some additional Info:

Looks like I got it compiled...

When not selected P1GATEWY I get errors like above and it looks like a complete and readable telegram.

When selecting P1GATEWY to gpio3 and connect to a syslog server I get ESP-P1 : serial not available 

with an occasionaly (every 5 to 10 minutes) ESP-P1read: / found folowed by ESP-P1 error: DATA corrupt, discarded input

Module worked with ESP-easy, so I know that the serial connexion is (must be) ok.

Thanks and cheers,

kees

Op zondag 11 maart 2018 13:01:27 UTC+1 schreef Martijn Hendriks:

P1 smart meter MQTT wifi gateway. 

[Martijn Hendriks]

Kees,

I build and tested this extension on 5.12.0d. I will rebuild it to the latest version when time is on my side.

Some times i get the same error. I didn't figure out the cause ... but it looks like a timing / memory issue.

The datagram is received  every second (depends on the smart meter) and only the last correct datagram is processed and sent to the mqtt broker. So the broker receives always correct data.

cheers, Martijn
Op donderdag 8 november 2018 00:13:06 UTC+1 schreef Kees van Hoekelen:

[Kees van Hoekelen]

Tnx Martijn,

One last mention, when I uncomment the test section in your code the telegram parses correct and the correct MQTT-messages are sent. I Also noticed that my telegram contains over 30 values while you got 9 in your code.

Again, tnx for your help and effort!

Cheers,

kees

Op vrijdag 9 november 2018 18:19:23 UTC+1 schreef Martijn Hendriks:

[Kees van Hoekelen]

Ok, got it working in the latest build.

In addition to the guide in the first post I added   GPIO_P1GATEWAY at he end of in the list kGpioNiceList (at about line 382) in sonoff_template.h

I made some changes to xsns_91_p1gateway.ino:

line 26: #define XSNS_91 to #define XSNS_91 91 according the manual in the wiki

line 36: changed #define EVERYXSEC 10 since my meter put out a telegram every 10 secs (DSMR4.2)

line 66: changed the {"GAS","0-1:24.2.1(",26,"*m3)",} key like in the post above

lines 172 - 175: commented out the P1gSerial->available() I don't know why, but without it works flawless

line 270: changed in   P1gSerial = new TasmotaSerial(pin[GPIO_P1GATEWAY], -1, 1); (added , 1 at the end) because it was also in the xsns_18_pms5003.ino

line 272: added if (P1gSerial->hardwareSerial()) { ClaimSerial(); } because it was also in the xsns_18_pms5003.ino

I also changed the telegramdata to match my meter (Kaifa).

let you guys know if this is stable…

Edit…

So far it is stable.

Last change, I added some code to strip the leading zeros from the values in the mqtt message. Last version of the driver attached.

My sensor in Home Assistant:

sensor:
  - platform: mqtt
    name: "Meter_P1"
    unit_of_measurement: "W"
    state_topic: "tele/sonoffP1/SENSOR"
    value_template: "{{ (value_json.P1GATEWAY.P1 | float *1000) | round(0) }}"
    icon: mdi:flash
    force_update: true
  - platform: mqtt
    name: "Meter_T1"
    unit_of_measurement: "kWh"
    state_topic: "tele/sonoffP1/SENSOR"
    value_template: "{{ value_json.P1GATEWAY.T1 | round(2) }}"
    icon: mdi:flash-circle
    force_update: true
  - platform: mqtt
    name: "Meter_T2"
    unit_of_measurement: "kWh"
    state_topic: "tele/sonoffP1/SENSOR"
    value_template: "{{ value_json.P1GATEWAY.T2 | round(2) }}"
    icon: mdi:flash-circle
    force_update: true
   
  - platform: mqtt
    name: "Meter_Gas"
    unit_of_measurement: "m3"
    state_topic: "tele/sonoffP1/SENSOR"
    value_template: "{{ value_json.P1GATEWAY.GAS | round(2) }}"
    icon: mdi:fire
    force_update: true
  - platform: mqtt
    name: "Meter Tarief"
    state_topic: "tele/sonoffP1/SENSOR"
    value_template: "{% if value_json.P1GATEWAY.TAR is equalto 1 %}Laag{% else %}Hoog{% endif %}"
 #   value_template: "{{value_json.P1GATEWAY.TAR}}"
    icon: mdi:swap-vertical
    force_update  : true  

Entry in ui-lovelace.yaml

      - type: entities
        title: Energie en gasverbruik
        entities:
          - entity: sensor.meter_p1
            name: Huidig verbruik
          - entity: sensor.meter_tarief
            name: Huidig tarief
          - entity: sensor.meter_t1
            name: Totaal verbruik laag
          - entity: sensor.meter_t2
            name: Totaal verbruik hoog
          - entity: sensor.meter_gas
            name: Totaal gas verbruik

Hope this is usefull for someone ;)

Cheers,

Kees

[C Bruggeman]

Maybe anyone here can help me with my problem.

I have the p1 cable connected to usb on the Raspberry pi 3b.

The Rpi runs Raspbian Jessie (with gui), Domoticz, Node Red, Mosquitto, monit, OpenHab..

It functions perfectly (I think); nice power usage graphics on Domoticz.

However when I enable MQTT in Domoticz, it post every second a nice JSON message with the power data. I did not really look into the values that were posted. Just the frequency totally clouded my MQTT traffic. So I disabled the Domoticz MQTT again.

But I want to use Domoticz with MQTT for my Tasmota Sonoff switches and temperature readings of a few esp8266, and my washing machine 8266, and my esp32 robot car (which I still have to build).

So Domoticz posting MQTT messages all the time is not preferred, but enabling MQTT is preferred.

I have looked into using OpenHab2 for the home automation, but I prefer Domoticz.

To install Domoticz on a separate Rpi for use with home automation kinda runs against the purpose of using less energy.

all kinds of extra hardware possibilities, but if someone just could disable Domoticz sending power-related  mqtt messages, I could enable Domotics MQTT...

[Colin Law]

On Sat, 24 Nov 2018 at 16:55, C Bruggeman <dakh...@gmail.com> wrote:
>
>
> Maybe anyone here can help me with my problem.
> I have the p1 cable connected to usb on the Raspberry pi 3b.
> The Rpi runs Raspbian Jessie (with gui), Domoticz, Node Red, Mosquitto, monit, OpenHab..
> It functions perfectly (I think); nice power usage graphics on Domoticz.
> However when I enable MQTT in Domoticz, it post every second a nice JSON message with the power data. I did not really look into the values that were posted. Just the frequency totally clouded my MQTT traffic. So I disabled the Domoticz MQTT again.

If you can't find how to slow it down why worry about it? If it were
posting messages 100 times/sec then possibly it could be a problem.
Just don't subscribe to those messages and forget about it.

Colin

[C Bruggeman]

I like to monitor the topic "#" on my smartphone when I am tinkeri g with some iot sensor. so I do not want a flood.

Why worry? Just let it be?

If I do not like it, it has to change. '!' . or go.

Verzonden door BlueMail

Op 24 nov. 2018, om 17:59, Colin Law <cla...@gmail.com> schreef:

[Moussy]

Dag Martijn,

Weet je waarom ik continu de onderstaande fout krijg?

sonoff_template.h:205: error: expected ',' or ';' before 'D_SENSOR_P1GATEWAY'

   D_SENSOR_P1GATEWAY ;

Alvast bedankt.

Badr

Op zondag 11 maart 2018 13:01:27 UTC+1 schreef Martijn Hendriks:

P1 smart meter MQTT wifi gateway. 

[Kees van Hoekelen]

Op donderdag 20 december 2018 17:08:44 UTC+1 schreef Moussy:

Dag Martijn,

Weet je waarom ik continu de onderstaande fout krijg?

sonoff_template.h:205: error: expected ',' or ';' before 'D_SENSOR_P1GATEWAY'

   D_SENSOR_P1GATEWAY ;

Alvast bedankt.

Badr

Does the content of sonoff_template.h around line 205 look like:

  D_SENSOR_TX20_TX "|"
  D_SENSOR_RFSEND "|" D_SENSOR_RFRECV "|"
  D_SENSOR_TUYA_TX "|" D_SENSOR_TUYA_RX "|"
  D_SENSOR_P1GATEWAY;

Check the placement of the ;

Place the part of sonoff_template.h here...

Cheers,

Kees

[Otto Elter]

Hi Kees,

Beautiful!

Is there any way you could compile and share the firmware in a bin file format to flash the SONOFF using ESPEasy? I'm not into programming but I'd love to use your solution in my Homeassistant.

Thanks in advance,

Otto

[Bram van de Kerkhof]

Tried to add the module. I'm not seeing the p1gataway in the pulldown in order to connect it to the pin.

I see that Kees had modified the p1 amount of info. I have a sagemcom. Should I use the original or the one from kees. 

I also would like to add the solar panel power delivery.

How can I do that?

[dik....@gmail.com]

Could someone tell me where I can fond the compiled tasmota firmware for the P1 interface. I am now using tasmota firmware from this site:
www.esp8266thingies.nl but I receive empty sensor Mqtt information when I select P98gateway for the Rx pin. But when I set it to none I receive all the meter data as a DSMR telegram.

[Alex van den Hoogen]

Hi,

I've tried this firmware too but also have a hard time getting this working. I've got a DSMR 4 reader and had a working solution with ESPEasy. But since I've migrated from Domoticz to Home Assistant that wasn't really sustainable anymore. But I cannot seem to get any result from either the firmware from esp8266thingies, Martijn's or Kees's versions. 

I've set up my Wemos D1 mini with a P1 shield and Tasmota set as a generic module. First I've tried to only setup the P1Gateway component to RX GPIO3, but that wasn't really working. I've alos tested to set up all the pins to point to P1Gateway, but that doesn't seem to yield any result either. Not even a single sensor update inside the logs. Same for None pins though.

Anyone got a suggestion or at least point me in the right direction?

- Alex

Op zaterdag 16 maart 2019 22:32:57 UTC+1 schreef dik....@gmail.com:

[dik....@gmail.com]

I've tried the same things as you did, without any results. I am back now on the working ESPEasy solution and found a way in HA to get the meter values. This is how I did it:

  - platform: scrape
    resource: http://<esp8266_ip_address>/P1
    name: p1
    select: 'th:nth-of-type(1)'
    value_template: '{{ value }}'
    scan_interval: 60

  - platform: template
    sensors:
      pnachttotaal:
        value_template: '{{ states.sensor.p1.state.split(":")[1].split("*")[0] }}'
        unit_of_measurement: 'kW'
      pdagtotaal:
        value_template: '{{ states.sensor.p1.state.split(":")[2].split("*")[0] }}'
        unit_of_measurement: 'kW'
      phuidig:
        value_template: '{{ states.sensor.p1.state.split(":")[5].split("*")[0] }}'
        unit_of_measurement: 'kW'
      gastotaal:
        value_template: '{{ states.sensor.p1.state.split(":")[7].split("*")[0] }}'
        unit_of_measurement: 'm3'

I am still struggling with it, to find a way to get the used gas for a certain period. Perhaps you have a solution for this, as I just started with HA and have to find out a lot of things.

Dik

Op zondag 24 maart 2019 15:38:24 UTC+1 schreef Alex van den Hoogen:

[Rico Vis]

hello kees can you help me  to a bin file from the p1-gateway for me  or downloading from link 

ferry tanks 

Op zaterdag 22 december 2018 09:08:24 UTC+1 schreef Kees van Hoekelen:

[Rico Vis]

Hello Kes Hoeke .
Nice project Can you help me
can i download the sketch from yor project ??
its ferry nice project im have tray this and whil not im have 6 day tray it . mabby you whil help me please
Tanks rico

[Ronaldo Schuurman]

Hello Martijn Hendriks,

Verry nice project.

Until now i used domoticz with the espeasy p1 gateway.

Now i'm moving to MQTT with node red, and came across this project.

i'm just starting with tasmota and the current version is v7.1.2.

Unfortunately I cannot apply your p1 project in this tasmota version.

Are you planning to update this p1 project to the new tasmota version?

thanks and have a nice day.

Regards Ronaldo

Op zondag 11 maart 2018 13:01:27 UTC+1 schreef Martijn Hendriks:

P1 smart meter MQTT wifi gateway. 

[Martijn Hendriks]

Update using Tasmota.

At the moment I don't use the special P1 module anymore.

Inspired by this post : https://github.com/arendst/Tasmota/wiki/P1-Smart-Meter I figured out an alternative implementation in this case for OpenHab. But other Home Automation tools could be used too.

Reguirements:

- P1 hardware (some resistors and a transistor to invert the P1 signal) see the post mentioned above.

- D1 mini of other ESP8266 hardware

- Home automation tool, OpenHab in this case

Steps:

1- flash the latest version of Tasmota to the esp device (at this moment I'm using 8.1.0 on a D1 mini)

2- enter in the console : SerialDelimiter 44 (makes datagram MQTT packages after '.  (with Delimiter 10 each P1 Datagram line is send but 50% is corrupt in my case)

3- add the P1 values to the *.items file (example below)

String P1 "test [%s]" {channel="mqtt:topic:mosquitto:slimmemeter2:p1" } 

Number P1_P "Actueel vermogen [%s]"

Number P1_GAS "Gasmeter: [%.1f]"

Number P1_T1 "kWh Teller 1: [%s]"

Number P1_T2 "kWh Teller 2: [%s]"

Number P1_I "Stroom: [%.2f A]"

4- add the P1 values to the *.sitemap file (example below) 

Text item=P1_T1 label="kWh Teller 1: [%.1f kWh]"

    Text item=P1_T2 label="kWh Teller 2: [%.1f kWh]"

    Text item=P1_P label="Actuele vermogen: [%.3f kW]"

    Text item=P1_I label="Actuele stroom: [%.1f A]"

    Text item=P1_GAS label="Gasmeter: [%.1f m3]"

5- add P1 channel to *.things file

// P1 - slimmemeter2

    Thing topic slimmemeter2 "Slimmemeter2" @ "Hal" {

    Channels:

    Type string : p1 "kWh Teller 1"                [ stateTopic="tele/P1/RESULT", transformationPattern="JSONPATH:SerialReceived" ]

}  

6- add rule to the *.rules file (example below)

//ESMR 5.0 DATAGRAM (MOST ESSENTIAL VALUES for me ;-)

//P1_T1 1-0:1.8.1(000185.000*kWh) (Totaal verbruik tarief 1 (nacht))

//P1_T2 1-0:1.8.2(000084.000*kWh) (Totaal verbruik tarief 2 (dag))

//P1_P 1-0:21.7.0(00.523*kW)

//P1_GAS 0-1:24.2.1(191222151005W)(02728.449*m3)

// --- ADDITIONAL SEE http://domoticx.com/p1-poort-slimme-meter-hardware/ for more info

//P1_T1R 1-0:2.8.1(000013.000*kWh) (Totaal geleverd tarief 1 (nacht)) 

//P1_T2R 1-0:2.8.2(000019.000*kWh) (Totaal geleverd tarief 2 (dag))

rule "P1 P"

when

    Item P1 received update

then

if (P1.state == NULL || P1 == "") return;

    val rawString = P1.state.toString

//P1_I 1-0:31.7.0(002*A)

val rawString3 = transform("REGEX", ".*([0-9]{3}).A.*", rawString)

if ( rawString3 != NULL && rawString3 !== null && rawString3 != "") { 

P1_I.postUpdate(rawString3) 

} 

//P1_T1 1-0:1.8.1(000013.000*kWh) (Totaal geleverd tarief 1 (nacht))

rawString3 = transform("REGEX", ".*1.8.1.([0-9]{6}.[0-9]{3}).kWh.*", rawString)

if ( rawString3 != NULL && rawString3 !== null && rawString3 != "") { 

P1_T1.postUpdate(rawString3) 

} 

//P1_T2 1-0:1.8.2(000084.000*kWh) (Totaal verbruik tarief 2 (dag))

rawString3 = transform("REGEX", ".*1.8.2.([0-9]{6}.[0-9]{3}).kWh.*", rawString)

if ( rawString3 != NULL && rawString3 !== null && rawString3 != "") { 

P1_T2.postUpdate(rawString3) 

} 

//P1_GAS 0-1:24.2.1(191222151005W)(02728.449*m3)

rawString3 = transform("REGEX", ".*[0-9]{12}.*([0-9]{5}.[0-9]{3}).m3.*", rawString)

if ( rawString3 != NULL && rawString3 !== null && rawString3 != "") { 

P1_GAS.postUpdate(rawString3) 

} 

// P1_P 1-0:21.7.0(00.523*kW)

rawString3 = transform("REGEX", ".*21.7.0.([0-9]{2}.[0-9]{3}).kW.*", rawString)

if ( rawString3 != NULL && rawString3 !== null && rawString3 != "") { 

P1_P.postUpdate(rawString3) 

} 

end

7- change the above according to your measuring wishes.

Suggestion to improve ... please let me know!

Cheers, Martijn

Op zaterdag 14 december 2019 13:06:12 UTC+1 schreef Ronaldo Schuurman: