Voice control via Google Home

[Stuart Hanlon]

Hi

In the hunt for the impossible, I stumbled across this article about Google home voice control API.

http://www.theverge.com/platform/amp/2016/10/4/13164882/google-assistant-actions-on-google-developer-sdk

Not being a software tech, it doesn't mean much to me, but is it something that we should consider looking into?

[Stuart Hanlon]

https://developers.google.com/voice-actions/

[orct...@gmail.com]

Hi

Has any one implemented this or any integration with Siri or other voice activated solutions for voice based commands?

[Rainer Hitz]

I’ve started to configure an Alexa-OpenRemote integration. It’s possible to do that with minimal programming with a combination of Alexa Home Skill, AWS lambda, AWS IoT and MQTT. Note that this solution does not support automatic device discovery.

The following resources were insightful for me :

Instructables - Home automation using RPi + Alexa + IoT

The previous Instructables link is very insightful because it explains steps by step how to configure OAuth by means of 'AWS Login with Amazon (LWA)'. OAuth is needed for the Alexa Skill.

The following YouTube videos are helpful to understand the link between AWS IoT and MQTT on the client side.

Amazon Web Services (AWS) IoT MQTT Node-RED Raspberry Pi & PLC

Raspberry Pi Node-RED Tutorial with Modbus & MQTT

I’ve haven’t yet finished the integration but managed to configure an Alexa Home Skill so that discovered devices appear in the Alexa App. Discovered devices means devices that were hard coded in the AWS lambda handler. I’ve also played around with AWS IoT and MQTT on the client side (Node-RED). 

To finish up this work I still have to do the following:

- Add devices to the AWS lambda handler and AWS IoT (manually no automatic discovery)

- Create Node-RED flows that are activated by means of MQTT and that finally call the OpenRemote API to execute commands. 

[Michal Rutka]

What Rainer've written is true, although for the minmum Alexa skill you don't need AWS IoT and MQTT. What you will need though is AWS developer account beside AWS account itself. You will need provide you credit card info to activate it. I've did some integration and all you need to create a skill is AWS lambda. In any case some programming skills would be required and the process itself is a bit cumbersome. What I'm missing in my integration is:

- devices discovery - right now all devices are hard coded into lambda;

- end to and encryption - right now I'm using plain text UDP protocol. For some kind security I use Google Authenticator key, it is better than nothing.

But, there is a simple way to drive OpenRemote controller with Amazon Echo. I'm using this method to drive devices which I've integrated with OpenRemote and they are not easily available in other platforms, mainly Bose Home Cinema (RS232) and EnOcean devices.

In the simpler solution I use Philips Hue emulator which sends UDP messages to my OpenRemote controller. It works very good and you need just few hours to get it running.

The Philips Hue home automation bridge is here https://github.com/bwssytems/ha-bridge/

I run it through docker with a command:

docker run -d --restart always --name="Home-Automation-Bridge" --net="host" -e SERVERIP="192.168.x.x" -e SERVERPORT="8083" -v /volume1/public/hab-philips-hue/:/config:rw -v /etc/localtime:/etc/localtime:ro aptalca/home-automation-bridge

Running it like this survives my docker machine reboots keeping the configuration intact. You can configure the bridge via web interface http://192.168.x.x:8083

All my devices are accessed through UDP packets, an example of configuring switching ON of my home cinema is 

udp://orb:23456/home-cinema-on

where orb is an IP address of my openremote controller, you can add it to /etc/hosts file.

The in the Openremote I've created UDP listener command 'udpl' which listens on port 23456. Adding fallowing rule:

rule "home cinema on"

when

  Event($s:source=="udpl", value=="home-cinema-on")

then

   RockerPress rp = new RockerPress();

  rp.setCnt(1);

  rp.setSource("TR1");

  rp.setValue("ROCKER_BI"); // TV, "ROCKER_BI" Sat

  insert(rp);

end

In the RHS part of the rule I'm simply emulating EnOcean rocker as I can switch on my home cinema with this rocker too (I've did it years ago and there is a demo on YouTube of me doing this. Of course you can put anything on the RHS of the rule which suits your situation.

This solution would work not only with Amazon Echo but also with Google Home and any other solution which is integrated with Philips Hue. As far as I know almost every IoT platform supports Philips Hue ;)

Hope this help,

Michal

@Rainer: as we are doing similar work with Amazon Echo integration, perhaps we can join forces towards an official Openremote Alexa skill? I wonder how big an interest for it is?

Lets anybody interested in this reply to this post with 'me too' :-)

[Tehnoinstyle Aleksey]

Hi

Yes, I have the following integration:

On my Synology NAS (last f/w) is installed the controller of the OpenRemote 2.6, then in the Docker is installed the homebridge + module of the homebridge-openremote.

On my iPad mini (with iOS 9.3.5, not iOS10), there is a Devices app that supports HomeKit. In this application, all the devices from OppenRemote are displayed.

Next, Siri. Speak the voice command and it performs!

вторник, 1 августа 2017 г., 5:45:26 UTC+3 пользователь orct...@gmail.com написал:

[Rainer Hitz]

One advantage of the MQTT solution is that it is standard based. In addition MQTT data communication is encrypted and no port forwarding configuration on the internet router or stuff like that is needed.

[Marcus]

I am also using ha-bridge which works quite well for small actions like on/of and level.

[orct...@gmail.com]

Thank you for the hints - I will read and try some of those links.

Although it would be nice to have audio commands natively supported through openremote. There is definitely a lot of interest in voice support on the home automation front at least here in the US.

[Michal Rutka]

I've started integrating Amazon Echo as soon as I've got the device in my hands, which was this March. I'm still unable to officially buy it in The Netherlands. I've red red my second device again on my friends address in Germany. As most OR developers are based in Europe you can see these delays. I still can only dream about Echo Show or even Echo Tap...

Nevertheless, IMHO web/app based UI are domed. In my case they are single use only and usually I don't open dedicated apps after few days anymore. Voice control is the way to go. This is the most natural user interface, and when done right it will dusk all other solutions.

[Liam Whiteside]

I've got a Vera controller which has an Echo skill in beta that you can request access to. It works well and allows me to turn lights on as I'm walking to a room etc.
For Alexa development and testing, look at the Reverb apps and web site. They enabled me to create a skill long before I had an actual Echo device.
I was hoping to also have OR setup to control Vera as voice control isn't always appropriate, but I'm finding the documentation is so poor and inaccurate it's not a great first time experience (the iOS app just crashes whenever I try to trigger a Vera command). If the professional version is better I've been put off long before I consider paying.

[orct...@gmail.com]

Hi Michael

Let me know if there is any way I can help. I am no software programmer but if you need any devices for testing, I can certainly buy if available in US and either ship them to you or I am planning to be in Europe in November so can bring them with and ship from within Europe.

Also if you can perform testing via remote acces in any way, will be happy to set up whatever you need and give you access from here.

[orct...@gmail.com]

Hi everyone

Thanks for all the pointers. I have made some progress on this but not sure how to proceed.

I have installed and I am able to login to the ha-bridge. It is on the same Raspberry Pi that is running the openremote controller.

Any detailed examples on how to link the ha-bridge to a switch and a dimmer currently working with Openremote (zwave) would be very helpful.

I am not clear on how ha-bridge will interface with Openremote.

Also I don't have the Amazon Echo yet. I am going to order one soon but meanwhile is there anyway to test the connection between Openremote and Ha-bridge with out Amazon Echo?

[Liam Whiteside]

On Monday, 7 August 2017 15:29:45 UTC+1, orct...@gmail.com wrote:

Also I don't have the Amazon Echo yet. I am going to order one soon but meanwhile is there anyway to test the connection between Openremote and Ha-bridge with out Amazon Echo?

Setup an Amazon account and link it to one of the solutions here https://reverb.ai/ 

[orct...@gmail.com]

Thanks Liam. On the Alexa app, which skills did you enable to interface it with ha-bridge?

[Michal Rutka]

This is built in skill. You just say "Alexa,discover devices" and they should pop up. Take a look in Smart Home section in the Alexa app.

[Michal Rutka]

Thank you for your kind offer. I might use it ;). Where you will be in Europe this fall?

[Michal Rutka]

For linking ha-bridge with openremote I'm using UDP commands and UDP listener sensor defined in Openremote. There is an example in my previous post.

[orct...@gmail.com]

Thanks Michael- I have not used Openremote with UDP listener so please excuse my ignorance. All my devices are zwave so far so I am not familiar with UDP listener protocol.

In you example, what is "home-cinema-on"

Based on the rule it appears to be a state of a sensor you have udpl- is it configured as a custom sensor?

How would adopt it for controlling z-wave devices?

[orct...@gmail.com]

I will be in Istanbul first 2 weeks of November. If you send me the devices you need, I can purchase from US and should be able to ship from Istanbul.

[Peter Truman]

The easiest way to run anything like this is via the OpenRemote HTTP REST API - as you can call the button presses via REST.

The "problem" is that natively, both Google and Amazon need to connect to OpenRemote to drive it.

The HA-Bridge software https://github.com/armzilla/amazon-echo-ha-bridge) can do this for you, as it run s as a bridge - but it requires Java 1.8, which can be a problem on some kit (my server runs Wheezy for example, and there is no 1.8 that is easily acquirable)

I'm looking at using IFTTT, which can access Google Home voice feeds, combined with Maker WebHooks - which can call internet URLS.  Add to that ngrok.com (which provides you a secure tunnel from your LAN to the internet), and you can link back to your HTTP REST accessible buttons, and voila....

[Michal Rutka]

The home-cinema-on is in fact a macro which turns many devices on and switches to right channels and volume levels. Anyway, in your case you have probably defined commands to drive your z-wave devices. Then this would work like this, in ha-bridge you define that when X device is switched on then an UDP packet is sent to the openremote controller. Then, in upenremote controller, an UDP custom sensor with the UDP listener should receive this command and trigger your Z-wave device command when it does. This action should be defined in a rule. Something like this:

rule "abc"
when
Event(source=="name of your UDP listener custom sensor", value=="UDP message defined in ha-bridge to switch the device X")
then
execute.command("your Z-wave device switch on command");
end

Of course you can have many rules like this for all your devices.

[orct...@gmail.com]

hi 

 i am able to get the interface between Ha-bridge and openremote working using the udp method and rules as suggested. It seems to be working fine with the test buttons on the Ha-bridge and i can control my z-wave devices.

however, i cant figure out how to link ha-bridge to Alexa? i have the reverb app on my phone and i am logged in to it with my amazon account but it does not discover any z-wave devices. 

do i have to configure any settings on the ha-bridge so it can connect to my amazon account? i clicked on the my echo tab on the ha-bridge and logged in to it also with my amazon account. 

I am missing the understanding of how ha-bridge connects to amazon Alexa account so that my z-wave devices on added on ha-bridge will be discovered. To clarify, i do not have any of the Amazon devices (echo or dot)

Thanks

[Michal Rutka]

I never tried the path you are trying to do. All I know is that when the Alexa and ha-bridge are on the same LAN then saying "Alexa, discover devices" is enough. Otherwise, only devices known to skill's cloud API are discovered, which is not the case of ha-bridge as the Philips Hue skill is built in and it should work even without internet connection, but Alexa won't respond without Internet as all it's "intelligence" is in the cloud. All in all, IMHO you need a phisical Amazon Echo device to get it going. Usually pressing 'Test' buttons in ha-bridge is enough to check.
Don't hesitate to purchase Alexa, you'll love it in an instance ;). If you like to listen music then standard Alexa is better, otherwise Echo Dot is enough (you can still onnct it to the better audio if you wish).

[orct...@gmail.com]

Just a quick update - I am able to get a basic On / off function working. I had to have a Amazon dot device to get Alexa to discover ha-bridge. Once the dot is online, I can use the reverb app from anywhere to send voice commands.

One thing that prevented this from working that was not obvious in any start up tutorials online was on the ha-bridge the upnp ip must be set to the ip of the device it is running on ( RPI in my case). The default setting did not work for me.
Here is a link that pointed me in that direction:

https://github.com/bwssytems/ha-bridge/issues/278#issuecomment-322183346

Thanks for all the help to get me started. I will update as I add more devices and test more functionality.

[orct...@gmail.com]

Hi Aleksey,

i am playing around with homebridge to get siri / apple home kit to work with openremote and having some issues. can you share more details on how you implemented homebridge with the openremote pluggin? I installed it but the documention for this plugin is very poor. Can you share your implementation with this pluggin, devices your are controlling and the config file?

https://www.npmjs.com/package/homebridge-openremote

I also tried the multiswitch udp pluggin and it seems to send UDP messages to Openremote but i must be doing something incorrect in the config file and the payload portion coz openremote UDP listerner sensor is not recognizing the text i am sending from homebridge. 

if you can provide more details on how you are using homebridge with openremote, that would be great

[Tehnoinstyle Aleksey]

Hi.

I tried this way:

The image for Docker on the Synology I downloaded from here: https://hub.docker.com/r/sybdata/homebridge/

My configuration files for install and config for homebridge-openremote (for Fibaro Dimmer and the Two-channel relay) in the attachment.

In log:

*** WARNING *** For more information see <http://0pointer.de/avahi-compat?s=libdns_sd&e=node&f=DNSServiceRegister>
stdout
13:53:32
npm info it worked if it ends with ok
stdout
13:53:32
npm info using npm@4.2.0
stdout
13:53:32
npm info using node@v7.10.1
stdout
13:53:32
npm info ok 
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m Loaded plugin: homebridge-openremote
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m Registering accessory 'homebridge-openremote.openremote'
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m ---
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m Loaded plugin: homebridge-server
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m Registering platform 'homebridge-server.Server'
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m ---
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m Loaded config.json with 3 accessories and 0 platforms.
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m ---
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m Loading 3 accessories...
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Dimmer][39m Initializing openremote accessory...
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Switch 1][39m Initializing openremote accessory...
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Switch 2][39m Initializing openremote accessory...
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Dimmer][39m Switch received status http://192.168.1.3:8688/controller/rest/status/403148 state is currently 0
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Dimmer][39m Setting power state to off
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Switch 1][39m Switch received status http://192.168.1.3:8688/controller/rest/status/403150 state is currently 0
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Switch 1][39m Setting power state to off
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Switch 2][39m Switch received status http://192.168.1.3:8688/controller/rest/status/403151 state is currently 0
stdout
13:53:32
[37m[7/28/2017, 1:53:32 PM][39m [36m[Switch 2][39m Setting power state to off
stdout
13:53:33
[37m[7/28/2017, 1:53:33 PM][39m [36m[Switch 1][39m HTTP set power function succeeded!
stdout
13:53:33
[37m[7/28/2017, 1:53:33 PM][39m [36m[Switch 2][39m HTTP set power function succeeded!
stdout
13:53:33
[37m[7/28/2017, 1:53:33 PM][39m [36m[Dimmer][39m HTTP set power function succeeded!
stdout
13:53:33
Scan this code with your HomeKit App on your iOS device to pair with Homebridge:
stdout
13:53:33
[30m[47m                       [49m[39m
stdout
13:53:33
[30m[47m    ┌────────────┐     [49m[39m
stdout
13:53:33
[30m[47m    │ 031-45-154 │     [49m[39m
stdout
13:53:33
[30m[47m    └────────────┘     [49m[39m
stdout
13:53:33
[30m[47m                       [49m[39m
stdout
13:53:33
[37m[7/28/2017, 1:53:33 PM][39m Homebridge is running on port 51826.
stdout
13:54:43
[37m[7/28/2017, 1:54:43 PM][39m [36m[Switch 1][39m Switch received status http://192.168.1.3:8688/controller/rest/status/403150 state is currently 0
stdout
13:54:43
[37m[7/28/2017, 1:54:43 PM][39m [36m[Switch 1][39m Setting power state to off
stdout
13:54:43
[37m[7/28/2017, 1:54:43 PM][39m [36m[Switch 1][39m HTTP set power function succeeded!
stdout
13:54:03
[37m[7/28/2017, 1:55:03 PM][39m [36m[Switch 1][39m Switch received status http://192.168.1.3:8688/controller/rest/status/403150 state is currently 1
stdout
13:54:03
[37m[7/28/2017, 1:55:03 PM][39m [36m[Switch 1][39m Setting power state to on
stdout
13:54:03
[37m[7/28/2017, 1:55:03 PM][39m [36m[Switch 1][39m HTTP set power function succeeded!
stdout
13:54:29
[37m[7/28/2017, 1:55:29 PM][39m [36m[Switch 1][39m Switch received status http://192.168.1.3:8688/controller/rest/status/403150 state is currently 0
stdout
13:54:29
[37m[7/28/2017, 1:55:29 PM][39m [36m[Switch 1][39m Setting power state to off
stdout
13:54:29
[37m[7/28/2017, 1:55:29 PM][39m [36m[Switch 1][39m HTTP set power function succeeded!

пятница, 18 августа 2017 г., 6:44:09 UTC+3 пользователь orct...@gmail.com написал:

[orct...@gmail.com]

Thanks for sharing! What configuration did you use on Openremote side (commands and sensors) for receiving these messages and executing on them? I appologize I have not used http before so this new to me.

[Tehnoinstyle Aleksey]

Hi.

This is the OpenRemote Controller API. Previously, in the old forum, there was a section: OpenRemote Controller API, in which there were examples (xml and json).

Now I can not find this information. In my examples of configuration files, there are a few commands that I remember.

In the web browser commands

Output of panels:

http://192.168.1.7:8688/controller/rest/panels

Answer:

<Panel id = "5" name = "My Office" />

Panel composition:

http://192.168.1.7:8688/controller/rest/panel/MyAndroid

Answer: (piece)

<Switch id = "397">

<Link type = "sensor" ref = "5837807">

<State name = "on" value = "LightOnN.png" />

<State name = "off" value = "LightOffN.png" />

</ Link>

</ Switch>

</ Absolute>

<Absolute left = "260" top = "202" width = "38" height = "46">

<Switch id = "399" />

In the answer we are looking for switch id =

For example: <switch id = "395"> Switch 1

For him, the command:

http://192.168.1.7:8688/controller/rest/control/395/on

http://192.168.1.7:8688/controller/rest/control/395/off

For example: <switch id = "397"> Switch 2

For him, the command:

http://192.168.1.7:8688/controller/rest/control/397/on

http://192.168.1.7:8688/controller/rest/control/397/off

For <switch id = "401" /> Dimmer

http://192.168.1.7:8688/controller/rest/control/401/on

http://192.168.1.7:8688/controller/rest/control/401/off

суббота, 19 августа 2017 г., 18:11:17 UTC+3 пользователь orct...@gmail.com написал:

[orct...@gmail.com]

Thanks you! I think I am starting to understand how this might be working- in your example above and the config.jason, what are the IP addresses 192.168.1.7 and 192.168.1.3 assigned to?

[Tehnoinstyle Aleksey]

IP addresses 192.168.1.7 and 192.168.1.3 - IP addresses OpenRemote Controllers.

воскресенье, 20 августа 2017 г., 16:19:38 UTC+3 пользователь orct...@gmail.com написал:

[orct...@gmail.com]

Alekey thanks for posting all your configs. Helped a lot. I have the lights working fine with homebride and Sirir now using the openremote plugin.

A couple of things i noticed was it only understands values "on" and "off"

Also when i try using it for status of sensors like a door sensor, if i assign the states "on" and o\"off" it seems to work ok buti have to restart the Home app for the status to refresh. It does not refresh automatically. Have you used this homebridge for any other sensors other than switches? below is a sample config i am using for a door sensor which does not seem to update status automatically. I tried using GET and POST but same result. I wonder is there is any other way or plugins for sensors.

{
        "accessory": "openremote",
        "name": "Patio door",
    "switchHandling": "realtime",
    "http_method": "GET",
    "status_url": "http://192.168.1.30:8688/controller/rest/status/408980"        
    }

[Tehnoinstyle Aleksey]

Hi.

I do not have a Home app (since my iPad mini has only iOS 9.3.5). I use the Devices application (from this) for connect to homebridge.

But I also noticed this behavior of this application. It also sometimes does not determine the state of the devices - i.e. If you change the state of devices (i have dimmer, switchs, thermostats) from the Devices application itself, this works fine. But if the state of the devices changes from a third-party application or device, then the device state in the Devices application is not always correct.

With sensors I'm not very versed, but the behavior is similar.

четверг, 24 августа 2017 г., 7:22:14 UTC+3 пользователь orct...@gmail.com написал: