Google Assistant - Home - Nest - Souliss integration
55 views
Skip to first unread message
Fernando Mancebo
Jan 15, 2020, 1:24:55 PM1/15/20
to souliss
Good night,
I've been waiting for this for quite some time. It turns out that Google seems to have decided to allow connectivity to its universe without relying on external Hubs, making it local.
Verbal automation with Google servers will still be necessary but the recognition of the devices seems to be local.
I have been trying different alternatives and it seems that this is the best. I am in the study phase. I will upload this post to the three languages of Souliss groups. Especially the Italian, because surely they are able to perform a clean integration.
Interesting links:
Now I'm involved with this, with dialogflow and firebase:
Enjoy
F
Fernando Mancebo
Jan 15, 2020, 1:32:55 PM1/15/20
to souliss
Di Maio, Dario
Jan 15, 2020, 2:32:45 PM1/15/20
to sou...@googlegroups.com
Hi Fernando,
seems nice, but isn't completely clear to me which device in the LAN send the binary command.
In the drawing I see the Google Hub, could this be replaced somehow?
In the example that you have linked, the natural language is not processed by Google and they only use the Ok Google vocal key.
Which would be your suggested implementation and architecture?
Thanks.
Regards,
Dario.
Fernando Mancebo
Jan 16, 2020, 3:37:00 AM1/16/20
to sou...@googlegroups.com
Hi Dario, everyone
I am not an expert on these issues but after reading and playing a few days with the documentation I sent in my mail yesterday, I will try to give a simple explanation to a complex ecosystem such as the Google Assistant universe. If I am wrong, in whole or in part, I will appreciate the corrections.
I am not an expert on these issues but after reading and playing a few days with the documentation I sent in my mail yesterday, I will try to give a simple explanation to a complex ecosystem such as the Google Assistant universe. If I am wrong, in whole or in part, I will appreciate the corrections.
- Google Assistant is a system that allows acting through natural language.
- The basic devices for entering information are smartphones, Google Home and Google Nest. (Home and Nest are SOC´s with microphone and speaker expressed in a very basic way.)
- Natural language is processed in the cloud and returned to the device.
- To handle branded home devices (eg Fibaro), the manufacturer (Fibaro) has an environment in the cloud where your personal Google account is synchronized with the manufacturer's account (Fibaro) that you have been able to register when you buy your hardware.
- Your Google Home device does not speak directly with your Fibaro sensor or with your switch, Google Home speaks with Fibaro in the cloud and Fibaro is the one who acts on your device.
- With this, Google manages to have control (information) of the devices that are controlled in its universe.
- With this, Fibaro manages to protect its hardware by making it compatible and exclusive with the Google ecosystem.
- So far there have been many attempts to create an open source system to connect SOC´s to Google Assistant (until the use of IFTTT).
In my opinion, Google has opened a backdoor to its Google Assistant universe.
As I explained above, there is always an intermediate actor who is usually a hardware manufacturer.
As I was able to read, this past month Google has enabled the local execution option. This means that the Google Home or Nest device (today the smartphone is not worth) communicates directly with the devices without having to go through the cloud and without having to pay the price of branded devices. (The smartphone is valid if it is connected to a Google Home or Nest)
As I explained above, there is always an intermediate actor who is usually a hardware manufacturer.
As I was able to read, this past month Google has enabled the local execution option. This means that the Google Home or Nest device (today the smartphone is not worth) communicates directly with the devices without having to go through the cloud and without having to pay the price of branded devices. (The smartphone is valid if it is connected to a Google Home or Nest)
The technique proposed by Google is to upload a piece of code, written in TypeScript or JavaScript to the SOC (Google Home or NEST) and execute it there. It allows the use of various methods to communicate with devices such as UDP, MDNS, UPNP that are documented here.
I want to understand that through UDP it should be easy to integrate Souliss.
The definitive system, in blocks, is very similar to a Raspberry running Openhab, with a piece of interconnection code (API) between the devices and OH as Souliss already has.
In fact, Google released the Google Assistant code to load it into a Raspberry and convert it into a Google Home. In my opinion it is not worth it because now the Google Nest costs less than € 30 and with a 7-inch screen € 70. I would only understand installing it on a Raspberry if you want deep hardware integration.
In short, what is sought with local execution is a lower latency (waiting time between the order and its execution), greater independence from the manufacturers and greater flexibility in the dialogue with the devices.
An API to govern them all..... ;)
Your questions:
Seems nice, but isn't completely clear to me which device in the LAN send the binary command : The Google Assistant device (Home, Nest) with a piece of code inside.
In the drawing I see the Google Hub, could this be replaced somehow? : For communication between Google SOC and device, the cloud is not necesary, for voice functions yes. The power of Google SOC device is the voice control.
In the example that you have linked, the natural language is not processed by Google and they only use the Ok Google vocal key : Natural language is processed every time. Always start a conversation using OK, Google and then the command you want to execute in natural language, with the options you want.
Steps with hardware manufacturer dependency
1 human voice: Hey Google!
1 human voice: Hey Google!
2 human voice: turn on the dining room light
3 Google Home sends the phrase to the cloud and processes it. There are thousands of "intentions" stored in the cloud and you can add your own intentions.
4 Google Cloud responds to Google Home ON and DINING
5 Google voice: very well, turning on the dining room light
6 Google Home processes that information and according to the SYNC that has with the hardware manufacturers, sends ON DINING LOGIN CODE to Fibaro (ie).
7 The hardware manufacturer (Fibaro ie) processes the information and sends the device (Switch) the order to turn on.
7 The hardware manufacturer (Fibaro ie) processes the information and sends the device (Switch) the order to turn on.
Steps in local mode
DINING ROOM LIGHT ON
1 human voice: Hey Google!
2 human voice: turn on the dining room light
3 Google Home sends the phrase to the cloud and processes it. There are thousands of "intentions" stored in the cloud and you can add your own intentions.
4 Google Cloud responds to Google Home ON and DINING
5 Google Home voice: very well, turning on the dining room light
6 Google Home processes that information with the API loaded in the SOC and sends the device (Switch) the order to turn on.
TEMPERATURE IN DINING ROOM
1 human voice: Hey Google!
2 human voice: tell me the temperature in the dinning room
3 Google Home sends the phrase to the cloud and processes it. There are thousands of "intentions" stored in the cloud and you can add your own intentions.
4 Google Cloud responds to Google Home TEMPERATURE and DINING and GET
5 Google Home processes that information with the API loaded in the SOC and sends the device (Sensor) the order to get temperature.
6 The SOC (esp8266 ie) returns 25.5 degrees celsius to the Google Home API in number format.
7 Google Home voice: very well, the dinning room temperature is 25.5 degrees
8 An intention may be programmed (inside Google Home environment) after step 7. Google Home can ask if you want to raise or lower the temperature or if you are cold or hot and act accordingly.
That is a small step for Souliss, a great leap for open source humanity. :)
From now on, it's time to dream ...
Steps with Spotify (For understanding)
Open an account on spotify. (web access)
Synchronize and authorize google home the spotify integration with my personal Google account (using Smartphone Google Assistant)
Synchronize and authorize google home the spotify integration with my personal Google account (using Smartphone Google Assistant)
1 human voice: Hey Google!
2 human voice:
: I want to hear police
3 Google Home sends the phrase to the cloud and processes it. There are thousands of "intentions" stored in the cloud and you can add your own intentions. (Maybe this typical intention is stored in the Google Home firmware and never sent to Google Cloud, I ignore it.)
4 Google Cloud responds to Google Home "The Police" and Google Home identify the sound system synced and processes and route the order to Spotify server. The same behavior when you say "Police" to the Spotify application.
5 Google Home voice: I will play the police on Spotify6 Google Home sounds every breath you take
Enjoy
F
If you have any questions, don't hesitate to ask me.
--
You received this message because you are subscribed to a topic in the Google Groups "souliss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/souliss/ruRyPLZE6Ns/unsubscribe.
To unsubscribe from this group and all its topics, send an email to souliss+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/souliss/CAKGhMP%2BDdokTL%3Dfe%2BFNyb9sBBkoHT%2BHHBDOkVC7yHkca4UQa5w%40mail.gmail.com.
Fernando Mancebo
Jan 16, 2020, 4:09:54 AM1/16/20
to sou...@googlegroups.com
I'm so sorry, forget to comment on the last point.
Q: Which would be your suggested implementation and architecture?
In my opinion, it is enough to develop a piece of code that interprets the Google Home actions to convert them into Souliss nodes and slots and vice versa. Very similar to OH.
This code is uploaded to Google Home SOC with the Google Assistant Tools and methods.
The rest of the programming logic of the Google natural language environment and the responses to the actions that are executed in Google Asistant are developed with Google Assistant tools.
A list of devices and behaviour can be find here:
Enjoy
F
Alessandro
Jan 16, 2020, 5:07:21 AM1/16/20
to sou...@googlegroups.com
>
I want to understand that through UDP it should be easy to integrate Souliss.
good luck with that.
I had a look to Google Home's "API" and from what I got it's impossible to have a proficient integration between google and souliss if not using cloud and HomeGraph API.
Everything too much web-based, to me, but that's just my irrelevant point of view. What _is not_ irrelevant is the amount of time required to transcode devices, commands and semantics
I'll follow this thread, though.
Alessandro
You received this message because you are subscribed to the Google Groups "souliss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to souliss+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/souliss/CALW9kc82D0WCEgyfsJXMW%2BuZhz%2B3ty4dM32FXp%2B%2Bh%3D%3DBSzy8Xg%40mail.gmail.com.
Fulvio Spelta
Jan 18, 2020, 3:29:26 AM1/18/20
to souliss
Very interesting.
Me too will follow this nice thread.
Me too will follow this nice thread.
Reply all
Reply to author
Forward
0 new messages