Error reporting linked to Google Chat through webhook not outputting anything

[Quentin Laville]

Hi everybody,

I am trying to get Error reporting messages to Google Chat and by email through Notification channels. 

I added two new notification channels, one using my email and one by creating a new webhook in a new Google Chat group. When an error occurs, I get an email as expected but nothing on the Google Chat side.

Note that when adding the webhook to the Notification channels, the "Test connection" action isn't outputting anything either.

Is there permission I could be missing? Is there an issue with the format Error reporting is using?

Thanks for your help!

---

https://alquant.com

https://www.linkedin.com/company/alquant

Alquant SA

Chemin de Mongevon 25

1023 Crissier

Switzerland

Email: in...@alquant.com

Regulatory information (FIDLEG): https://alquant.com/regulatory-information

Ombudsman: FINOS (https://www.finos.ch/mediation/?lang=en)

This message (and any attachments) is for the addressee's use only and may contain confidential, proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you are not the intended recipient, any direct or indirect use, disclosure, printing, copying or distribution of this message or of the parties to it is prohibited and may be unlawful. Please notify the sender urgently and delete the message and any copies (including hardcopies) of it immediately. There are risks in communicating by e-mail. E-mail is susceptible to data corruption, delay, interception and unauthorized amendment. Whilst all adequate care has been taken, Alquant SA (and its group entities) do not accept responsibility or liability as to the completeness or accuracy of the information and for any damages resulting from e-mail use.

Alquant SA (and its group entities) reserve the right to monitor and save all email communication through their networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of Alquant SA (and/or its group entities).

The information provided does not constitute an, and is not provided in relation to any, offer or solicitation to buy or sell any security, currency, investment fund or other product, service or information to anyone in any jurisdiction in which an offer or solicitation is not authorized or cannot legally be made or to any person to whom it is unlawful to make an offer or solicitation.

Access to this information is forbidden to persons in any jurisdiction where the publication of or access to the product section of the Alquant Platform (https://platform.alquant.com/products/protech/overview) would result in a violation of any applicable law or regulation (by way of domicile, residence, nationality, investor status, or otherwise). Learn more: https://platform.alquant.com/compliance/terms-of-use

Past performance is no guarantee of future results.

[ashok...@google.com]

Hello,

There certain roles that are necessary to configure a notification channel and the log entry format is required for using Cloud Logging to report error events. In order to configure notification via Webhooks, select the webhooks tab under the Create notifictaion Channel and follow the instruction. I hope this helps.

Thank You

[Quentin Laville]

Thank you for your answer. I followed the instruction, I created a new webhook in a Google Chat group, got the URL of the webhook, and added it to the Notification Channels in the "Webhook" section. However, as I wrote previously when I do the "Test Connection", I don't receive anything in the Google Chat group. 

I would have thought that "Monitoring" of GC (where you set up the "Notification channels") sends a correct format for Google Chat webhook.

Am I mistaken? is there anywhere I can find logs about the failing "Test connection"?

[ANTHONY CROOKE]

Quentin,

I had a similar outcome on a webhook I experimented with months ago. Please share your Google findings on this message thread.

Best Regards,

Tony Crook

Mobile: 248-977-7106

On Apr 6, 2022, at 2:15 PM, 'Quentin Laville' via Google Cloud Developers <google-c...@googlegroups.com> wrote:

Thank you for your answer. I followed the instruction, I created a new webhook in a Google Chat group, got the URL of the webhook, and added it to the Notification Channels in the "Webhook" section. However, as I wrote previously when I do the "Test Connection", I don't receive anything in the Google Chat group. 

--
You received this message because you are subscribed to the Google Groups "Google Cloud Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-dev/671ce146-7826-4d6f-a525-6db743301484n%40googlegroups.com.

[Bruno (Cloud Platform Support)]

Hi, 

There may be several reasons why you did not receive a notification (e.g: using a private endpoint as mentioned in our documentation). To examine your Cloud Logging log entries and gather more information about the root cause of failure in your use case, follow this guide. This will also help confirm if the issue has to do with the wrong formatting. 

It may also help to provide which documentation/guide you followed when setting up the notifications channels. This will allow us to attempt to reproduce the issue. 

[Quentin Laville]

Hi Bruno,

Thanks to you I was able to find the logs, and indeed it looks like a format issue.

For reproduction, I followed the documentation you already provided:
1. Created a webhook on google chat, and got a link to use.

2. Added a webhook in "Notification channels" of "Monitoring" in GC, using the link created in step 1.
3. In "Error reporting" went on the tab "Configure notifications" and from the dropdown "Notification channels" selected the new webhook created in step 2, in the section "Webhook with Token Authentication"
4. In "Error reporting" marked one of the errors I can repeat in my backend on an application deployed on app engine to resolve.

5. Triggered manually the error, and checked in the log to see that the webhook failed.

I haven't seen a way to change the format of the error message sent from "Error reporting", should I push this further and post an issue somewhere?

[Darrell (Cloud Platform Support)]

Hi,

Thank you for providing your reproduction steps as I was also able to get the same results.

You are correct that this is a formatting issue of the payload. In fact, the specification of the payload that is delivered by Stackdriver webhook(Monitoring) and the specification expected by Google Chat are different. Stackdriver webhook payloads represent a "Stackdriver incident" and include details that make sense in the context of Stackdriver. Google Chat messages represent information to be displayed that is specific to Google Chat and its various messaging capabilities.

In general, attempting to point a Stackdriver webhook directly at another service (whether Google Chat or any other API, whether Google-owned or third party), will not work. The expectation with the webhook channel is that you will implement your own microservice to receive them and that is designed to accept the particular format of the Stackdriver webhook payload.

This microservice can be implemented on Google Cloud (e.g. using Cloud Run, Cloud Functions, App Engine, Kubernetes Engine, Compute Engine, etc.) or on premise. That microservice can then convert the Stackdriver webhook payload, which typically looks something like [1] into a Google Chat message, which can use simple text or a more sophisticated card representation [2], and issue its own HTTP request to the Google Chat API. A guide on how to do with Cloud Run so can be found here[3]

In addition, I found this feature request[4] for Native Integration of the Webhook GCP alert to Google Chat. I suggest you comment and click on the star on top of the page. I hope you find this information useful.

[1]

{

"incident": {

"incident_id": "f2e08xxxxx393bz",

"resource_id": "i-4xxxa2d",

"resource_name": "wxxxxr-85",

"state": "open",

"started_at": 1385085727,

"ended_at": null,

"policy_name": "Wxxxxer Health",

"condition_name": "CPU usage",

"url": "https://console.cloud.google.com/monitoring/alerting/incidents?project=PROJECT_ID",

"summary": "CPU for wxxxxr-85 is above the threshold of 1% with a value of 28.5%"

},

"version": 1.1

}

[2] https://developers.google.com/hangouts/chat/reference/message-formats/cards

[3] https://cloud.google.com/blog/products/operations/write-and-deploy-cloud-monitoring-alert-notifications-to-third-party-services

[4] https://issuetracker.google.com/219749842