Difficulty with with Organization Log Sink to Pub-Sub and Cloud Function

[Dave Carmocan]

I have been looking to implement a solution for logs from /cloudaudit.googleapis.com%2Factivity to trigger a Cloud Function using Pub-Sub, but I haven't had much luck.

Pub-Sub to Cloud Function seem to be working fine, but it appears that the activity logs are never getting to Pub-Sub, even when removing the filter.  

Has anyone actually had any luck with Cloud Audit Logs and Pub-Sub?  Trying to see if this is worth tinkering around with or if I should look into another solution.  

[Jun Lu]

Hey Dave, 

Can you let us know what steps you've done, or what documentation you've followed to trigger the Cloud Functions using Cloud Pub/Sub? If you're looking to trigger Cloud Functions by Cloud Pub/Sub messages, then you should look at the documentation of Google Cloud Pub/Sub Triggers at [1]. 

[1] https://cloud.google.com/functions/docs/calling/pubsub

[Dave Carmocan]

Sure thing.  I have actually been going through the documentation extensively as I was butting my head against this and I think that I may have finally found a solution (although, I am keeping an eye on the logs to make sure).

The main issue that I had been having appeared to be with Organization Audit Logs (organizations/${OrgID}/logs/cloudaudit.googleapis.com%2Factivity) and the Log Sink that was created to route the logs matching the filter to my Pub Sub topic.  From there the Subscription (Push) would be triggering a Cloud Function to get the gropupUniqueId from the protoPayload of the logs and make a Patch request to Group Settings API. 

I hadn't been seeing any errors linked to permissions or any activity and I making sure that I was following this:

But as I was looking, I also saw this bit, which is where I think I was encountering the problem:

When I got up this morning to do some more testing, I finally started seeing activity from the Writer Identity account that was created with the Log Sink so I think that this is what I had been running into. 

[Preston Holmes]

Hi Dave, you might also look at the recently released https://cloud.google.com/blog/products/serverless/build-event-driven-applications-in-cloud-run Eventsarc integration with Cloud Run which brings audit logs as a first-class "trigger" source.

It may not give you as much up-front filtering capability, but it sounds like you are already doing further logic in your function.

 

-Preston