Stackdriver Logging API authentication issue - "The caller does not have permission"

[vaibhav...@harness.io]

Hi,

I am trying to fetch stackdriver log entries using this REST API: https://logging.googleapis.com/v2/entries:list?key={MY_API_KEY}

I have created an API Key as per the steps mentioned here and set API restrictions to Stackdriver API, Stackdriver Logging API and Stackdriver Monitoring API.

Request Body:

"resourceNames": [

    "projects/{MY_PROJECT}"

]

}

Response:

{

    "error": {

        "code": 403,

        "message": "The caller does not have permission",

        "status": "PERMISSION_DENIED"

    }

}

What am I missing here? What do I need to modify to make this work?

The API works via OAuth2.0, but not with the API Key. Please help me out.

Note: My use-case requires me to use the REST APIs, and I cannot use the Service Account + SDK flow.

[Marc Unangst]

Hi Vaibhav,

The Stackdriver Logging APIs require requests to be authenticated (OAuth2) and don't allow requests to be made with only an API key. The requesting user must be authorized with the appropriate Cloud IAM permissions (see here: https://cloud.google.com/logging/docs/access-control). Most Google Cloud APIs work this way, because they usually access or modify resources that are protected through IAM, and there's no "principal" to grant access to if the request only uses an API key.

Can you describe a bit more about your use case, and why you're not able to provide a user or service account credential with the request?

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.
---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To post to this group, send email to google-stackdr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/33728013-94d2-4f3d-85c3-7c8e064bb07e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shuki Lehavi]

Marc, then what's the point of the API? I am looking to write an application that grabs the error logs (nightly). How will I do that without using the api key method?

Shuki

On Tuesday, September 25, 2018 at 8:05:46 PM UTC-7, Marc Unangst wrote:

Hi Vaibhav,

The Stackdriver Logging APIs require requests to be authenticated (OAuth2) and don't allow requests to be made with only an API key. The requesting user must be authorized with the appropriate Cloud IAM permissions (see here: https://cloud.google.com/logging/docs/access-control). Most Google Cloud APIs work this way, because they usually access or modify resources that are protected through IAM, and there's no "principal" to grant access to if the request only uses an API key.

Can you describe a bit more about your use case, and why you're not able to provide a user or service account credential with the request?

On Fri, Sep 21, 2018 at 6:11 AM <vaibhav...@harness.io> wrote:

Hi,

I am trying to fetch stackdriver log entries using this REST API: https://logging.googleapis.com/v2/entries:list?key={MY_API_KEY}

I have created an API Key as per the steps mentioned here and set API restrictions to Stackdriver API, Stackdriver Logging API and Stackdriver Monitoring API.

Request Body:

"resourceNames": [

    "projects/{MY_PROJECT}"

]

}

Response:

{

    "error": {

        "code": 403,

        "message": "The caller does not have permission",

        "status": "PERMISSION_DENIED"

    }

}

What am I missing here? What do I need to modify to make this work?

The API works via OAuth2.0, but not with the API Key. Please help me out.

Note: My use-case requires me to use the REST APIs, and I cannot use the Service Account + SDK flow.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 

Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdriver-discu...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-discussion+unsub...@googlegroups.com.

[Marc Unangst]

Hi Shuki,

For that application you would need to use a service account (https://cloud.google.com/iam/docs/understanding-service-accounts) to authenticate the script, and then grant the service account the logReader role. API keys don't provide secure authentication and can't appear in a Cloud IAM policy, so they can't be used to authorize access to sensitive data like logs. If you're running your code on GCP (on a GCE VM, GAE app, GKE cluster, etc.) then there will already be a service account associated with that compute resource that your code will use to authenticate. If you're running outside of GCP, you'll need to create a new service account, and download and securely manage the service account credentials (https://cloud.google.com/iam/docs/creating-managing-service-account-keys)

However, for this use case (grabbing all error logs nightly) you might consider using a Log Export (https://cloud.google.com/logging/docs/export/) to send all of the logs you're interested in to a GCS bucket and then read them from the bucket. This will likely be faster than using the log-reading API directly, and many processing tools (Dataflow, Dataproc, Cloud Functions, etc.) support reading directly from GCS.

On Tue, Aug 20, 2019 at 1:19 AM 'Shuki Lehavi' via Google Stackdriver Discussion Forum <google-stackdr...@googlegroups.com> wrote:

Marc, then what's the point of the API? I am looking to write an application that grabs the error logs (nightly). How will I do that without using the api key method?

Shuki

On Tuesday, September 25, 2018 at 8:05:46 PM UTC-7, Marc Unangst wrote:

Hi Vaibhav,

The Stackdriver Logging APIs require requests to be authenticated (OAuth2) and don't allow requests to be made with only an API key. The requesting user must be authorized with the appropriate Cloud IAM permissions (see here: https://cloud.google.com/logging/docs/access-control). Most Google Cloud APIs work this way, because they usually access or modify resources that are protected through IAM, and there's no "principal" to grant access to if the request only uses an API key.

Can you describe a bit more about your use case, and why you're not able to provide a user or service account credential with the request?

On Fri, Sep 21, 2018 at 6:11 AM <vaibhav...@harness.io> wrote:

Hi,

I am trying to fetch stackdriver log entries using this REST API: https://logging.googleapis.com/v2/entries:list?key={MY_API_KEY}

I have created an API Key as per the steps mentioned here and set API restrictions to Stackdriver API, Stackdriver Logging API and Stackdriver Monitoring API.

Request Body:

"resourceNames": [

    "projects/{MY_PROJECT}"

]

}

Response:

{

    "error": {

        "code": 403,

        "message": "The caller does not have permission",

        "status": "PERMISSION_DENIED"

    }

}

What am I missing here? What do I need to modify to make this work?

The API works via OAuth2.0, but not with the API Key. Please help me out.

Note: My use-case requires me to use the REST APIs, and I cannot use the Service Account + SDK flow.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 

Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.

To post to this group, send email to google-stackdr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/33728013-94d2-4f3d-85c3-7c8e064bb07e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 

Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/48f094b3-ec05-4306-9bfd-d21e7e8e8cd4%40googlegroups.com.

[Alexander]

Hi,

I have process automation tools that need to query something in logs before the tools do something.

My tools only able to call REST API, so I tried to used API Key but got the same error.

"message": "The caller does not have permission",

"status": "PERMISSION_DENIED"

Since the tools already built-in, we can't used ADC as authentication.

Any idea to use the API using REST? Thanks

[Marc Unangst]

Hi Alexander,

As I said in my other message, the Logging API doesn't support API key authentication. You need to provide an OAuth credential to authorize the request, and the user in the credential must have the appropriate Cloud IAM role(s). You don't have to use ADC; there are a number of ways to obtain the appropriate credentials. See https://cloud.google.com/docs/authentication/best-practices-applications for more info.

BTW, I should also mention that using logs as a communication path between two parts of your app isn't a great idea, since developers might change or remove a log statement without realizing that something else depends on it. If you're able, you might consider using another mechanism to signal to the process automation tool that it can move on to the next step, like a Cloud Pub/Sub message or writing a marker record into a database.

-Marc

Eng Director, Cloud Logging 

--
© 2021 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.
---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/979601e7-3a41-48c0-9eb6-394a23f5d005n%40googlegroups.com.