Trying to Reproduce the Payload for Getting a Refresh Token

[Chad Wood]

https://groups.google.com/g/adwords-api/c/bAgqtTr5OoY
This issue has branched into a follow up question that should probably be handled separately.

I'm investigating the process of requesting refresh tokens. I use the client library for Python titled "generate_user_credentials.py" for Desktop app flow. In truth, the app is just querying data from Ads and using it for internal reporting.

The library spins up a server that waits for a callback with a string query specifying:
1. state

2. code

3. scope

State and Scope are easy to get, as they're provided in the original oauth link used to authenticate. Code is much more difficult to reproduce, though it seems that you guys actually generate that on your end and send back in one of the page's POST requests.

So I investigated that POST request, which goes to "https://accounts.google.com/signin/oauth/consent/approval". It appears to require the following:
params:
1. authuser

headers:

1.Content-Type

2.Cookie
payload:
f.req = some_array

at = some_string

Most of these seem to be constants, but I am struggling with the some_array and some_string values.

some_array contains a token that begins with "!ChR" + a ' ∙ ' separator for a second value, and I can not reproduce it so far. It must be generated client-side, as I have checked all incoming data for this token and can not find it. I have the feeling it's a hash of something else, but when reviewing the javascript I was completely lost. I don't know javascript, so it's difficult to read (on top of there being A LOT of js to sort through).

I haven't gotten to the at variable that contains its own token yet, but I'll need to understand how this is generated as well.

Can someone help me get to a point where I can reproduce the token beginning with "!ChR"? Also the value for "at" in the payload.

Thank you

[Chad Wood]

Here's an example of the payload I want to reproduce (values changed to prevent reuse)

test_req = {
    'f.req': [
        True,
        '!ChRxa0FhVGI1X294dHwOOsPwQrVIMxIfZ3g3TmFyZklDMFlTa013TWpqbVNUV1h5Vk17D3KqZw∙AD98QVYAAAAAY3VIexEJuF7PQTpFZd2Q1Zaaw8v28eK3',
        None,
        None,
        [
            'gf.sad',
            ['scope_goes_here'],
            [],
            []
        ],
        [
            None,
            'client_id_goes_here_but_my_message_is_deleted_when_a_mock_is_included_:(',
            [' scope_goes_here '],
            '!ChRxa0FhVGI1X294dHwOOsPwQrVIMxIfZ3g3TmFyZklDMFlTa013TWpqbVNUV1h5Vk17D3KqZw∙AD98QVYAAAAAY3VIexEJuF7PQTpFZd2Q1Zaaw8v28eK3,
            None,
            None,
            None,
            None,
            None,
            None,
            None,
            'redirect_url_goes_here',
            None,
            False,
            None,
            None,
            None,
            True,
            None,
            None,
            None,
            None,
            14,
            None,
            True,
            [],
            False,
            False,
            True,
            None,
            [
                [
                     'scope_goes_here',
                    1200,
                    False,
                    False
                ]
            ],
            None,
            True,
            None,
            1,
            True,
            None,
            None,
            2,
            False,
            None,
            False
        ]
    ],
    'at': 'AFofLUWPdG5b6SxxxapUVk_BaGsQI9uqcA:1643372251920'
}

I honestly just need to know what my browser is going to generate these two variables;
'f.req'[2] ----- !ChRxa0FhVGI1X294dHwOOsPwQrVIMxIfZ3g3TmFyZklDMFlTa013TWpqbVNUV1h5Vk17D3KqZw∙AD98QVYAAAAAY3VIexEJuF7PQTpFZd2Q1Zaaw8v28Dg4
'at' ---- AFofLUWPdG5b6SxxxapPWk_JaKsQI9uqcA:1643372251920


What is it using as input, and the process to mutate it into this provided format for the POST request.

[Chad Wood]

Again, this is all sent from the OAUTH flow for Desktop Apps. It's at the last step; basically this sends and you guys respond with a URL to be used  for the callback.

[Chad Wood]

Found this:

• f.req is an array of “envelopes” for each payload in the batch. I’ll dive deeper into the structure of this later.
• at is another XSRF mitigation parameter, this time tied to the user’s Google account and paired with a UNIX timestamp.

This is helpful for the at call. I guess I just need to understand whats at index  2 of f.req

[Google Ads API Forum Advisor]

Hi Chadwood,

Thank you for reaching us out. I am Sherwin from Google Ads API support team. I hope that you are doing well today.

I can see that this is regarding OAuth Desktop and Web Application Flows. And since you have questions with regard to Oauth with Python, we highly suggest you to reach out the client library owner of Python using this link.

Kind regards,

Sherwin Vincent Google Ads API Team

ref:_00D1U1174p._5004Q2gPExm:ref

[Chad Wood]

Thanks Sherwin, but the technical part that I struggle with has nothing to do with the Python library. It actually has to do with some proprietary piece of Google tech used during the OAUTH procedure.
All the library does is spin up a server, produce a link for OAUTH verification, and makes the server wait for a call back. I want to discuss what happens inside that link for OAUTH verification.

During the verification process there are a few steps:

1. Open the link
2. Select your Google account

3. Hit "Continue" after reviewing the app permissions

4. A callback automatically occurs to the server span up by the Python library

 At step 3 (having to do with Google-not the Python library), a payload is sent to this address "https://accounts.google.com/signin/oauth/consent/approval"
That payload includes (most importantly) two variables; "f.req" and "at". It would seem that "at" is an XSRF mitigation parameter, tied to the user’s Google account and paired with a UNIX Timestamp

Focusing on just variable "f.req" right now, it takes on the structure of an array and includes 1 token that I can not figure out how to reproduce.
The 1 token in question begins with "!ChR", and the browser's webpage uses Javascript to generate it.
That "!ChR"  is then placed into the "f.req" array, which is placed inside the payload to "https://accounts.google.com/signin/oauth/consent/approval".
Finally, once sent via POST, the "https://accounts.google.com/signin/oauth/consent/approval" server will respond with a link to be used for the callback.

I want help reproducing the "!ChR" thing manually.

[Chad Wood]

I want to better clarify myself.

Yes, I am using this library: https://developers.google.com/google-ads/api/docs/samples/generate-user-credentials#python
However, that is merely the door into another process (the OAUTH process). Using this library, I can generate a link to perform OAUtH verification.

It produces a link like so:

I click on that link to access the OAUTH procedure. Following, I get to this point:

I've split the above photo into 3 sections. In section one, hitting "Continue" results in section 2's POST request.
Reviewing section 3 shows that POST's payload. It contains a variable entitled "f.req", which is an array.

At index 1 (after 'true') in the payload, there is a token that always begins with "!ChR". I have searched all incoming data and can not find this "!ChR" token anywhere. So I assume the token MUST be generated client-side.

I want to be able to generate these tokens for my workflow, just like my browser already does automatically.
but there is no documentation on this token.

I just need to understand how the token is generated and how I can produce one myself.

[Google Ads API Forum Advisor]

Hi Chadwood,

Thank you for getting back to us.

Thank you for providing us your main goal. Please note that Google the access token and refresh token is generated by Google side. Based on this article "The authorization sequence begins when your application redirects a browser to a Google URL; the URL includes query parameters that indicate the type of access being requested. Google handles the user authentication, session selection, and user consent. The result is an authorization code, which the application can exchange for an access token and a refresh token.

You may check this article for more information.

[Chad Wood]

Thanks, and you are right that the access token and refresh token is generated by Google side.

About this: "the URL includes query parameters that indicate the type of access being requested"

This is also correct. Here's what the url would look like:

https:// accounts. google. com/o/oauth2/auth?
    response_type=code&
    client_id=995929737391-l3og1l1d44gpf4985e0l7elkk4b26nn.apps.googleusercontent.com&
    redirect_uri=http%3A%2F%2F127.0.0.1%3A8080&
    scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadwords&state=efcck0648b44377bedj20e97d2eca2745ba5979cf2cfa4sk30d2da6365sk2681&
    access_type=offline&prompt=consent&

    include_granted_scopes=true

See the parameters it's talking about here are:  response_type, client_id, redirect_uri, scope,  access_type, and  include_granted_scopes

What I'm talking about goes deeper into the process than any documentation covers though. See, on my side of things, the browser actually generates a "code" that it eventually sends to your server once I hit the "Confirm" button.
That "code" is what I showed in the picture from my last message (section three). This code is basically verified by your server, as a type of security feature I assume. As it would likely be very difficult for a bad actor to get all the input data needed to reproduce that "code".

But I am not a bad actor. I basically want to reproduce that "code" inside my own personal program--not inside a browser. This way, I can send authorization requests to your server manually.
I have tested ripping the payload from the browser and sending it in my program. In this case, your server WILL respond with the redirect URL containing the refresh token. This is confirmed working.
The only issue is that it requires me to manually rip the payload, which defeats the entire purpose.

Now, I can reproduce most of the payload. The only part I can not reproduce is the "code" I am referring to. It is at index 1 of the "f.req" variable inside the payload.
Somehow, my browser is generating this "code" using the page's JavaScript instruction to do so. Then it places the "code" inside an array labeled "f.req" and sends that to your server. Your server validates it and responds with the refresh token stuff.

I only want to understand how that "code" is created. What is my browser doing? Is it hashing cookie variables? If so, what type of hash and which variables?
Again, it is the token that always begins with "!ChR" at index 1 of variable "f.req" in the payload that gets sent to your authorization server for the refresh token.

Can I have more insight on how my browser generates this code/token? I tried reviewing the JavaScript in the page, but it's a mile long and none of the function names appear meaningful--so it is very difficult to figure out,

[Google Ads API Forum Advisor]

Hi Chad,

Thanks for your patience on this and for your detailed description. Please allow me to raise this question to our team. We will provide an update on this shortly.

Thanks,

Matt Google Ads API Team

ref:_00D1U1174p._5004Q2gPExm:ref

[Chad Wood]

Thanks. I'll keep an eye out until then.

[Google Ads API Forum Advisor]

Hello,

You can check out the OAuth 2.0 documentation, particularly this page: https://developers.google.com/identity/protocols/oauth2/javascript-implicit-flow#oauth-2.0-endpoints

This goes into a lot of detail on how to fetch access tokens using just curl requests, which should be translatable into any context you want. One sentence I found of particular note in my reading through it: "If the user approves the request, then the response contains an access token."

So it seems like the access token is generated on the server side, not the client side. Perhaps it's encrypted, which is why sniffing your own network traffic didn't produce the results you were looking for? My team is not experts on OAuth in particular, so hopefully this resource is able to help.

Regards,
Mike, Google Ads API Team

ref:_00D1U1174p._5004Q2gPExm:ref