Awkward API format - some feedback

[Phil Brock]

Hi,

I've just finished integrating this API and have some feedback/thoughts - I can't find any other channel to provide them, so here they are for your consideration.

I'm wondering why the API chooses to return all the different objects (header, footer, rows) in a list, rather than in a map/object?

E.g. the current format:

[

  {

    "header": {

      "dateRange": {},

      "localizationSettings": {}

    }

  },

  {

    "row": {

      "dimensionValues": {},

      "metricValues": {}

    }

  },

  {

    "row": {

      "dimensionValues": {},

      "metricValues": {}

    }

  },

  {

    "footer": {

      "matchingRowCount": "2"

    }

  }

]

It's pretty unwieldy to work with as a list, why not use a map?

Proposed format:

{

  "header": {

    "dateRange": {},

    "localizationSettings": {}

  },

  "rows": [

    {

      "dimensionValues": {},

      "metricValues": {}

    },

    {

      "dimensionValues": {},

      "metricValues": {}

    }

  ],

  "footer": {

    "matchingRowCount": "2"

  }

}

I'm actually transforming the returned responses into this format, and it's a lot more usable as I don't have to constantly check if my header/footer/row is *actually* a header/footer/row.

Further to that: JSON supports numeric types, so why is "matchingRowCount" returned as a string?

Also, the date object in the request makes no sense - the API returns dates in a concise format, e.g. ""20201122" but if sent through in a request this has to be passed in as an unwieldy JSON construction (made worse as it comes as a range, so there's a matching "endDate"):

"startDate": {

  "year": 2020,

  "month": 11,

  "day": 22

}

Why not just take in a concise date string? The API user is forced to consider your date format anyway, it's not any more work to pass it in with the request too, and would be a lot more compact.

Here's how it currently looks in Python (start_date/end_date are date objects):

date_range = {

  'startDate': {

    'year': start_date.strftime('%Y'),

    'month': start_date.strftime('%m'),

    'day': start_date.strftime('%d')

  },

  'endDate': {

    'year': end_date.strftime('%Y'),

    'month': end_date.strftime('%m'),

    'day': end_date.strftime('%d')

  }

}

Here's how it could look:

date_range = {

  'startDate': start_date.strftime('%Y%m%d'),

  'endDate': end_date.strftime('%Y%m%d')

}

There are other awkward things to the API, but these were some of the bigger things, I hope you'll give them some consideration. A key point is that your API does not have to match Google's/AdMob's internal data representations, it's allowed to be nice/easy to work with, in fact, that's kind of the point.

Cheers,

Phil

[Maksym]

Hi Phil, 

I believe the list of messages returns because the report is a stream and not a single message response. Such an approach allows receiving data immediately, and your code can start process data without waiting for the entire stream to finish. Also, the single message response would require pagination, could slow down the whole report downloading.  Probably having both report types (streaming and single-message pagination one) could cover different needs.

Regarding the 'matchingRowCount' footer field, the documentation  https://developers.google.com/admob/api/v1/reference/rest/v1/ReportFooter declares returns format as a string treats as int64. I've just googled a topic related to the problem:

https://github.com/protocolbuffers/protobuf/issues/1823 or https://groups.google.com/g/protobuf/c/4-BY-k-Lk-g, https://stackoverflow.com/questions/13502398/json-integers-limit-on-size. I hope it helps a little. 

I'm kind of agree with the point that date could be better, but at the same time I like how the request look in my code:

```python

...
from datetime import date
from datetime import timedelta

...

today = date.today()
week_ago = today - timedelta(days=7)

date_range = {'startDate': {'year': week_ago.year, 'month': week_ago.month, 'day': week_ago.day},
                          'endDate': {'year': today.year, 'month': today.month, 'day': today.day}}
dimensions = ['DATE', 'APP', 'PLATFORM', 'COUNTRY']
metrics = ['ESTIMATED_EARNINGS', 'IMPRESSIONS', 'CLICKS', 'AD_REQUESTS', 'MATCHED_REQUESTS']
sort_conditions = {'dimension': 'DATE', 'order': 'DESCENDING'}

report_spec = {'dateRange': date_range,
                           'dimensions': dimensions,
                           'metrics': metrics,
                           'sortConditions': [sort_conditions]}

...
```

Thanks

[Phil Brock]

Hi Maksym,

Thanks for your reply.

I can almost accept the report as a stream argument, however this is the 29th reporting API I've integrated and is definitely one of the hardest to work with (measured by lines of code required to work with it, it is *the* hardest). The reports are capped at 100,000 rows, and the google-api-python-client doesn't return the response until the whole thing has been downloaded, so any gains from streaming are lost and the entire thing ends up in memory anyway. If the reports came in a more concise format a lot of the issues around size could be ameliorated - AdMob's is one of the most wasteful reporting formats I've come across.

Regarding the int64 vs string thing - again, the reports are capped at 100,000 rows, I don't think JSON's numbers are going to have any trouble representing that.

Cheers,

Phil

[Phil Brock]

Regarding the report footer and the "matchingRowCount" the doc says this about it:

"string (int64 format)

Total number of rows that matched the request.

Warning: This count does NOT always match the number of rows in the response. Do not make that assumption when processing the response."

It's hard to know what to do with it. What's its actual purpose? Is it just for warning in cases where more than 100,000 rows matched but not all were returned? If the number of rows is less than 100,000 will the number match the row count? If I try and alert on cases where that number is > 100,000 is that a legitimate use for it?

More detail would be great thanks!

Phil

[Maksym]

Yeah, I agree that some language's API client libraries do not work well/optimal with the streaming. Also, streaming is a more challenging development model, and the benefits it could provide does not fit everyone. But at the same time, it still relatively easy to integrate (see examples on developers getting started page: https://developers.google.com/admob/api/v1/getting-started#python-client-library or on Github: https://developers.google.com/admob/api/v1/libraries, https://github.com/mprokhorenko/googleads-admob-examples).

I treat the "matchingRowCount"  as the number of records that fits your request on the server-side, which could be more than the limit of 100K records. It signals me that not all the data loaded and the request's range should be narrowed down — something like it.

Thanks,

Maksym

[Phil Brock]

Thanks, I do have the integration working now.

I referred to the examples, but they do very little with the data after executing the request, e.g.:

# Display results.

for report_line in result:

  print(report_line)

(Source: "Generate a network report." https://developers.google.com/admob/api/v1/getting-started#python-client-library)

It may as well print "Hello world!", no one is retrieving these reports to output them to a terminal.

After being stung by the AdSense APIs I tend to use a specific query with a very limited set of dimensions/metrics, so hopefully the 100k limit won't ever be a problem. I've added a check just in case.

Cheers,

Phil

[AdMob API Forum Advisor]

Hi Phil,

Thanks for providing feedback on the AdMob API, we value your input and will take it into consideration.

All the best,

Amira Badreldin AdMob API Team

 

ref:_00D1U1174p._5004Q28m8RK:ref