New Google Storage Limits and GAM

1,015 views
Skip to first unread message

Ullfig, Roberto Alfredo

unread,
Apr 27, 2022, 2:03:03 PM4/27/22
to google-ap...@googlegroups.com
Hello, Google is implementing new storage quotas in May and we'd like to use GAM to monitor our situation:


Per their document only new or modified data counts towards the quota. Can GAM get this information? Can GAM also determine how much of our data is long and untouched? We have nearly 400TB but we've been on the service for over a decade so we expect that most of that data is not new or will actively be modified after May. The admin GUI gives an "estimated" number that is disingenuous as it assumes all our data will be new or modified.


---
Roberto Ullfig - rul...@uic.edu
Systems Administrator
Enterprise Applications & Services | Technology Solutions
University of Illinois - Chicago

Ross Scroggs

unread,
Apr 27, 2022, 3:10:04 PM4/27/22
to google-ap...@googlegroups.com
Roberto,

This solution requires Advanced GAM.

This example shows the file counts for all users in the Test OU; replace with one of your OUs or you can say "all users" (which will take a while)

gam config auto_batch_min 1 redirect stdout - multiprocess redirect csv ./FileSizeCounts.csv multiprocess ou /Test print filecounts showsize query "modifiedTime < '2021-05-02T00:00:00Z'"


Unfortunately, the Drive API doesn't return size information for Google Docs/Sheets/Forms/..., so the Size here is for files that Google considers to have binary content, e.g., pdt/jpeg/txt/...


Ross


--
You received this message because you are subscribed to the Google Groups "GAM for Google Workspace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/PH7PR13MB5480AADD949F899F1E6CFF72B0FA9%40PH7PR13MB5480.namprd13.prod.outlook.com.


--

Ullfig, Roberto Alfredo

unread,
Apr 27, 2022, 3:47:00 PM4/27/22
to google-ap...@googlegroups.com
I am getting size data with standard gam and the data appears to match what the GUI says:

gam user ne...@uic.edu show filelist name modifieddate filesize

I then look in the GUI and Drive Data usage matches the sum. I have a script that parses the data GAM returns and separates it by year with a grand sum.

any reason to not believe this info is correct?

---
Roberto Ullfig - rul...@uic.edu
Systems Administrator
Enterprise Applications & Services | Technology Solutions
University of Illinois - Chicago
From: google-ap...@googlegroups.com <google-ap...@googlegroups.com> on behalf of Ross Scroggs <ross.s...@gmail.com>
Sent: Wednesday, April 27, 2022 2:09 PM
To: google-ap...@googlegroups.com <google-ap...@googlegroups.com>
Subject: Re: [GAM] New Google Storage Limits and GAM
 

Ross Scroggs

unread,
Apr 27, 2022, 4:14:47 PM4/27/22
to google-ap...@googlegroups.com
Roberto,

Do the following:
gam user ne...@uic.edu show filelist name modifieddate filesize mimetype

Are you getting a fileSize value for Google Docs/Sheets?

Ross

To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/PH7PR13MB54802B5E92D7C99D78932DA3B0FA9%40PH7PR13MB5480.namprd13.prod.outlook.com.


--

Ross Scroggs

unread,
Apr 27, 2022, 4:28:23 PM4/27/22
to google-ap...@googlegroups.com
Roberto,

Note that fileSize is not populated for the Google Docs

$ gam user ro...@s.jaylee.us print filelist title mimetype filesize

Getting files for ro...@s.jaylee.us...

Got 3 Files for ro...@s.jaylee.us...

Owner,title,mimeType,fileSize

ro...@s.jaylee.us,SharedWithAnyoneLinkDoc,application/vnd.google-apps.document,

ro...@s.jaylee.us,MotD,application/vnd.google-apps.document,

ro...@s.jaylee.us,panther.jpg,image/jpeg,51220


Ross



--

Ross Scroggs

unread,
Apr 27, 2022, 4:46:10 PM4/27/22
to google-ap...@googlegroups.com
Roberto,

From the document link you posted:
You can see how much storage your entire organization or a single account is using.

See the storage usage for your entire organization

Important: The total storage won't include Google Docs, Sheets, Slides, Forms, Drawings, or Jamboard files created prior to May 2, 2022.

It will be interesting to see next week if fileSize is now going to be populated for Google Docs/Sheets/...


Ross

--

Ullfig, Roberto Alfredo

unread,
Apr 28, 2022, 9:46:13 AM4/28/22
to google-ap...@googlegroups.com
Right, that page shows total storage used right now but none of that counts towards our quota. Do you know if when May 2 arrives, we would be able to see both total storage used and total storage used that counts against our quota?

Yes, I see that Google Docs file sizes are not being returned by standard GAM.

This site https://edustoragereport.withgoogle.com/ appears to include ALL documents and has been referenced in our discussions with Google. The value there is 18 TB larger than what's on our storage page so that would appear to account for all the Docs, etc... which comes out to about 5% of our total usage. So standard GAM is reporting ~95% of our total.

As I said, we've been on Google since 2008 and have accumulated a lot of stale (or read-only) data over the years. GAM can help identify how old most of our data is. Hopefully Google will supply us with this information soon as well. Thanks for the info!

---
Roberto Ullfig - rul...@uic.edu
Systems Administrator
Enterprise Applications & Services | Technology Solutions
University of Illinois - Chicago
Sent: Wednesday, April 27, 2022 3:45 PM

Ross Scroggs

unread,
Apr 28, 2022, 11:21:52 AM4/28/22
to google-ap...@googlegroups.com
Roberto,

Good question. You can use these two queries to separate the two worlds:
query "createdDate < '2021-05-02T00:00:00Z'"
query "createdDate >= '2021-05-02T00:00:00Z'"

The edustorage report is clearly not using the public API to get the sizes; Google sites use internal APIs and they may well be returning sizes for all files.

It'll be interesting to see what's available on Tuesday.

Kim Nilsson

unread,
May 2, 2022, 11:49:53 AM5/2/22
to GAM for Google Workspace
This thread will indeed be interesting to follow.

Ross Scroggs

unread,
May 3, 2022, 3:49:08 PM5/3/22
to google-ap...@googlegroups.com
Roberto,

I created a new Google Doc today and the API is not returning a size for it.

What does the storage report look like for you today?

Ullfig, Roberto Alfredo

unread,
May 5, 2022, 2:20:17 PM5/5/22
to google-ap...@googlegroups.com
I don't see a difference. The reports just show total usage and don't give an indication of how close you're to your quota. One problematic thing we noticed is that Shared Drives can't be assigned quotas so if a user hits their quota there's nothing stopping them from putting all their data on a shared drive. The contents of a shared drive are not owned by any particular user.

---
Roberto Ullfig - rul...@uic.edu
Systems Administrator
Enterprise Applications & Services | Technology Solutions
University of Illinois - Chicago
Sent: Tuesday, May 3, 2022 2:48 PM

Ullfig, Roberto Alfredo

unread,
May 5, 2022, 2:22:44 PM5/5/22
to google-ap...@googlegroups.com
Oh sorry, but GAM does return google doc file size now.

rul...@uic.edu,New File Quota,2022-05-05T18:20:51.612Z,1024,application/vnd.google-apps.document

---
Roberto Ullfig - rul...@uic.edu
Systems Administrator
Enterprise Applications & Services | Technology Solutions
University of Illinois - Chicago
From: Ullfig, Roberto Alfredo <rul...@uic.edu>
Sent: Thursday, May 5, 2022 1:20 PM
Reply all
Reply to author
Forward
0 new messages