over 225,000 users - gam print users - memory error

[raydlevel5]

I get an error message after waiting about two hours for the 'gam print users' command to finish executing.  We probably have close to 225,000 users.

 

I'm attempting to generate a *.csv file for all suspended accounts in our Google Apps domain.  I need the list so I can identify & delete suspended account...

 

c:\gam>gam print users firstname lastname username ou suspended > suspended_uses.csv
Getting all users in the ORGANIZATION.NET organization (may take some time on a
arge Google Apps account)... done.
Getting details for all users... (may take some time on a large Google Apps acc
unt)... Traceback (most recent call last):
  File "gam.py", line 6597, in <module>
  File "gam.py", line 4970, in doPrintUsers
  File "gdata\apps\multidomain\service.pyo", line 106, in RetrieveAllUsers
  File "gdata\apps\service.pyo", line 517, in _GetPropertiesList
MemoryError

[Jay Lee]

I've had no problems with the command on a Linux machine with 8gb of RAM and a 64-bit version of Python. How much RAM do you have and what OS are you running? Is it 64-bit?

Regards,

Jay Lee
LCS Deployment Lead   |  ⚡ Dito

☎ (267) 712-9533  |  ✉ j...@ditoweb.com

--
You received this message because you are subscribed to the "Google Apps Manager" group.
To post to this group, send email to
google-ap...@googlegroups.com
To unsubscribe from this group, send email to
google-apps-man...@googlegroups.com
For more options, visit this group at
https://groups.google.com/forum/#!forum/google-apps-manager

[Dave Bailey]

I've found it's best to run GAM in a Powershell.exe shell for the really big jobs.  I don't know this definitively but it seems to use system memory better.

--

David Bailey

Multnomah County 

Google BSA

General Government/Open Source

david....@multco.us | 503.988.6631

[raydlevel5]

Windows 7, 64bit, 8 GB.  I dumbed-down my query to just "gam print users username suspended > suspended.csv", and got a different error (see below).

 

Why can't I get a list of suspended users?  Why does it keep throwing back error messages?  What do I need to do to resolve this?

 

 

 

 

c:\gam>gam print users username suspended > suspended.csv
Getting all users in the organization.net organization (may take some time on a l
arge Google Apps account)... Traceback (most recent call last):

  File "gam.py", line 6597, in <module>

  File "gam.py", line 4923, in doPrintUsers
  File "gdata\apps\orgs\service.pyo", line 152, in RetrieveAllOrganizationUsers
  File "gdata\apps\service.pyo", line 514, in _GetPropertiesList
  File "gdata\apps\service.pyo", line 482, in AddAllElementsFromAllPages
  File "gdata\service.pyo", line 1108, in Get
gdata.service.RequestError: {'status': 500, 'body': '<?xml version="1.0" encodin
g="UTF-8"?>\r\n<AppsForYourDomainErrors>\r\n  <error errorCode="1000" invalidInp
ut="" reason="UnknownError" />\r\n</AppsForYourDomainErrors>\r\n\r\n', 'reason':
 'Internal Server Error'}

[raydlevel5]

OK, so it sounds like you guys want me to use PowerShell or Python.  I've never used Python before.... are there any special considerations?

[Jay Lee]

Are you using the latest GAM 2.53 release?

Can you try running the experimental x64 release of GAM and see how it does?

Regards,

Jay Lee
LCS Deployment Lead   |  ⚡ Dito

☎ (267) 712-9533  |  ✉ j...@ditoweb.com

On Mon, Oct 1, 2012 at 4:45 PM, raydlevel5 <raydl...@vblocal.net> wrote:

OK, so it sounds like you guys want me to use PowerShell or Python.  I've never used Python before.... are there any special considerations?

--

[Cliff Wakefield]

I know it doesn't solve the error but an accounts report will tell you who is suspended within your domain?

[Cliff Wakefield]

Weird I see the same under GAM 2.54 on OS X 10.8.0

$ python gam.py print users username suspended

Getting all users in the rmit.edu.au organization (may take some time on a large Google Apps account)... Traceback (most recent call last):

  File "gam.py", line 6597, in <module>

    doPrintUsers()

  File "gam.py", line 4923, in doPrintUsers

    all_users = org.RetrieveAllOrganizationUsers()

  File "/Users/cliff/Downloads/gam-2.54/gdata/apps/orgs/service.py", line 152, in RetrieveAllOrganizationUsers

    all_users = self._GetPropertiesList(uri)

  File "/Users/cliff/Downloads/gam-2.54/gdata/apps/service.py", line 514, in _GetPropertiesList

    property_feed, gdata.apps.PropertyFeedFromString)

  File "/Users/cliff/Downloads/gam-2.54/gdata/apps/service.py", line 482, in AddAllElementsFromAllPages

    next_feed = self.Get(next.href, converter=func)

  File "/Users/cliff/Downloads/gam-2.54/gdata/service.py", line 1108, in Get

    'reason': server_response.reason, 'body': result_body}

gdata.service.RequestError: {'status': 500, 'body': '<?xml version="1.0" encoding="UTF-8"?>\r\n<AppsForYourDomainErrors>\r\n  <error errorCode="1000" invalidInput="" reason="UnknownError" />\r\n</AppsForYourDomainErrors>\r\n\r\n', 'reason': 'Internal Server Error'}

[raydlevel5]

==follow-up==

 

-- I will have to try the 64-bit version as mentioned

-- I get the memory error even with PowerShell

-- I don't want to run "gam report accounts" as I'd have to distinguish & parse Suspended accounts from Active accounts

-- I'm running the latest version of GAM (I'm able to do smaller queries and create user accounts, if that's a confident test of connectivity)

-- Can I narrow my query to specific ORGs as to lighten the load?

-- How do you use Python?  Is it recommended over PowerShell?