Speed up rename

692 views
Skip to first unread message

Graham Ingleby

unread,
Nov 28, 2017, 6:18:20 AM11/28/17
to GAM for G Suite
Hi

We are going through a rename for approx 800 users and 200 groups
Using gam in a batch file (ie 1 command line per user rename) is averaging 8-10 seconds per rename, so we are looking at a way to speed this up as current estimates would be approx 3 hours

So my question is how best to do this? As I see it I have a couple of options but need to understand the pros and cons of each:
- Run GAM using a CSV instead of in a batch file, as I understand it this runs multiple threads instead of 1 at a time, is this correct?
- Install GAM on multiple (say 3) machines and run some renames on each machine - am I going to hit any API limits like this? If I use this option should I use the same GAM project or create multiple projects?

Any other alternatives I have missed?

Thanks
Graham

Ross Scroggs

unread,
Nov 28, 2017, 8:16:13 AM11/28/17
to google-ap...@googlegroups.com
Graham,

Both batch files and CSV files are processed in parallel.
In standard GAM, set the environment variable GAM_THREADS to the number of parallel processes desired.
In my advanced GAM, set num_threads in gam.cfg.
When you did your "gam batch <Filename>", the first line of ouput should have been something like this:
Using N processes...
What was the value of N?

This message is for the named person's use only.  If you receive this message in error, please delete it and notify the sender.  Ancoris reserves the right to monitor all e-mail communications through its networks. Ancoris Limited, Registered in England Number: 04830784, Registered address: 5a Frascati Way, Maidenhead, Berkshire SL6 4UY. Trading Address: Lily Hill House, Lily Hill Road, Bracknell, Berkshire RG12 2SJ



--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.
To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/6a55a640-c27b-457d-8ef7-0f2d997b1df9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Graham Ingleby

unread,
Nov 29, 2017, 3:48:10 AM11/29/17
to GAM for G Suite
Hi Ross

When I said we tested with a batch file, I meant a DOS type batch file - example.bat containing a line for each rename - so as I understand it this would run gam once for each line and only start the next line after the first completed

From what you have said, using the GAM CSV or GAM BATCH commands would allow parallel processing and so speed things up, and tweaking the GAM_THREADS would allow more or less threads to run in parallel, so this needs testing for maximum flow without hitting limits

Cheers
Graham
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.

Andrew Penner

unread,
May 1, 2018, 5:28:58 PM5/1/18
to GAM for G Suite
Hi Ross:
I'm using powershell to import csv files to suspend/restore a couple of hundred accounts; how can I tell how many threads are running?

- Andrew
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.

Ross Scroggs

unread,
May 1, 2018, 6:39:19 PM5/1/18
to google-ap...@googlegroups.com
Andrew,

What version of Gam do you have?

gam version

Ross

To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.
To post to this group, send email to google-apps-manager@googlegroups.com.

Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/417e4c22-e806-4a2b-b131-4dae73d199a7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Unhappy Buckaroo

unread,
Aug 14, 2018, 5:41:55 PM8/14/18
to GAM for G Suite
Sorry for the delay...
Currently running latest gamadv version 4.57.19.
The initial output of the most recent session said 5 threads; I just have to update the gam.cfg file to change this, correct? What's the upper limit?
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.
To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.



--

Ross Scroggs

unread,
Aug 14, 2018, 5:54:54 PM8/14/18
to google-ap...@googlegroups.com
Andrew,

Latest version is 4.60.04 but that's not an issue here.

Jay suggests no more than 20.
Make 20 threads be the default: gam config num_threads 20 save
Then: gam csv file.csv gam ...

Make 20 threads for a single command: gam config num_threads 20 csv file.csv gam ...

Ross


To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.



--


  Find us on Facebook:
  Follow us on Twitter: 

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/65bccd03-9f31-40d8-ae9d-90a1e8b2a1c0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--

Andrew Penner

unread,
Aug 14, 2018, 6:09:10 PM8/14/18
to google-ap...@googlegroups.com
Thanks Ross!
I'm timing a workflow using 5 threads at the moment, but I have to run it again in a couple of weeks so I'll try it with 20 then.

______________________________________________________
Andrew Penner, Technical Analyst III (A+, MCP, MCDST)
Office: 780-460-3712 (333)
Mobile:780-913-7576
60 Sir Winston Churchill Ave
St. Albert, AB T8N 0G4




You received this message because you are subscribed to a topic in the Google Groups "GAM for G Suite" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-apps-manager/V9oF5IldCE8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/CAJkvRS-1U5ZkDHQy39_Cj4aa%2BDqjSWiKPMRVn4zhGD5mx53PDg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Unhappy Buckaroo

unread,
Aug 22, 2018, 5:26:48 PM8/22/18
to GAM for G Suite
Hi Ross:
I've updated GAM so i'm running 4.60.
I'm getting an error when trying to change the number of threads:
"Error: config is not a valid argument for gam"
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.



--


  Find us on Facebook:
  Follow us on Twitter: 

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-manager+unsub...@googlegroups.com.


--

Jay Lee

unread,
Aug 22, 2018, 5:29:02 PM8/22/18
to google-ap...@googlegroups.com

To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.



--


  Find us on Facebook:
  Follow us on Twitter: 

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.


--


  Find us on Facebook:
  Follow us on Twitter: 

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/6f96717a-b013-446d-b0ec-e353eff0f1f9%40googlegroups.com.

Ross Scroggs

unread,
Aug 22, 2018, 5:30:40 PM8/22/18
to google-ap...@googlegroups.com
Unhappy Buckaroo,

That was an Advanced GAM (https://github.com/taers232c/GAMADV-XTD) command.
For basic GAM (4.60) you have to set an environment variable:

export GAM_THREADS=10

Ross

To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.



--


  Find us on Facebook:
  Follow us on Twitter: 

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.


--


  Find us on Facebook:
  Follow us on Twitter: 

--
You received this message because you are subscribed to the Google Groups "GAM for G Suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/6f96717a-b013-446d-b0ec-e353eff0f1f9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--

Andrew Penner

unread,
Aug 22, 2018, 5:52:06 PM8/22/18
to google-ap...@googlegroups.com
Thanks Ross.
Thanks Jay.

______________________________________________________
Andrew Penner, Technical Analyst III (A+, MCP, MCDST)
Office: 780-460-3712 (333)
Mobile:780-913-7576
60 Sir Winston Churchill Ave
St. Albert, AB T8N 0G4



You received this message because you are subscribed to a topic in the Google Groups "GAM for G Suite" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-apps-manager/V9oF5IldCE8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-apps-man...@googlegroups.com.

To post to this group, send email to google-ap...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-apps-manager.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/CAJkvRS_y%3DQLbZxk10a1QEF8RbTypGT-sY8ovMZ67e6cw-2ferw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages