Gmail to Groups best practice

[Paul Zagoridis]

Thanks for GAM and Advanced GAM. Amazing!

We migrate ex-employees email to a Google Group but I'm looking for a way to better automate it.

I installed GAM on Google Cloud Shell for the speed increase instead of shunting data to up and down our 100/40 fibre connection. I could host GAM on one of our VPS servers in a USA data centre but I'd expect Google Cloud Shell would still be faster. The Google cloud session times-out every 500-1500 messages and/or the command fails with 

googleapiclient.errors.MediaUploadSizeError: Media larger than: 26214400

Which I understand means the attachments make the individual email bigger than what google groups can handle.

Ex-employees have 1gb-90gb of email depending on how long they've been with the company.

command I'm using is

gam user user...@company.tld archive messages grou...@company.tld doit

1. I can't work out how to limit migration to say 500 email batches then start the second and subsequent batch. obviously  [max_to_forward <Number>]
   
2. should i be prefetching a list of message_ids and then feeding back in 500 message id batches? If so how? Alternatively identify emails >25mb, copy and remove the attachment to Drive, then run the archive command?
3. What is the best practice for this migration? with some logging for error. e.g. chats generate this runtime error, but the job continues
   User: user...@company.tld, Message: 1750cf0909b88d63, Archive Failed: Unable to parse the raw message (1487/4880)

Regards

Paul

[Emily Wilkinson]

I was very happy to see archive messages show up recently (was previously using GYB with --action restore-group).

Our current workaround for the 25M limit is to do things piecemeal:

• gam user $SRC_USER archive messages $ARCHIVE_GROUP query "smaller:25m" doit to archive the smaller messages,
• a GYB job to take a full backup of the entire mailbox (or if you don't need to back up everything, you could filter this to just the >25m emails)
• gam user $SRC_USER print messages query "larger:25m" showbody showattachments todrive to create a Google Sheet that's at least vaguely searchable for message contents, store it on a shared drive
• gam user $SRC_USER print messages query "larger:25m" saveattachments to dump attachments into the shared drive alongside the Google Sheet

It's crude, but sort of works. If anyone ever needs to see an original message, I use GYB to restore from backup to a temporary user mailbox.

I have been toying with putting together a script along the lines of:

• Use GYB --action backup to download any messages over 25m to a temporary folder
• Iterate through each message:
  • While message size >25m:
    • Extract the largest attachment and save it to a shared Google Drive location - (probably something like $ARCHIVE_DRIVE/$SRC_USER/Extracted Mail Attachments/$MESSAGE_ID/$FILENAME)
    • Remove the attachment from the message
    • Generate a HTML file "Removed Attachment - $FILENAME" that contains a brief explanation and a Google Drive link to the original attachment
    • Attach the HTML file to the message
  • Replace the downloaded message with the shrunk one
• Use GYB --action restore-group to upload the shrunk messages to $ARCHIVE_GROUP

If Ross was feeling especially generous, the archive messages command could potentially be extended to modify the messages in-flight.

Maybe something like gam <UserTypeEntity> archive messages <GroupItem> pruneoversizeattachments [removednotice (drivelink|text <String>)] [toteamdrive <SharedDriveEntityAdmin>] where the attachments could either be preserved or discarded?

Or a standalone command that could be run against a mailbox to pre-treat it before archiving?

Kind regards,

Emily

---

DISCLAIMER: This message and its contents are confidential. If you received this message in error, do not use or rely upon it. Instead, please inform the sender and then delete it. Thank you.

[Ross Scroggs]

Emily,

Send me a Meet/Zoom and we'll see how generous I'm feeling about modifying archive messages.

I'm in California (PDT).

Ross

--
You received this message because you are subscribed to the Google Groups "GAM for Google Workspace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-apps-man...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/77740477-8a02-49a3-9048-ba367fd5b05cn%40googlegroups.com.

--

Ross Scroggs

ross.s...@gmail.com

[Semih Bekiroğlu]

How can I move a user's emails into a group email? Under the Conversations tab on groups.google.com. (the email is as is)
Migrate Gmail account to conversations on groups.google.com.

21 Mart 2023 Salı tarihinde saat 17:44:00 UTC+3 itibarıyla Ross Scroggs şunları yazdı:

[Ross Scroggs]

See: https://github.com/taers232c/GAMADV-XTD3/wiki/Users-Gmail-Messages-Threads#archive-messages

Ross

----

Ross Scroggs

Ross.S...@gmail.com

To view this discussion on the web visit https://groups.google.com/d/msgid/google-apps-manager/42c572a1-a61c-4f85-baeb-92cf52ab019fn%40googlegroups.com.