GYB backup failing with Gmail API 429 (rateLimitExceeded) + “sqlite3.OperationalError: database is locked”

[Dale Courtney]

Hi all,

I’m running a GYB version 1.80 Gmail backup using a service account, and the job consistently fails partway through with two errors:
1. Gmail API throttling:

“HttpError 429 … returned ‘Too many concurrent requests for user.’ … reason: ‘rateLimitExceeded’”

2. SQLite lock:

“sqlite3.OperationalError: database is locked”

followed by:

“Failed to execute script ‘gyb’ due to unhandled exception”

Redacted command structure (same flags/flow as my run):

/path/to/gyb –email us...@mydomain.com –local-folder ~/Downloads/GYB-GMail-...@mydomain.com –service-account

Context (from the run output):
• Total message IDs: ~71k
• Already backed up: ~56k
• Remaining to back up: ~15k
• Failure occurs during message fetch (GET users/me/messages/{id}?format=raw…).

Questions:
• Is there a recommended way to reduce concurrency / request rate in GYB for backups (especially with service accounts) to avoid the 429?
• Any known cause/fix for the SQLite “database is locked” error in GYB (and whether it’s triggered by the 429 retry behavior)?
• Any specific flags, config, or best practices you’d recommend to make this backup complete reliably?

If there’s any diagnostic output you want (with IDs/domains redacted), tell me what to capture and I’ll send it over.

Thanks,
Dale

Got Your Back 1.80

jaylee.us/gyb

Jay Lee - jay...@gmail.com

Python 3.12.1 64-bit final

google-api-client 2.114.0

macOS-26.2-x86_64-i386-64bit x86_64

Path: /Users/dale.courtney/gyb

ConfigPath: /Users/dale.courtney/gyb

OpenSSL 3.2.0 23 Nov 2023

gmail.googleapis.com connects using TLSv1.3 TLS_AES_256_GCM_SHA384

[Jay Lee]

- You're using an old version. Upgrade

- I'm guessing you're trying to store GYB backups directly to a shar d/mapped/network drive. Dont. SQLite doesn't like it. Finish the backup to local storage then copy all of it to network drive. Copy it back to local storage for restore.

- are you trying to backup multiple users concurrently? --batch-size defaults to 10 for backup actions. You can try reducing that.

--
--
You received this message because you are subscribed to the Google
Groups "Got Your Back: Gmail Backup" group.
To post to this group, send email to got-yo...@googlegroups.com
To unsubscribe from this group, send email to
got-your-bac...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/got-your-back?hl=en?hl=en

---
You received this message because you are subscribed to the Google Groups "Got Your Back: Gmail Backup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to got-your-bac...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/got-your-back/CAG56vfN-aAoksGaKpDfcxjWvSN8VxBhJfauJ9XKCv_hAmNsBUg%40mail.gmail.com.

Jay Lee

[Dale Courtney]

Jay,

Thanks for your help. I ran it again this morning (using that older version) and it did complete. Don't know why it didn't work yesterday. 🤷

1. I've upgraded. 

2. I'm running all of the files off of ~/Downloads. Nothing on a shared drive. 

3. This was a single backup. 

best,

Dale

To view this discussion visit https://groups.google.com/d/msgid/got-your-back/CA%2BVVBp89YGx435iryH%3DwgpR8-jNrbATJepacd31v9PeiLDdD6g%40mail.gmail.com.