Copybara cache behavior when using multiple GitHub runners for monorepo ↔ external repo sync

[Sainath Seelam]

Hi Copybara team,

I’m using Copybara in our organization’s GitHub Actions setup to sync parts of a monorepo with several external repositories. The workflow involves:

• Sharing a portion of the monorepo to external repositories.
• Creating pull requests back to the monorepo from external repo branches.
• Merging those changes into the monorepo and sharing again to the external repos.

Since the jobs run on different GitHub runners, each instance may have a different (or no) Copybara cache available.

Would this cause issues for Copybara’s ability to track previous migrations?
Specifically:

• Does Copybara rely on its cache across runs to correctly detect or reconcile historical changes between the monorepo and external repos?
• If the cache is missing or inconsistent, could it affect incremental migrations or change detection?
• What’s the recommended approach to handle this scenario — should the cache be persisted between runners, or can Copybara operate statelessly if needed?

Thanks for your time.

Regards,
Sainath Seelam

[Hai Nguyen]

Hi there Sainath,

Have you had a chance to investigate these questions further? I’m currently running a similar setup, hosting Copybara as a stateless workload and also persisting its cache data for future runs. I’m curious if you encountered any issues when the cache is missing or inconsistent. Did this cause any problems with replication or change detection in your workflow?

Thanks for sharing any insights!

[Adrià Vilanova Martínez]

Although my use case for Copybara is simpler—I just move code one-way from private to public repos—, I've always taken for granted the following fact from the project's README (emphasis mine):

> One of the main features of Copybara is that it is stateless, or more specifically, that it stores the state in the destination repository (As a label in the commit message). This allows several users (or a service) to use Copybara for the same config/repositories and get the same result.

I've been running Copybara in Zuul (CI) for the past 2 years for this simple use case, and I haven't had any single issue when the cache is cleared. But of course persisting it is helpful in speeding up migrations.

Maybe someone more experienced with Copybara can chime in, though.

Cheers :-)

[Mikel Alcon]

Copybara cache should only affect performance (unless that translates to a timeout for the GHA or runner container it is running in). 

--
You received this message because you are subscribed to the Google Groups "Copybara OSS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to copybara-discu...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/copybara-discuss/8109b98e-e359-4ebf-b088-c1292ffc6e10n%40googlegroups.com.