Question about Copybara

[Julia Xu]

Hi,

My name is Julia, I work at Netflix, and I'm currently investigating using Copybara to synchronize our fork of Envoy with the OSS Envoy. We have a few small patches we'd like to apply on top of the OSS Envoy version and hope to use Copybara to make maintenance easier.

Here's an overview of what I've attempted so far:

- We've tried using 'patch.apply' in 'transformations' to apply a patch file. This patch file works when applied with `git apply file.patch`, but fails when Copybara tries to apply it.

- I also explored 'autopatch_config'. While it did run successfully, it didn't generate any patch files for me. Is there more information on how autopatch should be used? There isn't much in the repo in terms of a config. 

Any advice or guidance on how best to do this would be greatly appreciated.

Thank you!

Best Regards,

Julia

[Kurtis Van Gent]

Hey Julia,

I met with +Henry Sudhof, who's the TL of the Copybara team and talked through the issues we were having. I've added him on the thread in case you have more questions. 

It sounds like our problems were that we were running in ITERATIVE mode, which was trying to apply the patch to every commit, and the earlier commits weren't able to apply the patch successfully (because they were too old).

Henry recommended that you use `git.mirror` instead of core.workflow. You can use `refspecs` to list which branches you want to mirror internally, and the `action` to call a `core.action` to handle the merge. It looks like it might look something like this:

```

# ctx here is of type git.mirrorContext

def _fetch(ctx):

ctx.console.info("Fetching from upstream...")

ctx.origin_fetch(refspec = ["refs/heads/main"])

ctx.console.info("Pushing to mirror branch...")

ctx.destination_push(refspec = ["refs/heads/target"])

ctx.console.info("Fetching from patch branch...")

ctx.destination_fetch(refspec = [// commits to pick])

commit_hash = ctx.references(refspec = [local_upstream]).values()[0]

ctx.console.info("Merging " + commit_hash + " into main...")

merge_result = ctx.merge(

branch = "target",

commits = [// commits to pick],

msg = "commit msg",

fast_forward = "NO_FF",

)

if merge_result.error:

ctx.console.info(

"Conflict merging " + commit_hash + " into main:\n" + merge_result.error_msg,

)

return ctx.error("Conflict merging " + commit_hash + " into play/main")

ctx.console.info("Pushing merge commit...")

ctx.destination_push(refspec = ["refs/heads/target"])

ctx.console.info("Success!")

return ctx.success()

def fetch():

return core.action(impl = _fetch, params = {})

git.mirror(

name = "crosvm_to_kiwivm",

origin = "git_uri_for_envoy_oss",

destination = "git_uri_for_stash",

refspecs = [

// list of refs to mirror

],

action = fetch(),

) ```

Thanks,
Kurtis

Kurtis Van Gent k...@google.com Staff Software Engineer | SenseAI

[Henry Sudhof]

Hi Julia,

Apologies for missing your email when it came in. Generally, if the use case is to mirror an entire repo and not mutate the files dynamically, we recommend using git.mirror as described above because SCMs are typically better at this kind of thing than other tools. If you only need some files or have mutations that do not fall into git's abilities, then core.workflow is absolutely fine, but it won't preserve SHA1s. The error is most likely due to using ITERATIVE mode and having copybara start at an very old revision. Does it work when you specify the --squash flag?

Cheers,

~H

--

Henry Sudhof |

Software Engineer |

hsu...@google.com |

US-NYC-CHEL-8-8C114G