Git LFS

[Andi McClure]

Does hg-git support git LFS (IE: pushing and pulling to an LFS repository, and having the files extracted and returned to the LFS server as appropriate)?

Are there plans to support git LFS?

[Gregory Szorc]

On Sat, Jun 30, 2018 at 2:25 PM, Andi McClure <andi.m....@gmail.com> wrote:

Does hg-git support git LFS (IE: pushing and pulling to an LFS repository, and having the files extracted and returned to the LFS server as appropriate)?

To my knowledge, no.

Mercurial itself now has an "lfs" extension that teaches Mercurial how to use LFS servers for blob storage. And there's even a mechanism where you can convert an existing repository to use LFS for storage of existing large files.

But I would expect hg-git would need to be taught how to recognize Git objects pointing to LFS blobs and vice-versa. I don't see any code like that in hg-git.

 

Are there plans to support git LFS?

To my knowledge, nobody is working on it. But I suspect support is theoretically possible. It might be easier to achieve if upstream Mercurial moved some of its LFS code (like the bits for interacting with LFS servers) out of an extension and into core. This isn't critical to LFS support in hg-git. But I think it would help.

I've CCd Matt Harbison (responsible for large parts of LFS in Mercurial) in case he wants to add context.

[Matt Harbison]

On Sun, 08 Jul 2018 19:28:15 -0400, Gregory Szorc
<gregor...@gmail.com> wrote:

> On Sat, Jun 30, 2018 at 2:25 PM, Andi McClure <andi.m....@gmail.com>
> wrote:
>
>> Does hg-git support git LFS (IE: pushing and pulling to an LFS
>> repository,
>> and having the files extracted and returned to the LFS server as
>> appropriate)?
>>
>
> To my knowledge, no.
>
> Mercurial itself now has an "lfs" extension that teaches Mercurial how to
> use LFS servers for blob storage. And there's even a mechanism where you
> can convert an existing repository to use LFS for storage of existing
> large files.
>
> But I would expect hg-git would need to be taught how to recognize Git
> objects pointing to LFS blobs and vice-versa. I don't see any code like
> that in hg-git.

The only thing I've used hg-git for is to maintain an hg copy of the Linux
kernel, so I don't have any plans to work on it, and don't really
understand how it works.

Assuming all of the normal hg functionality applies, I think recognizing
the pointer files is perhaps the only thing that needs to be done. Blobs
added on the hg side are pushed via a pre-push hook[1], and I'd expect
this to work now. Likewise, all of the plumbing is there to fetch LFS
blobs. The URL to the inferred blob store is even the same.

To flesh out what Gregory said, presumably the git data needs to be
flagged with the equivalent of revlog.REVIDX_EXTSTORED[2] as it's
written/pushed, and the EXTSTORED flag needs to be applied to the incoming
translated hg data on `hg pull`. The flag is already in core, so I don't
see any code that needs to be moved out of the extension. (Presumably,
enabling the lfs extension would be required anyway.) Watching
.gitattributes instead of .hglfs might be a future nice-to-have. If
someone wants to try making these changes, I can probably help debug any
problems.

Note that the LFS extension doesn't support the verify command. (Which is
nothing like `hg verify`, and more oriented towards storing blobs on a 3rd
party server, and then letting the original know you've completed the
upload.) Otherwise, I think the client protocol side of things is
reasonably complete.

[1]
https://www.mercurial-scm.org/repo/hg/file/4.6.2/hgext/lfs/wrapper.py#l300
[2]
https://www.mercurial-scm.org/repo/hg/file/4.6.2/hgext/lfs/wrapper.py#l133