Gerrit lfs-plugin and multisite plugin

411 views
Skip to first unread message

Liem Do

unread,
Oct 18, 2023, 8:10:09 PM10/18/23
to Repo and Gerrit Discussion
Hello,

Does anyone know if lfs-plugin work when having multi-site enabled in Gerrit? If using filesystem for backend how does the backend get synced to the other multi-site nodes?

Thanks,
Liem

Luca Milanesio

unread,
Oct 19, 2023, 6:32:35 PM10/19/23
to Liem Do, Luca Milanesio, Repo and Gerrit Discussion
Hi Liem,

On 19 Oct 2023, at 00:57, Liem Do <lie...@gmail.com> wrote:

Hello,

Does anyone know if lfs-plugin work when having multi-site enabled in Gerrit?

Generally speaking, the LFS support in Gerrit is very limited at the moment.
Even without multi-site, there are a lot of gotchas.

@Jacek may list them all, as he is the most knowledgeable on that plugin.

If using filesystem for backend how does the backend get synced to the other multi-site nodes?

Gerrit does not do any replication of the LFS data and also security is an issue.
It isn’t production ready IMHO.

Luca.


Thanks,
Liem

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/5d08e73b-cb73-4c10-bca6-8059fddfa4d7n%40googlegroups.com.

Liem Do

unread,
Oct 24, 2023, 2:53:56 PM10/24/23
to Repo and Gerrit Discussion
Thanks Luca for the response.

Is there any roadmap/plan for the git-lfs plugin features in Gerrit? We we looking to see if other storage options would be supported in the future like Artifactory.

Liem

Luca Milanesio

unread,
Oct 24, 2023, 3:55:44 PM10/24/23
to Repo and Gerrit Discussion, Luca Milanesio, Liem Do

On 24 Oct 2023, at 20:53, Liem Do <lie...@gmail.com> wrote:

Thanks Luca for the response.

Is there any roadmap/plan for the git-lfs plugin features in Gerrit?

Not really, nobody is actively working on it at the moment.

We we looking to see if other storage options would be supported in the future like Artifactory.

Well, if the storage is Artifactory, then Gerrit is completely outside the picture !
Git client (with LFS support) will talk directly to Artifactory, bypassing Gerrit altogether.

Luca.

Nasser Grainawi

unread,
Oct 31, 2023, 4:19:21 PM10/31/23
to Luca Milanesio, Repo and Gerrit Discussion, Liem Do
On Tue, Oct 24, 2023 at 1:55 PM Luca Milanesio <luca.mi...@gmail.com> wrote:


On 24 Oct 2023, at 20:53, Liem Do <lie...@gmail.com> wrote:

Thanks Luca for the response.

Is there any roadmap/plan for the git-lfs plugin features in Gerrit?

Not really, nobody is actively working on it at the moment.

We we looking to see if other storage options would be supported in the future like Artifactory.
We aren't sure if we're going to try to use git-lfs or not, but if we do, we'll probably want Artifactory support too and we'd be working to add it. I have no ETA on even that decision though. 
 

Well, if the storage is Artifactory, then Gerrit is completely outside the picture !
Git client (with LFS support) will talk directly to Artifactory, bypassing Gerrit altogether.

Why then does the Gerrit lfs plugin have support for S3? Does the git client w/ lfs not know how to fetch from there directly?

If you wanted to apply Gerrit ACLs to the LFS APIs, you'd still need a Gerrit plugin, correct?
 

Dariusz Luksza

unread,
Nov 1, 2023, 4:59:40 AM11/1/23
to Nasser Grainawi, Luca Milanesio, Repo and Gerrit Discussion, Liem Do
On Tue, Oct 31, 2023 at 8:19 PM Nasser Grainawi <nasser....@linaro.org> wrote:


On Tue, Oct 24, 2023 at 1:55 PM Luca Milanesio <luca.mi...@gmail.com> wrote:


On 24 Oct 2023, at 20:53, Liem Do <lie...@gmail.com> wrote:

Thanks Luca for the response.

Is there any roadmap/plan for the git-lfs plugin features in Gerrit?

Not really, nobody is actively working on it at the moment.

We we looking to see if other storage options would be supported in the future like Artifactory.
We aren't sure if we're going to try to use git-lfs or not, but if we do, we'll probably want Artifactory support too and we'd be working to add it. I have no ETA on even that decision though. 
 

Well, if the storage is Artifactory, then Gerrit is completely outside the picture !
Git client (with LFS support) will talk directly to Artifactory, bypassing Gerrit altogether.

Why then does the Gerrit lfs plugin have support for S3? Does the git client w/ lfs not know how to fetch from there directly?

If you wanted to apply Gerrit ACLs to the LFS APIs, you'd still need a Gerrit plugin, correct?

Yes, you'd probably need a Gerrit plugin for that, but why would you want to do it?

It will add another overhead to Gerrit and fetch operation and it would require at least one redirect (or data proxy) and index of LFS object IDs to branch names to enforce ACLs. IMO it's not worth it.

The LFS works like a "symbolic link" inside of the Git repository. It uses SHA265 of the object to point the user to a different location where that object is available. Before the user knows the LFS object SHA256, he must be able to fetch git data from Gerrit (this is where the ACLs are checked).

Also, AFAIR LFS by default uses one-time-tokens for granting access to the resources which also can leak to potential attackers. 

If someone knows the LFS server URL, the object SHA256, and one-time-token, they can download the object manually. But that requires insider knowledge. Having a Gerrit plugin that verifies ACL and redirects would not help in this scenario. Only proxying LFS data would prevent that, but it will also generate more load on the server and its network.
 

郭郭亚日

unread,
Dec 7, 2023, 10:17:52 AM12/7/23
to Repo and Gerrit Discussion
Why this is marked as abuse? It has been marked as abuse.
Report not abuse
hi, dear all~I met a problem on gerrit lfs in master-slave environment. The object has been successfully uploaded to S3 via master server, but fail to download it with the slave server: Downloading pic2.jpg (202 KB) Error downloading object: pic2.jpg (09bc75c): Smudge error: Error downloading pic2.jpg (09bc75c9aac30d55xxx): batch response: Repository or object not found: http://slave.cn/projectxxx/info/lfs/objects/batch
The slave cannot download any object on S3 unless I redirect the lfs.url=master/project/info/lfs. How can we enable a slave to directly download objects from S3?

I'm sure that my slave uses the same lfs storage as the master, however, it works fine on the master, but does not take effect on the slave. The specific configuration on slave is as follows: 
1、gerrit.config : 
[lfs] 
    plugin = lfs 
    ... 
[container] 
    slave = true 

2、lfs.config : 
[storage] 
    backend = s3 
[s3] 
    hostname = xxx.fds.xxx.net 
    region = s3-cnbj.xxx-fds.net 
    bucket = tv-lfs 
    accessKey = xxxxxx 
    secretKey = xxxxxx


My expression may not be precise enough: master means primary, and slave means replica.
Looking forward to your kind response~Thank you!"
Reply all
Reply to author
Forward
0 new messages