LFS times out even with sshExpirationSeconds set higher

[ja...@ladan.ca]

One of our devs is getting apparent auth timeouts on git pushes with git lfs even after I increased sshExpirationSeconds.

He ran into the auth timeout when doing a git push from home a few days ago. I increased sshExpirationSeconds to 60 (it was at the default 10), and confirmed with invoking git-lfs-authenticate that the setting was being used:

  $ ssh gerrit git-lfs-authenticate repo download
  {"expires_at":"2023-05-31T18:11:01.788Z","expires_in":60,"href":"https://example.com/repo/info/lfs","header":{"Authorization":"Ssh: nxzPPjts0y6byjUNH0qfigpBj/uJkkS7ZTZt85y9+ZaWjCLNwtBeWcd74W7AaISTAax6X76oIQHrQOWej49udRBS8Oa644oWm2F1iV/EtP4="}}$

He still ran into the timeout on his next attempt and ended up doing the push at the office the next day. Today it happened again for him.

Trace output appears to show an HTTP response from the LFS plugin that still has a 10 second timeout:

  May 30 18:16:33 < HTTP/1.1 200 OK
  May 30 18:16:33 < Transfer-Encoding: chunked
  May 30 18:16:33 < Connection: keep-alive
  May 30 18:16:33 < Content-Type: application/vnd.git-lfs+json; charset=utf-8
  May 30 18:16:33 < Date: Wed, 31 May 2023 01:16:33 GMT
  May 30 18:16:33 < Server: nginx/1.18.0 (Ubuntu)
  May 30 18:16:33 < X-Frame-Options: DENY
  May 30 18:16:33 <
  May 30 18:16:33 18:16:33.958539 trace git-lfs: HTTP: {"objects":[{"oid":"a63797ea1c79b50907cba7a3a0eed499bbcf821d0d33c7ba93fafe4d7c1eb107","size":115876352,"actions":{"upload":{"expires_at":"2023-05-31T01:16:43.945Z","expires_in":10,"href":"https://example.com/plugins/lfs/content/default/a63797ea1c79b50907cba7a3a0eed499bbcf821d0d33c7ba93fafe4d7c1eb107","header":{"Authorization":"<token>"

The transfer began, but then ended with "401 Unauthorized" about 12
seconds later:

  May 30 18:16:33 > PUT /plugins/lfs/content/default/a63797ea1c79b50907cba7a3a0eed499bbcf821d0d33c7ba93fafe4d7c1eb107 HTTP/1.1
  ...
  May 30 18:16:45 18:16:45.988570 trace git-lfs: HTTP: 401
  May 30 18:16:45
  May 30 18:16:45 < HTTP/1.1 401 Unauthorized

Is there some other setting I also need to change?

Thanks,

James

[James Ladan]

On Friday, 2 June 2023 at 11:40:37 UTC-7 I wrote:

One of our devs is getting apparent auth timeouts on git pushes with git lfs even after I increased sshExpirationSeconds.

In case anyone has the misfortune to have the same issue and find my message instead of a more helpful one:

Also set fs.expirationSeconds higher.