refs/changes/xx/xxx/meta did not point to the newest patchset after move repos to new machine

[Yingchun Li]

Hi,

I recently migrated repos from an old Gerrit server to a new one using these steps:

1. Stop Gerrit (gerrit.sh stop)
2. rsync the cache, index, and repos
3. Run an offline reindex on the new server

The new server is up and running, but some changes show incorrect states. The root cause appears to be that refs/changes/xx/xxx/meta doesn’t point to the latest patchset. For example, a change with 59 patchsets on the old server only shows 54 on the new one—even though all patchsets are present in packed-refs.

Manually updating the ref (git update-ref refs/changes/xx/xxx/meta <latest>) and reindexing (gerrit index changes ...) restores the missing patchsets.

However, I’m unsure why this happened or how many other changes might be affected. It seems like a cache issue—possibly some caches weren’t flushed to disk before the sync. Both servers run Gerrit 3.12.4 with ChronicleMap as the backend cache.

Any insights would be appreciated!

Best regards,
Yingchun

[Matthias Sohn]

Do you still have the old server ? Then I would compare the state of these refs between the old and the new server.

If the server was gracefully shutdown and all persistent data (git repos, caches, indexes) were correctly rsynced

no reindexing should be required assuming the index didn't have stale entries before the copy.

Check the configuration for timeout and graceful shutdown options.

See

• container.shutdownTimeout
• httpd.gracefulStopTimeout
• sshd.gracefulStopTimeout

if the shutdown takes longer than the configured timeout the server process is killed 

which can lose data which wasn't yet flushed to persistent storage.

If you are running Gerrit on k8s also check the k9s-gerrit gracefulStopTimeout value.

 

Best regards,
Yingchun

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/4bee77a6-219d-4fbd-99a4-fdc9228be2cdn%40googlegroups.com.

[Yingchun Li]

On Tuesday, February 24, 2026 at 8:58:12 PM UTC+8 Matthias Sohn wrote:

On Tue, Feb 24, 2026 at 1:01 PM Yingchun Li <sword.l...@gmail.com> wrote:

Hi,

I recently migrated repos from an old Gerrit server to a new one using these steps:

1. Stop Gerrit (gerrit.sh stop)
2. rsync the cache, index, and repos
3. Run an offline reindex on the new server

The new server is up and running, but some changes show incorrect states. The root cause appears to be that refs/changes/xx/xxx/meta doesn’t point to the latest patchset. For example, a change with 59 patchsets on the old server only shows 54 on the new one—even though all patchsets are present in packed-refs.

Manually updating the ref (git update-ref refs/changes/xx/xxx/meta <latest>) and reindexing (gerrit index changes ...) restores the missing patchsets.

However, I’m unsure why this happened or how many other changes might be affected. It seems like a cache issue—possibly some caches weren’t flushed to disk before the sync. Both servers run Gerrit 3.12.4 with ChronicleMap as the backend cache.

Any insights would be appreciated!

Do you still have the old server ? Then I would compare the state of these refs between the old and the new server.

If the server was gracefully shutdown and all persistent data (git repos, caches, indexes) were correctly rsynced

no reindexing should be required assuming the index didn't have stale entries before the copy.

Check the configuration for timeout and graceful shutdown options.

See

• container.shutdownTimeout
• httpd.gracefulStopTimeout
• sshd.gracefulStopTimeout

I do not set these options, and they should be default. 

if the shutdown takes longer than the configured timeout the server process is killed 

which can lose data which wasn't yet flushed to persistent storage.

can not remember how long it took when shutdonw, but seems there are no errors showing. 

If you are running Gerrit on k8s also check the k9s-gerrit gracefulStopTimeout value. 

Now I found other warnings in the error_log, seems also the cache or index issue, like:

gerrit-home/logs/error_log:[2026-02-24T20:42:19.271+08:00] [SSH git-upload-pack /path1/proj1 (user1)] WARN  com.google.gerrit.server.permissions.GitVisibleChangeFilter : Duplicate change datas for the repo path1/proj1: [ChangeData{Change{152302 (I7dfbee261860ef1102c135eaffbb197f2ab8fc41), dest=path1/proj1,refs/heads/master, status=A}}, ChangeData{Change{152302 (I7dfbee261860ef1102c135eaffbb197f2ab8fc41), dest=path1/proj1,refs/heads/master, status=A}}] [CONTEXT TRACE_ID="1771936930440-af448438" project="path1/proj1" request="GIT_UPLOAD" ]

gerrit-home/logs/error_log:[2026-02-24T20:42:19.405+08:00] [SSH git-upload-pack /path1/proj1 (user1)] WARN  com.google.gerrit.server.permissions.GitVisibleChangeFilter : Duplicate change datas for the repo path1/proj1: [ChangeData{Change{93933 (Icb9137acca3bc31c2021b4d191cccee3ab8da370), dest=path1/proj1,refs/heads/master, status=M}}, ChangeData{Change{93933 (Icb9137acca3bc31c2021b4d191cccee3ab8da370), dest=path1/proj1,refs/heads/master, status=M}}] [CONTEXT TRACE_ID="1771936930440-af448438" project="path1/proj1" request="GIT_UPLOAD" ] 

I have checked the related changes on gerrit web, seems all these change are normal.

[Yingchun Li]

On Tuesday, February 24, 2026 at 9:25:10 PM UTC+8 Yingchun Li wrote:

On Tuesday, February 24, 2026 at 8:58:12 PM UTC+8 Matthias Sohn wrote:

On Tue, Feb 24, 2026 at 1:01 PM Yingchun Li <sword.l...@gmail.com> wrote:

Hi,

I recently migrated repos from an old Gerrit server to a new one using these steps:

1. Stop Gerrit (gerrit.sh stop)
2. rsync the cache, index, and repos
3. Run an offline reindex on the new server

one more thing, when I did the offline reindex, the process hung and never return, so I used the control-C to abort

the reindex process.