Updates on NoteDb & Gerrit 2.15

[Dave Borowitz]

You may have noticed some relative silence from me over the past few weeks on the topic of the 2.15 release. The reason for this is our team at Google temporarily shifted all of our focus to finishing the NoteDb implementation of the internal group backend.

Fortunately, we have some good news to share on that front: we are completely finished the NoteDb migration at Google. The servers behind gerrit-review.googlesource.com store 100% of their data in NoteDb, and as of today literally do not compile in the client code to connect to Megastore (the database used for our ReviewDb backend).

In the next week or so, we should land the final groups schema migration in master, eliminating the AccountGroup-related tables. This means the only remaining tables in the ReviewDb schema in master will be the Change-related tables.

Personally, I now have the rest of the calendar year to focus on getting 2.15 out the door. There are just a couple remaining blocking bugs related to the online NoteDb migration that I will turn my attention to this week. I hope to get a new RC with these bugfixes soon.

[Matthew Webber]

I hope to get a new RC with these bugfixes soon.

Sounds great. I hope to spend Christmas doing a test install of a new RC !

Matthew

[Luca Milanesio]

Hi Dave,

I have to say that I tried to convert GerritHub.io to NoteDb and failed :-(

Need to troubleshoot the failures and possibly open some bugs on the issue tracker.

Luca.

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Borowitz]

Thanks for giving it a shot, Luca. I'd be a little surprised (and definitely interested!) if your problem is something other than https://bugs.chromium.org/p/gerrit/issues/detail?id=7668

[luca.mi...@gmail.com]

Let me run it again this morning to double check.

Luca

Sent from my iPhone

[lucamilanesio]

Yes, I confirm I have the NPE as well.

However, I see two other failure cases:

[2017-12-12 07:54:19,590] [OnlineNoteDbMigrator] ERROR com.google.gerrit.server.notedb.rebuild.NoteDbMigrator : Failed to save NoteDb state for ****/****

java.io.IOException: Update failed: PackedBatchRefUpdate[

  CREATE: 0000000000000000000000000000000000000000 6629c72930403638d66518951ba24c8c3faa1ce1 refs/changes/59/15259/meta  (OK)

  CREATE: 0000000000000000000000000000000000000000 73771527bb9bc361e328ae8d31b2d78377abe00a refs/changes/16/215316/meta  (OK)

  CREATE: 0000000000000000000000000000000000000000 463bfd13471c154ed60b1aefdf8a5e010ce68e6f refs/changes/27/214827/meta  (REJECTED_MISSING_OBJECT)

  CREATE: 0000000000000000000000000000000000000000 086b2c1b47b015709e97018b5a08f211a9ee72e6 refs/changes/86/228986/meta  (OK)

  CREATE: 0000000000000000000000000000000000000000 67aa1f5e764c11b528fead265a1cd5ed476a2ee9 refs/changes/49/217849/meta  (OK)

  CREATE: 0000000000000000000000000000000000000000 3a9b5027d7ea96fac0808d6c39b01a7bdc16842e refs/changes/72/214772/meta  (OK)

]

and 

[2017-12-12 07:54:19,808] [OnlineNoteDbMigrator] ERROR com.google.gerrit.server.notedb.rebuild.NoteDbMigrator : Failed to rebuild change 14820

java.io.EOFException: Unexpected EOF in partially created pack

at org.eclipse.jgit.internal.storage.file.PackInserter$Reader.open(PackInserter.java:535)

at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:234)

at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:859)

Before creating the tickets, I want to:

1) Identify the minimal use-case for the bug

2) Identify the section of the code involved

3) Allow to reproduce it

Luca.

On Tuesday, December 12, 2017 at 7:26:33 AM UTC, lucamilanesio wrote:

Let me run it again this morning to double check.

Luca

Sent from my iPhone

On 11 Dec 2017, at 23:22, Dave Borowitz <dbor...@google.com> wrote:

Thanks for giving it a shot, Luca. I'd be a little surprised (and definitely interested!) if your problem is something other than https://bugs.chromium.org/p/gerrit/issues/detail?id=7668

On Mon, Dec 11, 2017, 17:48 Luca Milanesio <luca.mi...@gmail.com> wrote:

Hi Dave,

I have to say that I tried to convert GerritHub.io to NoteDb and failed :-(

Need to troubleshoot the failures and possibly open some bugs on the issue tracker.

Luca.

On 11 Dec 2017, at 22:14, Matthew Webber <mat...@unsolvable.org> wrote:

I hope to get a new RC with these bugfixes soon.

Sounds great. I hope to spend Christmas doing a test install of a new RC !

Matthew

--
--

To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

[Matthew Webber]

However, I see two other failure cases:

[2017-12-12 07:54:19,590] [OnlineNoteDbMigrator] ERROR com.google.gerrit.server.notedb.rebuild.NoteDbMigrator : Failed to save NoteDb state for ****/****

java.io.IOException: Update failed: PackedBatchRefUpdate[

<.. snip ..>

[2017-12-12 07:54:19,808] [OnlineNoteDbMigrator] ERROR com.google.gerrit.server.notedb.rebuild.NoteDbMigrator : Failed to rebuild change 14820

java.io.EOFException: Unexpected EOF in partially created pack

<.. snip ..> 

I saw both of those as well, reported in ticket 7924

I did not supply a minimal use-case, so if you can generate one Luca, that would be great. Otherwise, I'm happy to test any fixes (if someone can sent me a .war) by re-running against a copy of our system.

Matthew

 

[Luca Milanesio]

On 12 Dec 2017, at 09:24, Matthew Webber <mat...@unsolvable.org> wrote:

However, I see two other failure cases:

[2017-12-12 07:54:19,590] [OnlineNoteDbMigrator] ERROR com.google.gerrit.server.notedb.rebuild.NoteDbMigrator : Failed to save NoteDb state for ****/****

java.io.IOException: Update failed: PackedBatchRefUpdate[

<.. snip ..>

[2017-12-12 07:54:19,808] [OnlineNoteDbMigrator] ERROR com.google.gerrit.server.notedb.rebuild.NoteDbMigrator : Failed to rebuild change 14820

java.io.EOFException: Unexpected EOF in partially created pack

<.. snip ..> 

I saw both of those as well, reported in ticket 7924

Ah yes, I can see it now on the ticket marked as duplicate :-)

I did not supply a minimal use-case, so if you can generate one Luca, that would be great. Otherwise, I'm happy to test any fixes (if someone can sent me a .war) by re-running against a copy of our system.

Matthew

Yes, and I would like to debug it and put some more unit-tests around them. We should be covering them in our automated testing.

 

--
--
To unsubscribe, email repo-discuss...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

[Duft Markus]

Hey,

 

Same here – I have a copy of our system which I can test against, provided a commit or war file J

 

Cheers,

Markus

--
--
To unsubscribe, email repo-discuss...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

SSI Schäfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht für Zivilrechtssachen Graz

[thomasmu...@yahoo.com]

Hmm it seems that the groups notedb migration changes were never uploaded or merged.

[Dave Borowitz]

They're not in 2.15.

On Wed, Jan 10, 2018 at 7:55 AM, thomasmulhall410 via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:

Hmm it seems that the groups notedb migration changes were never uploaded or merged.

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

[thomasmu...@yahoo.com]

I meant the master branch not stable-2.15 :).

[thomasmu...@yahoo.com]

“In the next week or so, we should land the final groups schema migration in master, eliminating the AccountGroup-related tables.” says that groups migration change would be in the master branch.