2.15 and ReviewDb/NoteDb in practise

[Gert van Dijk]

Hi all,

With the 2.15 RCs out I've been playing around with NoteDb, but I've got some practical questions/remarks. Along the way I hope to improve some documentation.

1. If I understand correctly, the [database] section from gerrit.conf is purely for ReviewDb. I'd like to see that more clear in the documentation. See change 133891.
2. If I understand correctly, a new 2.15(-rc1) initialised site uses ReviewDb by default. It just seems odd that the new storage backend is not the new default. Why not default to NoteDb for new sites? It's so confusing from a user point of view with all the "promo" regarding 2.15 NoteDb. Could we at least improve that in the release notes? See change 133951.
3. The NoteDb documentation suggests that NoteDb completely replaces ReviewDb, but unfortunately only groups are mentioned as a work-in-progress for now. Does that mean you'll still need a ReviewDb solely for storing group data when running a 2.15 release?
4. After migrating to NoteDb, would you say it's OK to have the remaining ReviewDb in H2 in production performance wise? I'd say the need for performance with a "real" RDMS like PostgreSQL isn't required purely for storing a few groups.
5. What about accountPatchReviewDb? Can/Will that be migrated to NoteDb too in the future? It's not mentioned in the NoteDb documentation. (Apart from NoteDb, I noticed this could use some documentation love too, see change 133892.)
   
6. If yes to 4 and depending on 5, I'd like to consider to get rid of a PostgreSQL server in my 2.14 installation by upgrading to 2.15 and migrating to NoteDb. Can I convert my now PostgreSQL ReviewDb to H2 after migrating to NoteDB with 2.15 for the data as mentioned in 3? I see the program to migrate accountPatchReviewDb (MigrateAccountPatchReviewDb), but nothing for ReviewDb.
7. After performing a migration to NoteDB, the gsql SSH command fails. Reported as Issue 7467.
   

Overall, it seems that in regular production systems coming from current Gerrit stable you'll still need to run a PostgreSQL server with 2.15. I had hoped that NoteDb would have been the single storage backend for all. I've worked with hosting quite some web applications, but Gerrit is the first to have several co-existing databases to run for primary functionality. I believe that it could be quite confusing when administrating a Gerrit instance. I hope I can help with some documentation updates.

Keep up the great work on 2.15 and I'm looking forward to the release!

Thanks,

Gert

[Gert van Dijk]

Overall, it seems that in regular production systems coming from current Gerrit stable you'll still need to run a PostgreSQL server with 2.15. I had hoped that NoteDb would have been the single storage backend for all. I've worked with hosting quite some web applications, but Gerrit is the first to have several co-existing databases to run for primary functionality. I believe that it could be quite confusing when administrating a Gerrit instance. I hope I can help with some documentation updates!

Keep up the great work on 2.15 and I'm looking forward to the release!

Thanks,

Gert

P.S. I tried posting this already ~30 mins ago, but the message still doesn't show up. Assuming it was lost, this is a re-post, sorry if this is due to e.g. moderation.

[thomasmu...@yahoo.com]

You need to use a mix of reviewdb and notedb as groups is still yet to be migrated to notedb.

[Edwin Kempin]

Thanks for this feedback and the improvements to the documentation.

On Wed, Oct 18, 2017 at 12:44 AM, Gert van Dijk <gert...@gmail.com> wrote:

Hi all,

With the 2.15 RCs out I've been playing around with NoteDb, but I've got some practical questions/remarks. Along the way I hope to improve some documentation.

1. If I understand correctly, the [database] section from gerrit.conf is purely for ReviewDb. I'd like to see that more clear in the documentation. See change 133891.
2. If I understand correctly, a new 2.15(-rc1) initialised site uses ReviewDb by default. It just seems odd that the new storage backend is not the new default. Why not default to NoteDb for new sites? It's so confusing from a user point of view with all the "promo" regarding 2.15 NoteDb. Could we at least improve that in the release notes? See change 133951.

That's a good proposal. +1 for initializing new 2.15 sites with NoteDb by default. 

1. The NoteDb documentation suggests that NoteDb completely replaces ReviewDb, but unfortunately only groups are mentioned as a work-in-progress for now. Does that mean you'll still need a ReviewDb solely for storing group data when running a 2.15 release?

Yes. With 2.15 you still need ReviewDb for the group data.

We are still in the transition phase from ReviewDb to NoteDb. When NoteDb is fully done, ReviewDb will be dropped completely. 

1. After migrating to NoteDb, would you say it's OK to have the remaining ReviewDb in H2 in production performance wise? I'd say the need for performance with a "real" RDMS like PostgreSQL isn't required purely for storing a few groups.

I still would not recommend to use H2 in production. It may work, but if it doesn't it's unlikely that we will make fixes for it.

1. What about accountPatchReviewDb? Can/Will that be migrated to NoteDb too in the future? It's not mentioned in the NoteDb documentation. (Apart from NoteDb, I noticed this could use some documentation love too, see change 133892.)

accountPatchReviewDb will stay as separate storage next to NoteDb. If you are interested in details about this have a look at [1] and the linked discussions.

[1] https://gerrit-review.googlesource.com/c/gerrit/+/78945

 

1. If yes to 4 and depending on 5, I'd like to consider to get rid of a PostgreSQL server in my 2.14 installation by upgrading to 2.15 and migrating to NoteDb.

We are working on removing the need for ReviewDb. I would recommend to wait until this work is fully done and released with Gerrit 3.0.

 

1. Can I convert my now PostgreSQL ReviewDb to H2 after migrating to NoteDB with 2.15 for the data as mentioned in 3? I see the program to migrate accountPatchReviewDb (MigrateAccountPatchReviewDb), but nothing for ReviewDb.

We don't have tools for migrating ReviewDb data from one database to another, but you can do this by exporting/importing database dumps.

 

1. After performing a migration to NoteDB, the gsql SSH command fails. Reported as Issue 7467.

Overall, it seems that in regular production systems coming from current Gerrit stable you'll still need to run a PostgreSQL server with 2.15. I had hoped that NoteDb would have been the single storage backend for all. I've worked with hosting quite some web applications, but Gerrit is the first to have several co-existing databases to run for primary functionality. I believe that it could be quite confusing when administrating a Gerrit instance. I hope I can help with some documentation updates!

Keep up the great work on 2.15 and I'm looking forward to the release!

Thanks,

Gert

P.S. I tried posting this already ~30 mins ago, but the message still doesn't show up. Assuming it was lost, this is a re-post, sorry if this is due to e.g. moderation.

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Borowitz]

+1 to everything Edwin said, thank you for the suggestions and documentation contributions.

On Wed, Oct 18, 2017 at 4:41 AM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:

Thanks for this feedback and the improvements to the documentation.

On Wed, Oct 18, 2017 at 12:44 AM, Gert van Dijk <gert...@gmail.com> wrote:

Hi all,

With the 2.15 RCs out I've been playing around with NoteDb, but I've got some practical questions/remarks. Along the way I hope to improve some documentation.

1. If I understand correctly, the [database] section from gerrit.conf is purely for ReviewDb. I'd like to see that more clear in the documentation. See change 133891.
2. If I understand correctly, a new 2.15(-rc1) initialised site uses ReviewDb by default. It just seems odd that the new storage backend is not the new default. Why not default to NoteDb for new sites? It's so confusing from a user point of view with all the "promo" regarding 2.15 NoteDb. Could we at least improve that in the release notes? See change 133951.

That's a good proposal. +1 for initializing new 2.15 sites with NoteDb by default. 

Done in https://gerrit-review.googlesource.com/c/gerrit/+/137111