After switching to chroniclemap, some cache shows worse result.

18 views

Skip to first unread message

Yingchun Li

unread,

4:11 AM (13 hours ago) 4:11 AM

to Repo and Gerrit Discussion

Hi,

After switching to cache-chroniclemap, some caches(mostly the diff_summary)

shows worse result than H2-V1.

The show-caches command output:

ssh -p 29418 localhost gerrit show-caches
Gerrit Code Review 3.12.3 now 08:57:53 UTC
uptime 19 hrs 34 min

Name |Entries | AvgGet |Hit Ratio|
| Mem Disk Space| |Mem Disk|
--------------------------------+---------------------+---------+---------+
adv_bases | | | |
changeid_project | 168 | | 83% |
changes_by_project | | | 0% |
default_preferences | | | |
external_ids_map | 1 | | 99% |
groups | | | |
groups_bymember | 3 | 1.2ms | 25% |
groups_byname | 1 | 12.0ms | 99% |
groups_bysubgroup | 7 | 766.0us | 58% |
groups_byuuid | 41 | 6.9ms | 99% |
groups_external | 1 | 67.3ms | 75% |
groups_external_persisted | | 64.9ms | 0% |
ldap_group_existence | | | |
ldap_groups | 15 | 2.9ms | 50% |
ldap_groups_byinclude | | | |
ldap_usernames | 159 | 244.2us | 67% |
permission_sort | 1024 | 21.4us | 63% |
plugin_resources | 6 | | 93% |
project_list | 1 | 40.3ms | 99% |
projects | 1024 | 649.4us | 99% |
prolog_rules | | | |
soy_sauce_compiled_templates | 1 | 92.6ms | 99% |
sshkeys | 244 | 20.9ms | 99% |
static_content | 28 | 4.6ms | 79% |
lfs-lfs_project_locks | 8 | 208.8us | 95% |
plugin-manager-plugins_list | | 130.7s | 0% |
uploadvalidator-patternCache | | | |
D accounts | 711 711 444.00k| 1.3ms | 99% |
D change_kind | 13838 13838 9.96m| 39.2ms | 91% |
D change_notes | 12415 19555 1.25g| 1.5ms | 95% |
D comment_context | 2011 2011 10.01m| 1.9ms | 69% |
D conflicts | 18055 18055 30.39m| 146.1ms | 70% |
D diff_intraline | 1924 1924 36.44m| 3.8ms | 26% |
D diff_summary | 2379 2379 69.37m| 5.4s | 85% |
D gerrit_file_diff |394557 394557 3.70g| 46.7ms | 54% |
D git_file_diff |388019 388019 2.44g| 47.3ms | 0% |
D git_modified_files | 3019 3019 674.57m| 2.1ms | 2% |
D git_tags | 2 2 160.00k| 2.4ms | 98% 0%|
D groups_byuuid_persisted | 41 52.00k| | |
D mergeability | 1359 1359 4.13m| 59.1ms | 50% 0%|
D modified_files | 1767 3806 175.48m| 2.5ms | 88% 0%|
D oauth_tokens | 2.52m| | |
D persisted_projects | 1047 7.70m| | |
D pure_revert | 8 8 32.00k| 56.5ms | 88% |
D quota-repo_size | 1024 1047 2.66m| 5.0ms | 0% |
D web_sessions | 1505 1505 168.26m| | 76% 0%|

SSH: 7 users, oldest session started 0 ms ago
Tasks: 9 total = 4 running + 0 ready + 5 sleeping
Mem: 38.41g total = 22.99g used + 7.42g free + 8.00g buffers
56.00g max
432 open files

Threads: 16 CPUs available, 230 threads.

Here the diff_summy AvgGet is 5.4s seconds. this is much higher than before.

just before it is a few milliseconds.

Here is result before the switching, the H2-v1 cache backend,

Gerrit Code Review 3.10.6 now 03:47:41 UTC
uptime 13 days 16 hrs

Name |Entries | AvgGet |Hit Ratio|
| Mem Disk Space| |Mem Disk|
--------------------------------+---------------------+---------+---------+
adv_bases | | | 99% |
change_notes | 6217 | 1.3ms | 96% |
changeid_project | 1024 | | 92% |
changes_by_project | | | 0% |
default_preferences | | | |
external_ids_map | 1 | | 99% |
groups | | | |
groups_bymember | 358 | 1.2ms | 99% |
groups_byname | 1 | 9.2ms | 99% |
groups_bysubgroup | 53 | 2.0ms | 99% |
groups_byuuid | 460 | 627.2us | 99% |
groups_external | 1 | 537.0ms | 99% |
groups_external_persisted | | 535.9ms | 0% |
ldap_group_existence | | | |
ldap_groups | 63 | 12.3ms | 99% |
ldap_groups_byinclude | | | |
ldap_usernames | 1024 | 223.9us | 99% |
permission_sort | 1024 | 22.8us | 89% |
plugin_resources | 9 | | 76% |
project_list | 1 | 183.5ms | 99% |
projects | 1024 | 2.3ms | 99% |
prolog_rules | | | |
soy_sauce_compiled_templates | 1 | 91.5ms | 99% |
sshkeys | 378 | 24.5ms | 99% |
static_content | 30 | 3.1ms | 85% |
lfs-lfs_project_locks | 17 | 302.4us | 98% |
plugin-manager-plugins_list | | 131.3s | 0% |
uploadvalidator-patternCache | | | |
D accounts | 779 2053 974.50k| 1.6ms | 99% 100%|
D change_kind | 17064 141951 17.54m| 3.4ms | 97% 99%|
D comment_context | 8773 39416 23.97m| 5.9ms | 92% 99%|
D conflicts | 37400 942956 128.04m| 66.9ms | 45% 99%|
D diff_intraline | 19995 99457 128.04m| 22.0ms | 30% 97%|
D diff_summary | 1417 288083 218.75m| 6.8ms | 99% 99%|
D gerrit_file_diff |299000 299358 345.42m| 20.7ms | 97% 39%|
D git_file_diff |150425 150725 172.73m| 25.6ms | 59% 17%|
D git_modified_files | 7040 14009 127.83m| 3.7ms | 4% 49%|
D git_tags | 2 3 138.15k| 38.1ms | 98% 100%|
D groups_byuuid_persisted | 136 86.31k| | 100%|
D mergeability | 8738 131366 20.07m| 63.6ms | 53% 84%|
D modified_files | 7451 26625 124.02m| 2.4ms | 95% 98%|
D oauth_tokens | 0.00k| | |
D persisted_projects | 2365 4.34m| | 100%|
D pure_revert | 79 2929 341.33k| 28.6ms | 99% 100%|
D web_sessions | 2048 17062 7.10m| | 99% 100%|

SSH: 4 users, oldest session started 0 ms ago
Tasks: 24 total = 2 running + 0 ready + 22 sleeping
Mem: 30.16g total = 17.57g used + 4.58g free + 8.00g buffers
56.00g max
219 open files

Threads: 16 CPUs available, 260 threads.

I adjusted the setting according to the result of

ssh -p 29418 localhost cache-chroniclemap analyze-h2-caches

And below is my cache settings:

[cache "web_sessions"]
maxAge = 4w
memoryLimit = 2048

[cache "change_notes"]
memoryLimit = 512m
diskLimit = 2g

[cache "git_modified_files"]
memoryLimit = 1024m
diskLimit = 5g

[cache "git_file_diff"]
memoryLimit = 1024m
diskLimit = 5g

[cache "gerrit_file_diff"]
memoryLimit = 768m
diskLimit = 4g

[cache "diff_intraline"]
memoryLimit = 512m

[cache "diff_summary"]
memoryLimit = 512m

[cache "mergeability"]
memoryLimit = 256m
diskLimit = 2g

[cache "conflicts"]
memoryLimit = 256m
diskLimit = 2g

############################
#* Chronicle-map template **
############################

[cache "persisted_projects"]
maxEntries = 3980
avgKeySize = 114
avgValueSize = 1438
[cache "mergeability"]
maxEntries = 18200
avgKeySize = 152
avgValueSize = 20
[cache "pure_revert"]
maxEntries = 123
avgKeySize = 119
avgValueSize = 20
[cache "git_modified_files"]
maxEntries = 29331
avgKeySize = 124
avgValueSize = 20415
[cache "git_file_diff"]
maxEntries = 1786337
avgKeySize = 336
avgValueSize = 888
[cache "accounts"]
maxEntries = 817
avgKeySize = 52
avgValueSize = 413
[cache "web_sessions"]
maxEntries = 311197
avgKeySize = 68
avgValueSize = 376
[cache "git_tags"]
maxEntries = 2
avgKeySize = 21
avgValueSize = 74183
[cache "groups_byuuid_persisted"]
maxEntries = 56
avgKeySize = 128
avgValueSize = 528
[cache "modified_files"]
maxEntries = 4483
avgKeySize = 125
avgValueSize = 28211
[cache "conflicts"]
maxEntries = 150377
avgKeySize = 134
avgValueSize = 20
[cache "change_kind"]
maxEntries = 52341
avgKeySize = 110
avgValueSize = 32
[cache "comment_context"]
maxEntries = 11923
avgKeySize = 161
avgValueSize = 547
[cache "change_notes"]
maxEntries = 108211
avgKeySize = 83
avgValueSize = 10904
[cache "diff_intraline"]
maxEntries = 23363
avgKeySize = 1006
avgValueSize = 361
[cache "gerrit_file_diff"]
maxEntries = 2602456
avgKeySize = 336
avgValueSize = 921
[cache "diff_summary"]
maxEntries = 28149
avgKeySize = 192
avgValueSize = 2004

Also the "git_file_diff" and "git_modified_files" suffer a very very low cache hits,

almost zero, (but the H2-V1 also have a low cache hits).

Br,

Yingchun

Reply all

Reply to author

Forward

0 new messages