Performance comparison runs of recent PE builds for 2019.1 and 2019.3 reveal that PuppetDB in these builds are using more memory than it does in the PE 2018.1.11 baseline. The current threshold for this alert is a variation greater than 10%. Although the 2019.1.4-rc0-228-g062cb5b result is only slightly higher than this threshold, 2019.3.0-rc3-4-ged33755 has reported memory usage 30% higher than the 2018.1.11 baseline.
These results may be indicative of a memory/performance issue with PuppetDB
I wouldn't expect a decrease in shared_buffers for postgresql to increase PuppetDB's use of memory. The tuning PR reduces connections from PuppetDB to postgresql which should lower the memory use by a few megabytes. The rest of the tuning doesn't apply to a standard install of PE.
I would expect the difference in 2019.3 to be related to the change to java 11 and the default GC change.
Austin Blatt, the reported dev build was using Java 11 with the new GC. The variance report failed again on a newer build with the new GC turned off. I will be running this job today with latest dev build and will report back its results.
For the performance comparison run of week 2020-02-17, this tolerance failed when testing recent master build (2019.5.0-rc0-29-gb8100c2) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-02-24, this tolerance failed when testing recent PE master build (2019.5.0-rc0-68-gf0a5db6) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-03-02, this tolerance failed when testing recent PE master build (2019.5.0-rc0-399-g3d3203b) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-03-09, this tolerance failed when testing recent PE master build (2019.5.0-rc0-520-gab23aad) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-03-16, this tolerance failed when testing recent PE master build (2019.5.0-rc1-16-gb20eb01) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-03-23, this tolerance failed when testing recent PE master build (2019.6.0-rc2-16-g099cf6a) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-03-30, this tolerance failed when testing recent PE master build (2019.6.0-rc2-26-g21f8c4c) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-04-06, this tolerance failed when testing recent PE master build (2019.7.0-rc0-11-g427cad1) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-04-013, this tolerance failed when testing recent PE master build (2019.7.0-rc0-49-g5037017) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-04-20, this tolerance failed when testing recent PE master build (2019.7.0-rc0-84-gf9ac17e) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-04-27, this tolerance failed when testing recent PE master build (2019.7.0-rc1-15-gcd7efbb) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-05-04, this tolerance failed when testing recent PE master build (2019.8.0-rc0-7-g4277622) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-05-11, this tolerance failed when testing recent PE master build (2019.8.0-rc0-108-ga105716) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-05-18, this tolerance failed when testing recent PE master build (2019.8.0-rc1-17-gf233795) to the LTS baseline (but not STS).
For the performance comparison run of week 2020-05-25, this tolerance failed when testing recent PE master build (2019.8.0-rc3-5-g3b0d0bc) to the LTS baseline (but not STS).