Current state of Performance Tests in the Keycloak Project?

[EXTERNAL Weimer Benjamin (TNG, IOC/PAU1)]

Hi,

 

We are going to run performance tests for our Keycloak scenarios. I wanted to ask what’s the current state of the performance tests that already exist in the Keycloak project (https://github.com/keycloak/keycloak/tree/master/testsuite/performance)?

 

·       Are the tests executed at a regular basis? E. g. before each release to check if there are performance regressions in the new version.

·       Which scenarios are tested from the available ones (login-logout, registration-logout, admin-console – many realm, clients, users)?

·       Are users of Keycloak rather supposed to run these performance tests themselves with the parameters that make sense for their scenarios?

 

I tried to run them and found some issues, but in general they seem to work fine. I provided fixes for those, maybe you can have a look:

 

·       https://github.com/keycloak/keycloak/pull/8377

·       https://github.com/keycloak/keycloak/pull/8379

 

 

Mit freundlichen Grüßen / Best regards

Benjamin Weimer

Project Delivery Berlin 2 (IOC/PDL2)
Bosch.IO GmbH | Ziegelei 7 | 88090 Immenstaad | GERMANY | www.bosch.io
Tel. +49 30 726112-0 | external.Ben...@bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

​

[EXTERNAL Weimer Benjamin (TNG, IOC/PAU1)]

Hi,

 

I just noticed there is another project for Keycloak performance tests “keycloak-benchmark” [1]. This project seems to be more recently updated than the performance test suite according to the commits. Should the performance test suite in Keycloak be deprecated in favor of the benchmark tool or how do these two relate to each other? Only had a brief look at the benchmark tool for now.

 

[1] https://github.com/keycloak/keycloak-benchmark

 

Mit freundlichen Grüßen / Best regards

Benjamin Weimer

Project Delivery Berlin 2 (IOC/PDL2)
Bosch.IO GmbH | Ziegelei 7 | 88090 Immenstaad | GERMANY | www.bosch.io
Tel. +49 30 726112-0 | external.Ben...@bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

​

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/42d54af8fb244b038e93674764c9f076%40bosch.io.

[EXTERNAL Weimer Benjamin (TNG, IOC/PAU1)]

Hi,

 

To follow up on this thread we’ve created a PR for a new performance test scenario in the keycloak-benchmark repository [1]. Are these kind of extensions welcome in the keycloak-benchmark repository or should we rather maintain a fork of it and maintain our own scenarios?

 

[1] https://github.com/keycloak/keycloak-benchmark/pull/24

 

Mit freundlichen Grüßen / Best regards

Benjamin Weimer

Project Delivery Berlin 2 (IOC/PDL2)
Bosch.IO GmbH | Ziegelei 7 | 88090 Immenstaad | GERMANY | www.bosch.io
Tel. +49 30 726112-0 | external.Ben...@bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

​