VocBench performance

[Eugene Morozov]

Hi All,

We recently run a workshop on VocBench and hit some performance issues with calls to retrieve project list timing out for most users. The setup that we used is ~100 projects for 40 concurrent users on a single VocBench3 9.0.0 instance running on roughly 2 x 12-core Intel Haswell 2.4 GHz E5-2673 v3 with 8GB RAM (2GB allocated to VocBench) and SSD storage on Microsoft Azure.

We since put together a better set of tests and looked further into VocBench/SemanticTurkey source code - the issue appears to be driven by the number of projects created on a single instance and the use of hard drive to store project configuration as opposed to memory or CPU contention. Here's the scripts that we used for setting up VocBench and running the tests in case anyone finds it useful: https://github.com/cadmiumkitty/vocbench3-integration I am happy to share more detailed results.

I have a couple of questions:

1. Wondering if anyone from the VocBench user community can share the setup that they run in production for their organisations? Would be interesting to find out the number of projects and users you are running on a single instance of VocBench, and if you run a multi-instance VocBench for the same project set. Also, if you hit any performance issues, what were they and what did you do to overcome them (e.g. caching specific SemanticTurkey Web API calls like "/semanticturkey/it.uniroma2.art.semanticturkey/st-core-services/Projects/retrieveProjects").

2. Probably for the dev team - do you have any thoughts about or plans for moving away from disk storage towards a relational DB or a triplestore, adding distributed cache for project and user management, etc.

Eugene.