Defining Elasticity and Scalability

[Yuna]

Hello all,

Based on the documentation this is what I understand about the SPEC benchmarks:

My understanding of elasticity is that it measures how load on the system (cloud) increases for example with increased provisioning time as more VMs are added to an AI or new AIs are created. Scalability is the measure of how much more work can be done as more VMs and AIs are added to the system. 

Is this understanding correct?

Would an ideal scalability number be Nx when you add N VMs ( If you have one VM and a second VM, the output should double so scalability would be equal to 2)?

Looking forward to hearing from you,

Yuna

[Marcio Silva]

Hello Yuna,

Yes, your understanding is, in a general manner, correct. A couple of clarifications, as explained in further detail in https://spec.cs.miami.edu/cloud_iaas2016/docs/runrules.pdf (pages 28-30)

a) Elasticity scores how the increasing load on the cloud (due to the deployment of additional workloads, a.k.a. Application Instances) degrades both the performance and deployment time, when compared to an initial baseline. It is dimensionless, with an ideal value of 1.
b) Your understanding of the behavior of the Scalability score for an "ideal" cloud is correct, but please note that the score is always expressed as <SCORE> @ <NUMBER OF APPLICATION INSTANCES>
c) Finally,  for Scalability, just keep in mind that the total "work capacity" contributed by a given Application Instance is always calculated relative to a Reference Platform (established per workload, e.g., Throughput in the case of YCSB/Cassandra and Completion Time for Hadoop/KMeans).

Regards,

Marcio
-------------------------------------------------------------
Marcio A. Silva, PhD.
Software Engineer
DataCenter Systems Software
IBM Thomas J. Watson Research Center
e-mail: mar...@us.ibm.com