Reviews: ProportionalEnsemble

[Rodrigo Fonseca]

Please post your reviews as a response to this message.
Rodrigo

[James Chin]

Paper Title: “Delivering Energy Proportionality with Non Energy-
Proportional Systems -- Optimizing the Ensemble”

Authors(s): Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash,
Parthasarathy Ranganathan, and Xiaoyun Zhu

Date: 2008

Novel Idea: This paper explores the viability and tradeoffs of
optimization-based approaches using two different case studies.
First, the authors show how different power-saving mechanisms can be
combined to deliver an aggregate system that is proportional in its
use of server power. Secondly, they show early results on delivering
a proportional cooling system for those servers.

Main Result(s): When compared to the power consumed at 100%
utilization, results from the authors’ testbed show that optimization-
based systems can reduce the power consumed at 0% utilization to 15%
for server power and 32% for cooling power.

Impact: This paper shows that it is possible to use optimization-based
techniques to approximate energy-proportional behavior at the ensemble
level. A system built according to this principle would, in theory,
use no power consumption growing in proportion to utilization.

Evidence: First, the authors show how the use of a VM migration
controller (that can also turn machines on or off in addition to DVFS
in response to demand changes) can reduce the power consumed by
servers and exhibit power-usage behavior close to that of an energy-
proportional system. Secondly, they demonstrate how a power and
workload-aware cooling controller can exhibit the same behavior for
cooling equipment such as server fans.

Prior Work: Over the last few years, a number of techniques have been
developed to make server processors more efficient, including better
manufacturing techniques and the use of Dynamic Voltage and Frequency
Scaling (DVFS) for runtime power optimization. The re-emergence of
virtualization and the ability to “live” migrate entire virtual
machines in a transparent and low-overhead manner, will enable a new
category of systems that can react better to changes in workloads at
the aggregate level.

Competitive Work: There is certainly work being done by others on
energy-proportional computing, but this paper doesn’t really touch
upon that specifically.

Reproducibility: Not sure -- it looks like the results can be
reproduced, but this paper is a bit vague at times.

Question: Besides internal fan control, should we consider external
HVAC systems as variables in the problem of cooling energy
proportionality? I think that’s important.

Criticism: Again, this paper is a bit vague at times. Also, it
doesn’t talk about competitive/future work that much.

Ideas for further work: Show how controllers would would work in a
federated environment where information would need to be shared
between controllers at different management layers and possibly from
different vendors.

[Siddhartha Jain]

Title: Delivering energy proportionality with non-energy systems - optimizing the ensemble

Novel Idea:

The idea is to use optimization, performance modelling, control theory, etc. and look at

all components including server fans and air conditioners to design an ensemble systems

whose power usage scales roughly proportionally with the amount of load using techniques

like turning off servers that idle.

Main Results:

2 case studies are done to show the effectiveness of the approach and experimental results are

shown.

Impact:

Might have impact since it considers a lot more factors when examining energy usage than just

the traditional factors of memory and cpu energy usage.

Evidence:

Power utilization at the low and middle end is much better.

Prior Work:

DVFS which uses dynamic voltage and frequency scaling for runtime power optimization. Other

work to optimize power consumption like FAWN though that does not deal with optimizing increase

in energy proportional to load but minimizing energy costs in general.

Reproducibility:

Hard to reproduce as no source code used for optimization available and few details of optimization

given.

Question/Future work:

It would be interesting to see how for other systems that try to conserve power, energy consumption

varies under load and whether one type of architecture can be optimized a lot more for energy usage

when varying load compared to some other architecture

[Basil Crow]

Title: Delivering Energy Proportionality with Non Energy-Proportional

Systems – Optimizing the Ensemble

Authors: Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash,

Parthasarathy Ranganathan, and Xiaoyun Zhu

Date: HotPower 2008

Novel idea: Optimization-based techniques can be used with commodity
hardware to approximate the behavior of energy-proportional systems at
the "ensemble" (aggregate) level.

Main results: The only "results" to speak of are two individual case
studies, detailed below. The authors rely on these case studies alone
in order to demonstrate the merit of their idea.

Impact: Designers of operating systems, server hardware, and
datacenter management hardware and software could use these results in
order to make their designs more efficient.

Evidence: In the first case study, the authors used a VM migration
controller to consolidate virtual machines and turn off idle machines
(utilizing the blade's power models and sensor readings from resource
monitoring agents). In the second case study, the authors used a
predictive fan controller in order to minimize the total fan power
consumption (utilizing temperature sensors in the blades as well as
software sensors to monitor server utilization). In both case studies,
the power usage for the ensemble was reduced proportionally to
utilization (in contrast with the base cases).

Prior work: This paper builds on previous work in optimization-based algorithms.

Competitive work: The authors mention previous studies that looked at
data center level cooling efficiency by manipulation of CRAC unit
settings or by temperature-aware workload placement; however, these
studies only looked at total power consumptions whereas the authors'
approach is more granular.

Reproducibility: Since the authors only describe their optimization
algorithm in the vaguest possible way, their results are not quickly
reproducible.

Criticism: It would have been useful to see how the authors set up
their constraint optimization problem.

Ideas for further work: It would be nice to see this work generalized
into a framework that could be used on arbitrary server hardware or
datacenter configurations.

[Visawee]

Paper Title :

Delivering Energy Proportionality with Non Energy-Proportional Systems
– Optimizing the Ensemble

Author(s) :

Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash, Parthasarathy

Ranganathan, Xiaoyun Zhu

Date :
Year 2008

Novel Idea :
Using optimization-based techniques to approximate energy-proportional
behavior, and deliver energy proportionality at the ensemble layer.

Main Result(s) :
The system can reduce the power consumed at 0% utilization to 15% for

server power and 32% for cooling power.

Impact :

More energy efficient system.

Evidence :
The authors gave two case studies to support their claim.

Prior Work :
There was some prior work that looked at data center level cooling

efficiency by manipulation of CRAC unit settings or by temperature

aware workload placement.

Reproducibility :
The results are irreproducible because the authors did not explain the
case studies' workload in detail.

Criticism :
The system might not be able to quickly response to a fluctuation of
workload.

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Shah]

Title:

Delivering Energy Proportionality with Non Energy-Proportional Systems

- Optimizing the Ensemble

Authors:

[1] Niraj Tolia
[2] Zhikui Wang
[3] Manish Marwah
[4] Cullen Bash
[5] Parthasarathy Ranganathan
[6] Xiaoyun Zhu

Source and Date:

Proceedings of the 2008 Conference on Power Aware Computing and
Systems, San Diego, California.

Novel Idea:

The new idea proposed by the authors in this paper is to use
performance modeling, optimization and control theory to reduce power
savings in ensembles (a logical collection of servers) effectively
driving costs downward.

Main Result:

The researchers buttress their central claim by performing two
experiments: The first involves the turning off of machines, smartly.
The second involves demonstrating how a power and workload-aware
cooling system works.

Impact:

The impact this paper has is not been clearly established.

Evidence:

Much of the meat of this paper revolves around two case studies. The
authors detail a fair amount of the hardware used but details about
the actual setup.

Prior Work:

The scientists mention that prior work revolves around efficiency
manipulation of Computer Room Air Conditioners (CRACs) settings. They
claim, though, that these studies looked at only total power
consumption and fail to give enough detail.

Competitive Work:

There is no mention of directly competitive work save for the one
mentioned in the earlier section.

Reproducibility

The authors don't divulge enough details to make the experiments
reproducible.

Question:

The authors briefly mention this but could imposing more stringent
standards in power consumption have a similar effect (something along
the lines of Energy Star)?

Criticism:

As the scientists mention, to directly evaluate and manage application
performance, more sensors that measure metrics such as throughput and
response time are needed.

Ideas for Further Work:

It would be interesting to conduct similar experiments when power
consumption standards for hardware are in place and see what the
combined effect is.

[Abhiram Natarajan]

Paper Title: Delivering Energy Proportionality with Non Energy-

Proportional Systems – Optimizing the Ensemble

Author(s): Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash,

Parthasarthy Ranganathan, Xiaoyun Zhu

Date: 2008, HotPower

Novel Idea: Usage of optimization-based techniques to build systems
with off-the-shelf hardware that approximates the behaviour of energy-
proportional systems.

Main Result(s): (1) Authors show how different power-saving mechanisms

can be combined to deliver an
aggregate system that is proportional in
its use of server power

(2) Authors also show early results on
delivering a proportional cooling system for these
servers

Impact: Optimization-based systems, such as the one proposed can

reduce the power consumed at 0%

utilisation to 15% for server power and 32% for cooling
power

Evidence: The authors provide empirical evidence of their claims. The
evidence appears to be conclusive
of their claims.

Prior Work: (1) The concept of energy-proportional computing! (Barroso
and Hotzle)
(2) Energy Scaledown (Mayo and Ranganathan)
(3) Manufacturing techniques for efficient server
processors
(4) Use of Dynamic Voltage and Frequency Scaling for
runtime power optimisation

Competitive Work: No directly opposing previous work, thus competitive
work was not required.

Reproducibility: Apart from more details, one would also need to
attain access to all the hardware used; hence not reproducible.

Question/Criticism: I did read a small part of this (http://
ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5472897&tag=1) paper,
and found it interesting. People might want to have a look.

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Dimitar]

Delivering Energy Propotionality with Not Energy-
Proportional Systems-
Proportional Systems-Optimizing the Ensemble

Authors: Niraj Tolia, Zhikui Wang, Manish Marwah,
Xiayun Zhu


Date:2008

Novel idea: In today's data centers power has become a major issue.
The paper advocates that
energy-proportional computing can be approximated by using software to
control power usage
at the ensemble level. The paper argues that with virtualization and
moving entire OSs and
applications in a low-overhead manner will enable new systems to
react better to changes in the
workload. The paper also argues that non system components should be
looked at in order to save
power such as fans, and air conditions.

Main Result/Evidence. The paper shows enclosure power usage using
three different policiy. The
policy with hardware base and frequency scaling VM migration
controller had the most power
saving because it allowed the system to greatly reduced power when
idle. Secondly, the paper
demonstrates the power saving using two different types of fan
controllers RFC and PFC. PFC aims
at minimizing fan power consumption by monitoring server utilization
and temperature sensors.

Impact: The ideas presented in this paper gives data centers a new
ways of saving power.

Prior Work : Dynamic thermal management of air cooled data center.

Reproducibility: The work reproducible because the authors don't
describe the design of their
software sensor in PFC.

Criticism: The authors mention that availability might be affected by
aggressive consolidation,
but it would have been better if they have shown how the system will
be affected in an experiment.

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Tom Wall]

Delivering Energy Proportionality with Non Energy-Proportional Systems
– Optimizing the Ensemble

Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash Parthasarathy
Ranganathan, Xiaoyun Zhu

Novel Idea:
The authors observe that in an ideal environment, a cluster's power
usage should scale with with its utilization. If doing no work, it
should consume no power, and at full utilization it should consume
100%. While most individual components do not support such operation,
it can be emulated to a certain degree in software at the
"ensemble" (i.e. groups of machines) level. They demonstrate how it
can be done with two case studies.

Main Result:
They show that it is indeed possible to have an energy proportional
system, even with limited hardware support. By running cluster level
optimizations they can scale the power roughly linearly with
utilization. They also note that there is much room for improvement.

Evidence:
Rather than detailing and testing a specific implementation, they
present two case studies of optimizing server hardware to achieve
energy proportionality. The first uses VM migration to easily move
application instances between servers so that VMs can be consolidated
onto fewer machines and the idle servers can be powered down. The
second example uses a predictive cooling model to regulate fan speed.
Both studies show improved power utilization.

Impact:
This seems like an easy way to save power if your application can
support it.

Similar Work:
They noted similarities to [4] which regulate data centers' air
conditioning and [14] which takes temperature into account when doing
load balancing. Their solution differs in that it emphasizes a more
finer grained approach and they are striving for proportionality
whereas the others simply aim to reduce power usage.

Criticisms:
I can't come up with any major criticism; I think it is a pretty well
done paper for what it is. They explain themselves well and their case
studies do a good job showing that their goals are indeed attainable.

My only complaint is that they don't really do very much. They only
demonstrate that it is possible to achieve energy proportionality, but
it seems obvious that that would be the case. I think a more
interesting paper would have been to implement and evaluate some sort
of general purpose framework for specifying and optimizing the data
center constraints to simplify the task of achieving energy
proportionality.

Future Work:
Incorporating more hardware into the optimization algorithms for
better performance. They only came within 15% power utilization
because things like network switches and disks could not be factored
in to their algorithms. By adding sensors to other hardware in a
cluster environment, this hardware can be considered and lower power
consumtion can be achieved.

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Hammurabi Mendes]

Paper Title

Delivering Energy Proportionality with Non Energy-Proportional Systems

- Optimizing the Ensemble

Authors

Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash Parthasarathy
Ranganathan, Xiaoyun Zhu

Date

December, 2008 - HP Technical Report

Novel Idea

Using software techniques to approximate the behavior of
energy-proportional systems with common hardware.

Main Results

They apply optimization techniques together with available
power-saving hardware features and virtual machine technology to
approximate energy-proportional systems. They consider both
traditional components and also peripheral systems as parameters of
their optimization procedure.

Impact

The paper shows that software can provide an approximation to energy
proportional systems. However, is appears clear that better
power-saving hardware features are important in performance-sensitive
cases.

Evidence

First they how the DVFS and DVFS associated with the ability to turn
equipments on/off perform under different utilization situations. They
also different techniques to control cooling and their consumption as
the utilization increases.

The analysis they provide is very single-dimensional. I talk more on
this on the Questions+Criticism section.

Prior Work

They certainly depend on hardware power-saving features (as DVFS), as
well as on good instruments to measure utilization and consumption.
Their approach also uses virtualization technology.

Competitive Work

I think that clear competitor to this technique is the approach of
using specialized, embedded instruments on each component (for
instance, processor-controlled instead of operating system controlled
cooling systems - this idea applied to other devices, such as routers,
air conditioners, etc).

Reproducibility

The experiments are not reproducible. They don't describe their
algorithms or the variables in the optimization problem.

Questions + Criticism

[Criticism] As their technique is software-based, without the
description of the algorithm (or simply the variables/constraints used
in the optimization problem), the experiments are not reproducible.

[Criticism] They do not evaluate the performance effects of software
control over the devices. [Question] Is the cost of moving VMs viable
on real workloads?

[Question] Moreover, if the industry provided their own specialized
power-saving algorithms/techniques applied to their devices, how would
it compare to the software-based approach?

Ideas for Further Work

Using *all* the available hardware features related to power-saving in
a real cluster, together with their optimization methods in a real
cluster, and evaluate the performance penalties.

On Wed, Oct 20, 2010 at 6:10 PM, Rodrigo Fonseca
<rodrigo...@gmail.com> wrote:

[Sandy Ryza]

Title:
Delivering Energy Proportionality with Non Energy-Proportional Systems

– Optimizing the Ensemble

Authors:
Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash, Parthasarathy

Ranganathan, Xiaoyun Zhu

Novel Idea:

The authors attempt to achieve energy-proportional computing (in which
the power consumed by a system is proportional to the extent to which
it is being utilized) through using software that minimizes power
consumption at the "ensemble" level by wrapping processes in virtual
machines and consolidating them to allow switching off systems. They
also present a "predictive fan controller" model for minimize fan
power usage.

Main Result(s):
The authors demonstrate that their methods can introduce a degree of
(but not full) energy proportionality. They show that their virtual
machines system can reduce server power consumed at 0% utilization to
15% of full utilization power, and that their cooling system can
reduce cooling power consumed by 32% of full utilization power.

Evidence:
Experiments were run on an HP c7000 BladeSystem enclosure, which has
16 server blades equipped with 16 GB of RAM and two AMD 2216 HE dual
core processors. 10 fans cool the system. Computation was run in 64
VMs, each which had 128 MB of RAM, a 4.4 GB virtual hard drive, and a
virtual CPU. They varied the utilization of each VM in 5% increments
and measured the full power used for the enclosure. Using the same
system, the compared their fan system with static fan power and a
reactive system, measuring power used for fans with different amounts
of utilization.

Prior Work:
The authors cite prior work that focuses on data center level cooling
efficiency by working with CRAC unit settings.

Competitive Work:
None that I know of.

Reproducibility:
While the setup is carefully detailed, little evidence is provided on
how their software interacts with the system and how optimization is
done.

Criticism:
The authors provide little insight into how they arrived at their
system and what types of applications it would be useful for. I
imagine that use of VMs incurs too much overhead for the kind of
applications we've looked at for other systems in class. The
experiments are also very limited.

Question:
Why is this paper so small? Is it supposed to serve a different
function than the other papers in this class?

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Zikai]

Paper Title: Delivering Energy Proportionality with Non Energy-
Proportional Systems – Optimizing the Ensemble

Author(s): Niraj Tolia (HP Labs) et. al

Date/Conference: HotPower '08: Workshop on Power Aware Computing and
Systems (December 2008)

Novel Idea: (1) Achieve energy-proportional computing on systems with
non energy efficient components by using software to control power
usage at ensemble level.
(2) Two strategies for delivering energy proportionality with non
energy-proportional systems: <1> Run workloads inside VMs in the
cluster. Migrate VMs (workloads) off under-utilized machines and turn
the idle machines off. <2> Use Predictive Fan Controller that
minimizes fan power consumption by exploring the variation in cooling
efficiency of different fans for different blades with the time-
varying demands of the workloads.

Main Results: The two strategies above work well in experiments.
Therefore, it is possible to use optimization-based techniques to

approximate energy-proportional behavior at the ensemble level.

Evidence: The two strategies are tested in Part 2 within a HP c7000
BladeSystem. In Part2.2, the first strategy is tested and compared
with no DVFS and DVFS only strategy. In Part 2.3, the second strategy
(PFC) is tested and compared with RFC.

Prior Work: Achieve data center level cooling efficiency by
manipulation of CRAC unit settings [4] or by temperature-aware
workload placement [14].

Reproducibility: For the first strategy, it is easy to implement it
and reproduce the experiment. However, description of the second
strategy is vague on how authors build the thermal model and how they
use the thermal model and the power model to model the problem as
convex, constrained optimization problem. Furthermore, the results of
the experiments may highly depend on cluster environments like spatial
layout of blades and fans, type of fan and air conditioning system,
their setup and so on. In sum, because of many vague or irreproducible
factors here, the paper has low reproducibility.

Question: Is the convex, constrained optimization method for fan
control scalable? First, build a thermal model for the entire data
center may be difficult. Second, if data center layout changes,
thermal model may become invalid and we have to remodel. Third, when
problem size grows, is the convex optimization solver able to solve
the problem in limited time?

Criticism: Authors may want to prove that the method in the paper is
scalable and applicable. For example, does it apply to something
larger than a single Blade system with 16 server blades which may have
complex spatial layouts and settings? Does it apply to clusters which
do not use VMs? Is it possible to migrate applications on modern
clusters into VMs?

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Joost]

Title: Delivering Energy Proportionality with Non Energy-Proportional
Systems – Optimizing the Ensemble
Authors: Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash,
Parthasarathy Ranganathan, Xiaoyun Zhu

Date: 2008
Novel Idea: Power costs are starting to drive the cost of running data
centers to a larger extent. As such CPUs have started being optimized
to deal with the load, however peripheral devices are no more
efficient than 10 years ago. By focusing on fixing the fan speed
algorithms the authors show that it is possible to drastically reduce
the fan power usage.
Main Results: By using the predicting cooling filters that the authors
proposed there was not only a decrease in cooling over baseline
results but also a statistically significant improvement over reactive
fan cooling models.
Impact: Using these filters fill be able to reduce the power usage by
server racks noticeably, however it is not enough to dam the
increasing tide of consumption.
Evidence: The tests that the authors ran did demonstrate the
aforementioned improvements in power usage. However, in non
homogeneous racks the authors claim that the model would continue to
work, but no empirical evidence was presented.
Prior Work: Previous work that worked to scale down peripheral units
including the DVFS work to lower overall load.
Competative Work: None mentioned.
Reproducibility: Given the algorithm and proper sensors on a rack the
results should be replicable.
Question: What overall portion of fan cooling has been reduced through
these algorithms?

On Oct 20, 6:10 pm, Rodrigo Fonseca <rodrigo.fons...@gmail.com> wrote:

[Duy Nguyen]

Title:

Delivering Energy Proportionality with Non Energy-Proportional

Systems – Optimizing the Ensemble

Authors:

Niraj Tolia, Zhikui Wang, Manish Marwah,...

Novel Idea:

This paper provides 2 case studies to indicate that it is possible to 

use optimization techniques to approximate energy-proportional behavior

at ensemble level. Energy-propotional computation is the one where the

power consumed by computers is proportional to what really utilizes CPU

resource.

Main Result(s)/Evidence:

In first case study, the approach is using VM migration technique in

conjunction with DVFS to experiment 3 scenarios: No DVFS, DFVS and DVFS

with VM migration. The result shows that only DVFS + VM migration can 

approximate energy proportionality at ensemble level.

In the second case study, they examine how fan control can be used to

achieve better energy proportionality for server cooling resources. They

propose 2 models to control fan energy: Reactive Fan Controller(RFC) and 

Predictive Fan Controller(PFC). RFC just bases on hardware sensor. PFC is 

like RFC but use a thermal model to predict future server temperature and

server utilization. After that, these data is passed to optimization solver

to set the fan speeds to values that minimize power consumption. Experiments

show that RFC gains better result.

Prior Work:

Hardware feature (DVFS), VM Migration, bin-packing,..

Competitive Work:

Not mentioned.

Reproducibility:

No, the paper just mentioned the main ideas.

Criticism:

Some arguments seem not to be convinced, such as they they spoke to the 

blade-system designers about the affect of turning off machines to hardware

components, and concluded that the answer is No.

On Wed, Oct 20, 2010 at 6:10 PM, Rodrigo Fonseca <rodrigo...@gmail.com> wrote: