power consumption when running on battery

[Ivan]

Hi list,

After years of using stock fedora as my main OS, I migrated to Qubes a
few weeks ago and I'm really happy I made the switch. Kudos to devs and
contributors !

The only minus compared to my former fedora install is the increased
power consumption when running on battery. With casual browsing, no CPU
intensive task, I remember I was using around 4.5 - 5W (on a thinkpad
T450s with high end specs). With the same work pattern/panel
backlight/... in Qubes, I'm at 7+ W. A power increase is of course
expected given the overhead of virtualization, but it looks like some
things can be improved. For instance:

- enabling usb autosuspend in sys-usb and power savings for the wireless
adapter in sys-net saves aroud 0.5W (just as I'm typing that email, I'm
at 6.6W after those tweaks, compared to ~7.2W before).
- then, enabling various other power saving features in dom0 saves
another 0.6-0.7W. I hand picked the scripts from tlp [1] since
installing it from rpm in dom0 would require installing a lot of
dependencies / uneeded stuff.

Those 2 things saved almost 1.2W with no noticeable decrease in
performance (at least for me - YMMV). Automating this in Qubes should be
pretty straightforward:
- pull TLP and its dependencies in dom0, or create a set of Qubes
specific similar power saving scripts.
- install TLP in template VMs
- on BAT/AC event, add a udev rule in dom0 that would qvm-run "tlp bat"
in guest vms.

By the way I noticed that the IA32_ENERGY_PERF_BIAS MSR power saving
feature didn't work (it used to worked on that laptop with stock f22):

in dom0:

sudo x86_energy_perf_policy performance
sudo x86_energy_perf_policy -r
cpu0: 0x0000000000000000
cpu1: 0x0000000000000000
cpu2: 0x0000000000000000
cpu3: 0x0000000000000000

sudo x86_energy_perf_policy powersave
sudo x86_energy_perf_policy -r
cpu0: 0x0000000000000000
cpu1: 0x0000000000000000
cpu2: 0x0000000000000000
cpu3: 0x0000000000000000

ie, it's stuck in performance mode and can't be changed. I remember the
grsec patches exhibited the same problem since writing to MSRs was
disabled altogether. Is that the same with Qubes ? If yes, maybe make an
exception for that specific register ? (By the way I haven't made a
power consumption comparison when enabling/disabling that feature on
stock fedora, maybe it's negligible).

There are probably other features that could be helpful provided they
are enabled in Qubes' kernel, eg. the upcoming reworked intel PSR
feature for intel cards and panels that support it (like in newer
thinkpads). I'm also wondering how/if setting power saving features in
guests other than hardware (eg. vm/disk stuff) would impact power
consumption - I didn't have time to research that yet.

I'd be happy to help if people are interested. Tests, power consumption
figures, scripts/udev rules, ...

Cheers,
ivan


[1] http://linrunner.de/en/tlp/tlp.html

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Jan 15, 2016 at 03:00:06PM +0200, Ivan wrote:
> Hi list,
>
> After years of using stock fedora as my main OS, I migrated to Qubes a few
> weeks ago and I'm really happy I made the switch. Kudos to devs and
> contributors !
>
> The only minus compared to my former fedora install is the increased power
> consumption when running on battery. With casual browsing, no CPU intensive
> task, I remember I was using around 4.5 - 5W (on a thinkpad T450s with high
> end specs). With the same work pattern/panel backlight/... in Qubes, I'm at
> 7+ W. A power increase is of course expected given the overhead of
> virtualization, but it looks like some things can be improved. For instance:
>
> - enabling usb autosuspend in sys-usb and power savings for the wireless
> adapter in sys-net saves aroud 0.5W (just as I'm typing that email, I'm at
> 6.6W after those tweaks, compared to ~7.2W before).

Do you use USB mouse? I remember some problems with such devices when
autosuspend is enabled...
Anyway it may be configurable thing.

> - then, enabling various other power saving features in dom0 saves another
> 0.6-0.7W. I hand picked the scripts from tlp [1] since installing it from
> rpm in dom0 would require installing a lot of dependencies / uneeded stuff.
>
> Those 2 things saved almost 1.2W with no noticeable decrease in performance
> (at least for me - YMMV). Automating this in Qubes should be pretty
> straightforward:
> - pull TLP and its dependencies in dom0, or create a set of Qubes specific
> similar power saving scripts.
> - install TLP in template VMs
> - on BAT/AC event, add a udev rule in dom0 that would qvm-run "tlp bat" in
> guest vms.

Probably it isn't good idea to unconditionally tell all the VMs when
you're running on AC or BAT. But at least for VMs having access to PCI
devices it may be useful information.

> By the way I noticed that the IA32_ENERGY_PERF_BIAS MSR power saving feature
> didn't work (it used to worked on that laptop with stock f22):
>
> in dom0:
>
> sudo x86_energy_perf_policy performance
> sudo x86_energy_perf_policy -r
> cpu0: 0x0000000000000000
> cpu1: 0x0000000000000000
> cpu2: 0x0000000000000000
> cpu3: 0x0000000000000000
>
> sudo x86_energy_perf_policy powersave
> sudo x86_energy_perf_policy -r
> cpu0: 0x0000000000000000
> cpu1: 0x0000000000000000
> cpu2: 0x0000000000000000
> cpu3: 0x0000000000000000
>
> ie, it's stuck in performance mode and can't be changed. I remember the
> grsec patches exhibited the same problem since writing to MSRs was disabled
> altogether. Is that the same with Qubes ? If yes, maybe make an exception
> for that specific register ? (By the way I haven't made a power consumption
> comparison when enabling/disabling that feature on stock fedora, maybe it's
> negligible).

That's mostly because of Xen in between. There is xenpm tool to tweak
those parameters.

> There are probably other features that could be helpful provided they are
> enabled in Qubes' kernel, eg. the upcoming reworked intel PSR feature for
> intel cards and panels that support it (like in newer thinkpads). I'm also
> wondering how/if setting power saving features in guests other than hardware
> (eg. vm/disk stuff) would impact power consumption - I didn't have time to
> research that yet.
>
> I'd be happy to help if people are interested. Tests, power consumption
> figures, scripts/udev rules, ...

I've checked and installing tlp in dom0 indeed pulls a lot of useless
(maybe even harmful) stuff - like wireless-tools, cups, qpdf-libs etc.

But having relevant parts available there would be good idea. I guess it
would be stuff like HDD configuration, LCD brightness, CPU (but may need
some customization because of Xen) etc. At lest checking what would be
useful where (dom0, sys-net, sys-usb) would be really useful for further
work on that.

> [1] http://linrunner.de/en/tlp/tlp.html
>

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWnvYwAAoJENuP0xzK19csGywH/32naypamVnIrR5UvFygxU8P
NjokgibUiNU62ILrYWYlVhPnTsnYy1Yoy18frGKkvTjGxy71cJKWCqkKKTjIy9FE
eepoT3iOOx+RSNZb0J2fA5+0bOYMS/ETwZQXpnWXOlkrO1kkk8b+/MASzKJOMyAR
EMjVkIHkFWGvDSOEQigUeYOaDGPMco5vTFX2/NgARkL4KmoKaQgby7XTmSBRvJNU
lKI6n6cu6eh3eReQkPTBWIMpsSENnVVae29We1C4LDLSjxDCVp+Ut9Xllvbp9Mlf
588AFeuW5ou9nPbYDHLuGGbCQ5RgX8OIS7xmdIy+FkfP7l28dUIzB1H7lUaAdFo=
=qW+t
-----END PGP SIGNATURE-----

[Ivan]

On 01/20/2016 04:51 AM, Marek Marczykowski-Górecki wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On Fri, Jan 15, 2016 at 03:00:06PM +0200, Ivan wrote:
>> Hi list,
>>
>> After years of using stock fedora as my main OS, I migrated to Qubes a few
>> weeks ago and I'm really happy I made the switch. Kudos to devs and
>> contributors !
>>
>> The only minus compared to my former fedora install is the increased power
>> consumption when running on battery. With casual browsing, no CPU intensive
>> task, I remember I was using around 4.5 - 5W (on a thinkpad T450s with high
>> end specs). With the same work pattern/panel backlight/... in Qubes, I'm at
>> 7+ W. A power increase is of course expected given the overhead of
>> virtualization, but it looks like some things can be improved. For instance:
>>
>> - enabling usb autosuspend in sys-usb and power savings for the wireless
>> adapter in sys-net saves aroud 0.5W (just as I'm typing that email, I'm at
>> 6.6W after those tweaks, compared to ~7.2W before).
>
> Do you use USB mouse? I remember some problems with such devices when
> autosuspend is enabled...
> Anyway it may be configurable thing.

hmm, no problem here with autosuspend and external USB mouse or with a
thinkpad external keyboard with integrated trackpoint.

>
>> - then, enabling various other power saving features in dom0 saves another
>> 0.6-0.7W. I hand picked the scripts from tlp [1] since installing it from
>> rpm in dom0 would require installing a lot of dependencies / uneeded stuff.
>>
>> Those 2 things saved almost 1.2W with no noticeable decrease in performance
>> (at least for me - YMMV). Automating this in Qubes should be pretty
>> straightforward:
>> - pull TLP and its dependencies in dom0, or create a set of Qubes specific
>> similar power saving scripts.
>> - install TLP in template VMs
>> - on BAT/AC event, add a udev rule in dom0 that would qvm-run "tlp bat" in
>> guest vms.
>
> Probably it isn't good idea to unconditionally tell all the VMs when
> you're running on AC or BAT. But at least for VMs having access to PCI
> devices it may be useful information.

Enabling power features in all VMs could be beneficial too. For instance
I'm not familiar with how xen's storage block device deals with syncs: I
imagine a sync in the guest will be propagated to the physical device,
so tweaking the commit timeout in the guest should decrease HD usage
(which may not even be noticeable with SSDs but that's for the sake of
the example).

>
>> By the way I noticed that the IA32_ENERGY_PERF_BIAS MSR power saving feature
>> didn't work (it used to worked on that laptop with stock f22):
>>
>> in dom0:
>>
>> sudo x86_energy_perf_policy performance
>> sudo x86_energy_perf_policy -r
>> cpu0: 0x0000000000000000
>> cpu1: 0x0000000000000000
>> cpu2: 0x0000000000000000
>> cpu3: 0x0000000000000000
>>
>> sudo x86_energy_perf_policy powersave
>> sudo x86_energy_perf_policy -r
>> cpu0: 0x0000000000000000
>> cpu1: 0x0000000000000000
>> cpu2: 0x0000000000000000
>> cpu3: 0x0000000000000000
>>
>> ie, it's stuck in performance mode and can't be changed. I remember the
>> grsec patches exhibited the same problem since writing to MSRs was disabled
>> altogether. Is that the same with Qubes ? If yes, maybe make an exception
>> for that specific register ? (By the way I haven't made a power consumption
>> comparison when enabling/disabling that feature on stock fedora, maybe it's
>> negligible).
>
> That's mostly because of Xen in between. There is xenpm tool to tweak
> those parameters.

Aahh - I'm really new to xen and I didn't know about that tool, thanks
for the pointer ! Enabling various power saving features seemed to save
another watt without hampering performances too much, but I'll have to
do more thorough tests.

>
>> There are probably other features that could be helpful provided they are
>> enabled in Qubes' kernel, eg. the upcoming reworked intel PSR feature for
>> intel cards and panels that support it (like in newer thinkpads). I'm also
>> wondering how/if setting power saving features in guests other than hardware
>> (eg. vm/disk stuff) would impact power consumption - I didn't have time to
>> research that yet.
>>
>> I'd be happy to help if people are interested. Tests, power consumption
>> figures, scripts/udev rules, ...
>
> I've checked and installing tlp in dom0 indeed pulls a lot of useless
> (maybe even harmful) stuff - like wireless-tools, cups, qpdf-libs etc.

Yep - that's why I didn't pull it but only took pieces here and there.

>
> But having relevant parts available there would be good idea. I guess it
> would be stuff like HDD configuration, LCD brightness, CPU (but may need
> some customization because of Xen) etc. At lest checking what would be
> useful where (dom0, sys-net, sys-usb) would be really useful for further
> work on that.

Cool, happy to read that you are interested. Right now I have most of
the stuff implemented in dom0 and VMs (eg. tlp , xenpm commands, ...)
but there's nothing automatic yet ; it should just be a matter of
writing a few udev rules. Then I'll need to adapt tlp scripts, or just
use tlp scripts as-is and write custom scripts for the xen part. It
should be quite easy. I guess much more time will be spent on
measurements: tweaking one parameter, measuring any change in power
consumption, reiterate, ..., while making sure the system is in a
reproducible/stable state.
One problem is that my new thinkpad's power_now reading takes *minutes*
to stabilize. In comparison my older T400 had an almost realtime
reading. I'll try with an amp-meter + volt-meter and see if I can get
accurate results.

I'll write when I have more results; once I find a good set of settings,
I'll try to compare the consumption with a default fedora install to get
an idea about Qubes' overhead (that was one of my concerns a while back
before trying Qubes and I couldn't find any answer [1], I'm sure I'm not
alone !).

[1] http://old.lwn.net/Articles/646765/

[Cyril LEVIS]

Hi,

Do you have news about this subject?

I used to use TLP on my T450s, i have instal it in dom0 but i think there is more thing to do with qubes?

For lenovo T4x owner, I plan to install kernel-devel and install kmod acpi_call from rpmfusion to be able to set battery thresholds, very usefull for his life.

I'm interested, If I can help, doing some test, say me :)

[Cyril LEVIS]

Anyone try to play with xenpm? http://wiki.xenproject.org/wiki/Xen_power_management ? And cpuidle boot settings?

[Ivan]

Hi,

On 05/30/2016 04:36 PM, Cyril LEVIS wrote:
> Hi,
>
> Do you have news about this subject?

Unfortunately I didn't have time to do thorough tests as I originally
planned to.

> I used to use TLP on my T450s, i have instal it in dom0 but i think there
> is more thing to do with qubes?

I did the following:

1- VMs: installed the tlp package.

2- dom0: since the tlp package is pulling too many unneeded dependencies
on fedora, I reviewed the scripts and installed them from source. Here's
a list below.

Note: some of the scripts are probably never called (eg. wifi, wwan,
...) since there's no network adapter in dom0, and they wouldn't work
anyway since some packages/functionality are missing (eg. rfkill).
The scripts are quite straightforward, with a little bit of time I could
probably filter out everything that is not needed.

/usr/local/bin/bluetooth
/usr/local/bin/run-on-ac
/usr/local/bin/run-on-bat
/usr/local/bin/tlp-stat
/usr/local/bin/tlp-usblist
/usr/local/bin/tlp-pcilist
/usr/local/bin/wifi
/usr/local/bin/wwan
/usr/local/sbin/tlp
/usr/local/lib/pm-utils/sleep.d/49tlp
/usr/local/share/tlp-pm
/usr/local/share/tlp-pm/tlp-rf-func
/usr/local/share/tlp-pm/tlp-functions
/usr/local/share/tlp-pm/tlp-nop
/etc/systemd/system/tlp-sleep.service
/etc/systemd/system/tlp.service
/etc/udev/rules.d/85-tlp.rules

Configuration is in:
/etc/default/tlp


3- On AC/BAT change I exec tlp in dom0 and in VMs and I set a few xenpm
parameters. For instance see:

https://github.com/taradiddles/qubes-os/blob/master/tlp.sh

Notes:
- since I haven't done any real measurement I can't say how much the
xenpm changes help. For instance I've picked a migration delay of 1000,
but that could be totally wrong.
- there's definitively an effect of all those changes though, the power
consumption drops by a few watts, which is great. But the min
consumption is still quite higher than a default fedora install, which
is to be expected given the overhead of running VMs, but also because
Qubes' kernel lack newer graphic power saving features like PSR & co.

That's it basically, so it's quite rough...

>
> For lenovo T4x owner, I plan to install kernel-devel and install kmod
> acpi_call from rpmfusion to be able to set battery thresholds, very usefull
> for his life.

Yep, the missing acpi_call module is a problem for me too.

I didn't go the kmod/rpmfusion route as I didn't want to install that
kind of stuff in dom0. Instead I have a minimal fedora install on a
memory stick, and I run it in single user mode when I need to reset
thresholds - that is, when the battery runs down completely and the
thinkpad's EC looses its settings - so very rarely.
Security wise, running another distro is not a good idea at all, but I
haven't found a better way.
I was thinking of opening an issue about including the acpi_call module,
but IIRC it wasn't included in fedora because of a license
incompatibility. Or maybe it was just that the module is too dangerous
when not used properly.

An alternate way would be to have a passthrough mechanism between a VM
with acpi-call installed, and the thinkpad's EC in dom0. Security-wise,
only the required acpi calls could be allowed, but who knows what kind
of exploits this would open the door to.

>
> I'm interested, If I can help, doing some test, say me :)

Well, doing measurements would be nice. But it's difficult to get right,
as you have to repeat the same workload over and over, over long periods
of time in order to obtain decent values, and you have to tweak one
parameter at a time.

Hope this helps,

Ivan

[Cyril LEVIS]

Thanks you very much, I will try this. Which laptop do you own?

I think I must put your tlp.sh in /etc/pm/power.d? It is right?

Le mardi 19 janvier 2016 20:04:39 UTC+1, Ivan a écrit :

[Chris Laprise]

On 05/30/2016 09:44 AM, Cyril LEVIS wrote:
> Anyone try to play with xenpm?
> http://wiki.xenproject.org/wiki/Xen_power_management ? And cpuidle
> boot settings?
>

I've played with xenpm and changing the CPU speeds. I have a 'todo' to
write some battery event scripts to lower CPU speed on battery, as well
as turn off turbo mode.

Ivan's approach is more comprehensive, and I would love it if he could
lower my power consumption by a few watts.


Chris

[Ivan]

FYI, I've updated the scripts and instructions at

https://github.com/taradiddles/qubes-os/tree/master/powermgnt

It's still a work in progress though (hardcoded paths, should catch more
error, ...), so no warranty given :)

BTW I didn't find a way to list running VMs reliably (qvm-ls's output is
not very script friendly), so I ended up adapting qvm-ls to my needs.

When time permits I'll try to write a spec file for TLP to ease the
installation of tlp in dom0; having a - say - qubes-tlp rpm would be
less messy than copying files by hand.

Ivan


>
>
> Chris