deadline Tunables

[malaroth]

This should speed things up some. Any questions that's what this is for!

#!/system/bin/sh
# chmod -R 755 /system/etc/init.d

echo "deadline" > /sys/block/mmcblk0/queue/scheduler
echo 350 > /sys/block/mmcblk0/queue/iosched/read_expire
echo 3500 > /sys/block/mmcblk0/queue/iosched/write_expire
echo 6 > /sys/block/mmcblk0/queue/iosched/writes_starved
echo 0 > /sys/block/mmcblk0/queue/iosched/front_merges
echo 0 > /sys/block/mmcblk0/queue/iosched/fifo_batch
echo 2 > /sys/block/mmcblk0/queue/nomerge
echo 2 > /sys/block/mmcblk0/queue/rq_affinity

[Joaquín]

init script download link: http://d-h.st/Ji4

[malaroth]

But I Also wanted y'all to see the additional 2 commands at the bottom. First prevents merging which "works best with flash memory"-Francisco,while the second one forces the CPU that starts the batch to not swap to the CPU. Franco explained it better but it works

[franciscofranco1990]

These are winners for me, hands down.

[Joaquín]

i just tested it and damn they feel great. Much more superior than row or cfw.

Great job

[Chris]

What are we doing for read_ahead_kb here? 512 like we agreed felt best before?

[franciscofranco1990]

I'm changing my trees to have 512 by default yes.

[Zach (malaroth)]

I think that's the consensus. 512 read ahead

[Chris]

K testing these mofos out.

[Joaquín]

here is an init script to set it on boot in the meantime if anyone is interested http://d-h.st/TRE

[Chris]

That should be:

echo 2 > /sys/block/mmcblk0/queue/nomerges;

So everyone make sure you're testing it correctly.

[Joaquín]

beat me to it. I was double checking because some of them are not sticking with the init.d

[Chris]

Yup most won't stick via init.d because currently Franco is setting his defaults later in the boot, after the run-parts call. Add sleep 30 before the io tunables and it should work. :)

[franciscofranco1990]

Just pushed a new mako kernel with this new tunables. Lets see how users react!

[Chris]

rq_affinity also doesn't accept 2, seems it's a Boolean. What do we want for that, 1?

[franciscofranco1990]

Its not a boolean. Maybe Tuna kernel doesn't have the latest code for setting it to 2, kernel 3.4 has it natively. I'll merge it later to Tuna.

[Joaquín]

some of the values are not sticking even with sleep 50 on the init.d

Here is how things are set automatically on grouper:

360/3500/6/1/1/2/2

[Chris]

Ah must be that. Thanks Franco, I'll try 1 for now I guess.

[Chris]

Make that sleep 35 before the iosched tunables to make sure they get set correctly by init.d; 30 was borderline.

[Chris]

Yeah I just ran into that on my GN Joaquin, I think it varies boot to boot. Hopefully there's a way Francisco can set them early at boot like the interactive tunables, that way we can have full control via init.d without having to mess with sleep workarounds.

[Chris]

I completely agree with these winners, even with rq at 1 instead of 2 currently, this is the fastest my GN has been since JB4.1.2. The transition from app drawer to home screen is finally smooth again, and the Settings had no speed bump scrolling to the bottom when it got to the Accounts. Very very nice!

I'm running our winning gov tunables as well, 15000 / 95 / 700 MHz / 45000 / 15000 / 85, which I still think are a bit snappier than the shipped defaults in r365. :)

[franciscofranco1990]

IO tunables are set exactly when the interactive ones are being set, no sooner or later.

[Zach (malaroth)]

Sorry guys I've been kinda sick today. Yeah rq_affinity 1 works great on the Gnex and these numbers are pretty consistent. I think that these are rtm ready if they keep up like this.

[Chris]

Hmm okay, so any ideas why the iosched tunables would require a sleep 35 to get set via init.d, where interactive tunables don't? Francisco, to me that indicates they're getting set 30 seconds later in the boot, somehow, so if we set them any earlier via the init.d script, then your commands are coming later and changing them back.. :/

[steve.adams.x10]

I don't know if this is too obvious an answer or not but instead of running 2 scripts why not include io and interactive in 1 merged script?

[Chris]

Personally I've only ever used one script, which has my color settings, and anything else I'm testing set on boot at the moment in it, so that sort of rules that out..

[steve.adams.x10]

Haha yeah figured the simple idea was too simple, I'll have a think and let you know if I come up with anything.These linux codes aren't exactly my strong point I can just about read them and understand some. I'll think over it while at work this afternoon

[steve.adams.x10]

What about trying different scripts then for each action? 1 for colours another for interactive and another for io running in that order? I guess again that might be too simple perhaps...

[Chris]

I'll try it, but it still should turn out the same since init.d scripts get run in order of filename. I'm more wondering if Francisco can shed some light on why a sleep 35 would be needed for the iosched tunables via init.d when it isn't for interactive tunables if all the kernel defaults are set at the same time during the boot.

[Zach (malaroth)]

I am running 2 different scripts so that's ruled out. Once I finalize something I merge it into a "tweaks" single script. But with all this being experimental I wanted to be able to wipe a script without having to rewrite.

[Zach (malaroth)]

Francisco... the set on boot option in fK.u, what does that entail? Could it be something as simple as set on boot options in the app taking priority over or after the init.d script?

[Chris (osm0sis)]

Shouldn't be that either since we don't have iosched in the f.Ku bootservice yet. :/

[franciscofranco1990]

There is no boot service for IO options yet which is also a bit of a challenge because I'm reading from the directory, filling an array with the files in the dir and creating preferences on the fly. I'm still trying to figure out a good mechanism to set them on boot.

I have no clue why is needs sleep 35. The IO settings in my ramdisk are being done exactly at the same time as interactive settings:

on property:sys.boot_completed=1

write /sys/devices/system/cpu/cpufreq/interactive/above_hispeed_delay 15000

write /sys/devices/system/cpu/cpufreq/interactive/go_hispeed_load 90

write /sys/devices/system/cpu/cpufreq/interactive/hispeed_freq 1000000

write /sys/devices/system/cpu/cpufreq/interactive/min_sample_time 45000

write /sys/devices/system/cpu/cpufreq/interactive/timer_rate 30000

write /sys/devices/system/cpu/cpufreq/interactive/target_loads 80

write /sys/block/mmcblk0/queue/iosched/read_expire 350

    write /sys/block/mmcblk0/queue/iosched/write_expire 3500

    write /sys/block/mmcblk0/queue/iosched/writes_starved 6

    write /sys/block/mmcblk0/queue/iosched/fifo_batch 0

    write /sys/block/mmcblk0/queue/iosched/front_merges 0

    write /sys/block/mmcblk0/queue/nomerge 2

    write /sys/block/mmcblk0/queue/rq_affinity 2

But thats really strange indeed.

[Chris (osm0sis)]

Could a

write /sys/block/mmcblk0/queue/scheduler "deadline"

Before the iosched settings somehow make it behave, I wonder?

[Philip (Yellowdie)]

Holy cow people!  I don't check in for one weekend and y'all EXPLODE my email with mass messages!  lol.  Goodness glaciers.  :eek:

That said, where do y'all want me to start.  I don't want to grab the init.d file that was posted earlier if you've already moved three steps ahead.  What's the latest ordeal?  And where's the write-up about what each variable means and does?  I wanna do some readin'!  lol

[Zach (malaroth)]

This thread on xda was the main source though I Googled all the schedulers
http://forum.xda-developers.com/showthread.php?p=20890005
[REF] Governors and I/O schedulers

[Zach (malaroth)]

Perhaps I'm just being dense, not having a great deal of experience with programming, but would it be a viable solut on to create an intent in the boot service to pull the scheduler script? I was looking at this article and didn't know if it'd be more of a pain to try http://www.vogella.com/blog/2011/12/11/automatically-starting-services-in-android-after-booting/

[Chris (osm0sis)]

Nah, that's two different things we were talking about. Francisco's currently setting them well early during the boot sequence as part of one of the init.*.rc files. The boot service is what franco.Kernel updater uses to set it's stuff after the OS has fully booted.

[Steve (Gingerbread Man)]

If I were to download this script for deadline and have it in my init.d folder would it cause conflict if the ROW script was present but governor ROW was not the selected governor?

[Zach (malaroth)]

I'm not quite sure but I think it would. The init.d script is telling it to set the script on boot and it can't set row and deadline at the same time. It changes to either row or deadline. But I think Franco has it set as defaults so if you wanna run one script as main and then swap to the other then the the values will still be there

[Chris (osm0sis)]

The tunables we've tweaked (so far) wouldn't cause an issue since both schedulers use completely different names for them. The problem would be with the line that sets the scheduler to the one that isn't default. You could likely get around that by putting them all in one file and sleeping it all the way until the f.Ku bootservice sets the scheduler of your choice. I'll experiment with the combined script for us now and post back in a bit. ;)

 

Edit: By the looks of it cfq and bfq have roughly the same iosched tunable names, so we'd only be able to include one in the combined script.

[Joaquín (joaquinf)]

What if that line is deleted from the script and let the fku app select the desired scheduler?

[Chris (osm0sis)]

That's what I'm doing, but init.d happens sooooo very far before f.Ku does that, so a lot of sleep is required. About 45 seconds, but I'm finishing up a far better solution that even worked when I cleared the dalvik to make the boot take longer.

[Chris (osm0sis)]

Okay, I've got two working ideas, one that relies on f.Ku being on the device and one that should be universal since it's not looking for any particular tweaking app. The f.Ku one waited for f.Ku to run via its bootservice and then waited briefly and then set the tweaks; the universal one is only waiting for the systemui (surfaceflinger) to load, which happens pretty early, so then has to wait a fair amount longer after, but still not as long as without that conditional wait. Attached is the universal one. It should set the correct tweaked iosched tweaks for either deadline or row, on any device, with any ROM, and with any tweaking app choosing the scheduler, regardless of how long the OS takes to fully boot. Please let me know if it works as advertised. :)

 

This could also be a good solution for Francisco setting the tweaked iosched defaults regardless of what the kernel default scheduler is and what the user has chosen as their scheduler in their tweak app.

 

 

Edit: Reuploaded with an increased secondary wait time to accommodate the fat cow that is grouper.

 

Edit 2: Latest script version in the row thread.

[Steve (Gingerbread Man)]

Your script indeed works as advertised, only thing a limitation exists in the fact you have to reboot to load the tweaked values for each scheduler. Perhaps a line can be added that can detect the change without a reboot?

[Chris (osm0sis)]

Creating an init.d script that never ends is a bit of a nasty thing to do. ;)

[Steve (Gingerbread Man)]

Cant tell if serious?

[Chris (osm0sis)]

Serious. A script that runs forever is bad. Could potentially keep the device awake forever.

[minooch]

Yup,

you should NEVER keep a script alive - unless ofc you're a mad hatter & know the consequences of your actions ;) 

[Steve (Gingerbread Man)]

So going forwards to ensure the optimal values are used would be just to preset at these values. I guess the never ending script would drain battery due to keeping the device awake. When you said it would be nasty I couldn't work out if you meant nasty in a positive sense.

[Chris (osm0sis)]

Haha yeah, though really there must be a way to change those default iosched tunables in the code, and that would fix all the problems.

[franciscofranco1990]

Of course we can set the values on the source. I thought the values on the r366 GN kernel were fucking the device up, is that true? Do we have new winning and working values that I can set up in the source code?

[Chris (osm0sis)]

I just think people are being dramatic little babies because Play Store/Services likes to hang. Deadline doesn't cause any such problems on the N7 unless you're actually in the Play Store and trying to scroll through a list, like Installed Apps, All Apps or search results.

[Chris (osm0sis)]

It seems nomerges was causing at least some of the Play Store crap, so we should revisit some of the other winning values that may be affected by reenabling merges. I'm mostly thinking deadline here. pkgnex and I have already gone over them and come up with front_merges 1 and fifo_batch 2 or 4, what do you think Zach?

[Zach (malaroth)]

We had the winning deadline 350/3500/0/1 without the nomerges and rq_affinity values already nailed down. Let me go back and post Franco's reasonings in xda again for why the nomerges and rq_affinity was added

[Chris (osm0sis)]

Ah I see, I thought it was 1/1 before we got into the queue tweaks. And that feels kind of slow still now with JB4.2.2. And so does 0/1. :(

 

nomerges makes it more like sio/noop (first-in-first-out), rq_affinity makes it finish the task on the core it started it on.

 

Hopefully row is okay...

[Zach (malaroth)]

OK....so you got my attention with deadline. Here's what I'll be running with for the next day or two but already it feels snappy (like deadline usually is but I'll check benchmark and tuning as well.
#!/system/bin/sh
# chmod -R 755 /system/etc/init.d
# 7-Mar-2013

# wait for android os
until [ `pidof com.android.systemui` != "" ]; do
sleep 1
done;
sleep 25;

# general queue tweaks
echo 1024 > /sys/block/mmcblk0/queue/nr_requests;
echo 512 > /sys/block/mmcblk0/queue/read_ahead_kb;
echo 2 > /sys/block/mmcblk0/queue/rq_affinity;
echo 0 > /sys/block/mmcblk0/queue/nomerges;
echo 0 > /sys/block/mmcblk0/queue/rotational;
echo 0 > /sys/block/mmcblk0/queue/iostats;

# deadline tweaks
echo 350 > /sys/block/mmcblk0/queue/iosched/read_expire;
echo 3500 > /sys/block/mmcblk0/queue/iosched/write_expire;
echo 6 > /sys/block/mmcblk0/queue/iosched/writes_starved;
echo 1 > /sys/block/mmcblk0/queue/iosched/front_merges;
echo 4 > /sys/block/mmcblk0/queue/iosched/fifo_batch;

[Chris (osm0sis)]

Nice! Try nr_requests at 512 and fifo_batch at 2 as well. Zadeis and pkgnex liked those. :)

[Chris (osm0sis)]

writes_starved 4 might be worth reconsidering too from my testing on the N7.

[Joaquín (joaquinf)]

zach test values + nr_requests at 512 and fifo_batch at 2 kicks ass on my grouper. Noticeably snappier 

[Zach (malaroth)]

Writes_starved back at 6 for me, front_merges 0 with nomerges disabled currently and Fifo_batch 2, nr_requests 512. Feels really good on the Gnex

[Zach (malaroth)]

Guess it'd also help to note that I've had a 950vmsettings running for the past week or so lemme know if you see something wrong
https://www.dropbox.com/s/kbww7cjoep0a2a0/950vmsettings

[Chris (osm0sis)]

K so we're between 350/3500/6/0/2 and 350/3500/4/1/2. writes_starved 4 felt a lot faster on my N7 than 6, and both felt about the same on my GN. And if I recall correctly, front_merges 1 seemed to improve the macro times in AndroBench somewhat consistently.

[Zach (malaroth)]

6/0/2 is mine. Nr_requests 512 rq_affinity 2 nomerges 0. Snappy and quick. 4 for writes starved isn't as responsive and adding in 1 for front merge with 4 writes starved feels worse. Haven't benchmarked them it's just subjective feel

[Zach (malaroth)]

Something we've done has bungled io speed on deadline. I know it's just a benchmark but when I was in the 1200s before on io operations and now low 800s. I'm resetting nomerges to 2, resetting nr to 1024. Fifo_batches and front merges to 0. Starting back where I know things were great.

[Chris (osm0sis)]

echo 512 > /sys/block/mmcblk0/queue/nr_requests;

echo 512 > /sys/block/mmcblk0/queue/read_ahead_kb;
echo 2 > /sys/block/mmcblk0/queue/rq_affinity;
echo 0 > /sys/block/mmcblk0/queue/nomerges;
echo 0 > /sys/block/mmcblk0/queue/rotational;
echo 0 > /sys/block/mmcblk0/queue/iostats;

These are the possibilities. We always used 512 rak, and nomerges and rotational have defaulted to 0 for a long time in f.K. That leaves rq nr and iostats. Franco also reverted some vm settings. Cache pressure was 300 for the longest time, for example.

[Zach (malaroth)]

I've reverted back to 200 cache pressure... thought it was 250 and had typed 25. I enabled nomerges and am at 6/0/0 again but with write expire lowered 500ms to 3000. So 351/3000/6/0/0 nr_requests 512 read ahead 512. These honestly are just the best settings for my Gnex. I'm glad pkGnex is getting great results but they aren't doing a damn thing for my phone

[Chris (osm0sis)]

We can't enable nomerges again because of the Play Store issues everyone was up in arms about.

[Chris (osm0sis)]

Just wanted to add the (condensed) input from the xda thread:

 

rey: 4/1/2 avg w speed, ~12.29 MB/s; 6/0/2, ~15.90 MB/s, max 17.7 MB/s. 4/1/2 better overall for feel.

Fen: 4/1/2 better than 6/0/2 for Play Store.

pkg: 4/1/2 takes the cake.

 

And nomerges back at 0 has fixed io speed for everyone else.

[Chris (osm0sis)]

So I was bored last night and decided to see if Steve's idea to have it always running would be feasible after all and I think I found a good way; I encased it in a sleep and it only sets the tweaks when it detects the scheduler has changed. Sleeping mostly should hopefully keep it from impacting performance. I also made sure that it doesn't impede the rest of the init.d scripts from running by ending the sleep loop with a "done &" which allows the loop to continue running independently.

I'm going to see if it affects deep sleep and if not, post it up to my d-h later today. :)

 

Edit: Bombs away! :P

http://forum.xda-developers.com/showpost.php?p=39784442

[Stu (Khrushy)]

Very nice work sir!

[Chris (osm0sis)]

Here's exactly what's in the "r372-osmod" modified ramdisk, since I know you guys will appreciate the details. :)

 

/init.tuna.rc (just the bottom):

on property:sys.boot_completed=1
 write /sys/devices/system/cpu/cpufreq/interactive/above_hispeed_delay 15000
 write /sys/devices/system/cpu/cpufreq/interactive/go_hispeed_load 95
 write /sys/devices/system/cpu/cpufreq/interactive/min_sample_time 45000
 write /sys/devices/system/cpu/cpufreq/interactive/timer_rate 30000
 write /sys/devices/system/cpu/cpufreq/interactive/target_loads 85
 write /sys/devices/system/cpu/cpufreq/interactive/io_is_busy 1
 write /sys/devices/system/cpu/cpu0/cpufreq/screen_off_max_freq 537600
 write /sys/block/mmcblk0/queue/nr_requests 512
 write /sys/block/mmcblk0/queue/read_ahead_kb 512
 write /sys/block/mmcblk0/queue/rq_affinity 2

 

/sbin/fixtunapower.sh:

#!/system/bin/sh
# 31-Mar-2013

# custom busybox installation shortcut
bb=/sbin/bb/busybox;

# disable sysctl.conf to prevent ROM interference with tunables
# backup and replace PowerHAL with custom build to allow OC/UC to survive screen off
$bb mount -o rw,remount /system;
$bb [ -e /system/etc/sysctl.conf ] && $bb mv /system/etc/sysctl.conf /system/etc/sysctl.conf.fkbak;
$bb [ -e /system/lib/hw/power.tuna.so.fkbak ] || $bb cp /system/lib/hw/power.tuna.so /system/lib/hw/power.tuna.so.fkbak;
$bb cp /sbin/power.tuna.so /system/lib/hw/;
$bb chmod 644 /system/lib/hw/power.tuna.so;
$bb mount -o ro,remount /system;

# disable debugging
echo "0" > /sys/module/wakelock/parameters/debug_mask;
echo "0" > /sys/module/userwakelock/parameters/debug_mask;
echo "0" > /sys/module/earlysuspend/parameters/debug_mask;
echo "0" > /sys/module/alarm/parameters/debug_mask;
echo "0" > /sys/module/alarm_dev/parameters/debug_mask;
echo "0" > /sys/module/binder/parameters/debug_mask;

# general queue tweaks
for i in /sys/block/*/queue; do
  echo 0 > $i/add_random;
  echo 0 > $i/iostats;
  echo 0 > $i/rotational;
done;

# set tweaks as scheduler defaults
scheduler=reset;
while sleep 1; do
  current=`$bb cut -d\[ -f2 /sys/block/mmcblk0/queue/scheduler | $bb cut -d\] -f1`;
  if [ $scheduler != $current ]; then
    scheduler=$current;
    if [ $scheduler == "deadline" ]; then

      # deadline tweaks
      echo 350 > /sys/block/mmcblk0/queue/iosched/read_expire;
      echo 3500 > /sys/block/mmcblk0/queue/iosched/write_expire;


      echo 4 > /sys/block/mmcblk0/queue/iosched/writes_starved;
      echo 1 > /sys/block/mmcblk0/queue/iosched/front_merges;

      echo 2 > /sys/block/mmcblk0/queue/iosched/fifo_batch;

    elif [ $scheduler == "row" ]; then
      # row tweaks
      echo 100 > /sys/block/mmcblk0/queue/iosched/hp_read_quantum;
      echo 75 > /sys/block/mmcblk0/queue/iosched/rp_read_quantum;
      echo 5 > /sys/block/mmcblk0/queue/iosched/hp_swrite_quantum;
      echo 4 > /sys/block/mmcblk0/queue/iosched/rp_swrite_quantum;
      echo 4 > /sys/block/mmcblk0/queue/iosched/rp_write_quantum;
      echo 3 > /sys/block/mmcblk0/queue/iosched/lp_read_quantum;
      echo 12 > /sys/block/mmcblk0/queue/iosched/lp_swrite_quantum;
      echo 10 > /sys/block/mmcblk0/queue/iosched/read_idle;
      echo 25 > /sys/block/mmcblk0/queue/iosched/read_idle_freq;
    elif [ $scheduler == "cfq" ]; then
      # cfq tweaks
      echo 8 > /sys/block/mmcblk0/queue/iosched/quantum;
      echo 70 > /sys/block/mmcblk0/queue/iosched/fifo_expire_sync;
      echo 125 > /sys/block/mmcblk0/queue/iosched/fifo_expire_async;
      echo 12582912 > /sys/block/mmcblk0/queue/iosched/back_seek_max;
      echo 1 > /sys/block/mmcblk0/queue/iosched/back_seek_penalty;
      echo 93 > /sys/block/mmcblk0/queue/iosched/slice_sync;
      echo 39 > /sys/block/mmcblk0/queue/iosched/slice_async;
      echo 2 > /sys/block/mmcblk0/queue/iosched/slice_async_rq;
      echo 0 > /sys/block/mmcblk0/queue/iosched/slice_idle;
      echo 7 > /sys/block/mmcblk0/queue/iosched/group_idle;
      echo 1 > /sys/block/mmcblk0/queue/iosched/low_latency;
    fi;
  fi;
# loop forever independently

done&

# lmk whitelist for common launchers
list="com.android.launcher org.adw.launcher org.adwfreak.launcher com.anddoes.launcher com.gau.go.launcherex com.mobint.hololauncher com.mobint.hololauncher.hd com.teslacoilsw.launcher com.cyanogenmod.trebuchet org.zeam";
sleep 60;
for class in $list; do
  pid=`pidof $class`;
  if [ $pid != "" ]; then
    echo "-17" > /proc/$pid/oom_adj;
    chmod 100 /proc/$pid/oom_adj;
  fi;
done;

 

I tried echoing all the general queue tweaks out to all the blocks but had a vague feeling that made things slightly worse, so I moved the non-overhead related ones back to just mmcblk0. Maybe someone else could look into that too at some point. I also wanted to add the -discard option to the fstab, but I couldn't find anything in franco's github about committing Samsung's patch to disable discard for brick bug affected EMMC chips, so I decided it wouldn't be safe to do so until we know for sure that it is or it gets added. I also just found where franco sets his vm tweaks - about halfway down init.rc, good to know!

Anyway, that's all from me until later in the week when I hope to finish getting those testing / issue tracker threads in order. Cheers, boys!
 

[Zach (malaroth)]

Bad ass osm0sis! Just flashed it and boot is even faster, gonna run like this for a couple days and let you know if (unlikely) I find any issues! Gratz man

[Joaquín (joaquinf)]

That's really nice from you chris. Thank you very much, i've loaded it on my maguro and so far is rock solid and very smooth. 

It's really cool to see all this collaboration and team work that you guys have done to improve the overall experience and performance of the kernel. Keep it up guys. Although i've been a little distant and MIA always remember ... 

You guys rock. Thanks for all the knowledge you've shared here. 

Keep it up, cheers

[Chris (osm0sis)]

Thanks fellas. It was something fun to do in the afternoon. :D

 

Though I don't like the idea of having multiple open-ended loops running, adding the latest 950iosettings to init.d seems to play nicely with the fixtunapower.sh version, with the init.d values winning, so this shouldn't interfere with our ability to test. I figure this should tide us over and be a decent way to do things until franco can figure out how to hardcode them directly into the scheduler source.

[Stu (Khrushy)]

Needed an:

if [date = April 1]

 write /sys/devices/system/cpu/cpu0/cpufreq/max_freq 192000

in there imo. ^^

[Paul (pkgnex)]

If anyone wants to help out with DOE data for Deadline, here's the data recording file like I provided for CFQ.  If you don't have Excel, responding with a text file as some of you did before is fine.  I can work with pretty much anything you send back (pkoe...@gmail.com), and I'll normalize everyone's data to a consistent mean.

There are 10 tests this time - the top 8 are for the 4 non-boolean tuning factors (all to be run with front_merges=1), the bottom 2 are duplicates of two of the runs with front_merges=0.  I chose the factors to encompass the current "winning" values and the "stock" values.  I hope that just repeating the two most extreme-value runs with front_merges off will give enough data to show what, if any, difference it makes.

I'll assume everyone who is going to participate for Deadline will have done so by Friday evening, unless someone tells me they want to be included and need more time.

Thanks!

[Zach (malaroth)]

I'm game Paul... I am running 350/3500/4/1/2. My entire premise with deadline was to shrink the deadline but also allow for writes to be processed. I personally like 350/3500/6/0/0 but alas it's not a favorite. Lemme brunch these numbers and I'll get back to you

[Paul (pkgnex)]

Thanks, Zach.  No hurry, but I'll look for your data sometime this week.

[Paul (pkgnex)]

After just running the analysis with two different benchmark sets (androbench and antutu) from my phone only, it looks like setting every number to the high value (750/7500/8/1/4) takes the cake... and it has over twice the statistical effect compared to the variances seen in CFQ.

I want more data from other people's phones before I jump to any conclusions, but this may be one of those times where increasing the values increases total actual throughput, but decreasing the values increases perceived performance (e.g. lower lag).  We'll have to see how the numbers work out when we have more data!

[Stu (Khrushy)]

Pretty busy this week, but I'm sure I'll find an hour or two to sneak some tests in somewhere.

/dl's

[Zach (malaroth)]

Paul try these numbers... 350/3000/6/1/2 or 6/0/2 I want to see what you find

[Chris (osm0sis)]

I'm in. Deadline is way more rewarding to tweak. I'll probably run the trials tomorrow, but seeing how more obvious results are with deadline I played with it and benchmarking for a couple hours this evening.

I think maybe we've been looking at deadline all wrong. I found lowering writes_starved improves write throughput (logically makes sense), a lower fifo_batch seems to keep the throughout smoother. Then I thought if we aren't favoring reads by starving writes anymore, perhaps we can just do it with the expires more effectively and not choke things up so severely.

So please try out 3500/750/1/1/1 and frig around with the first two numbers to see if you can find a better ratio for smoothness. For me this makes the averages in AndroBench micro test much better.

I tried equal 3500/3500 and that was slow, but 3500/500 brought the speed back then raising to 3500/750 seemed to improve throughput and reduce i/o choking.

[Chris (osm0sis)]

750 seems like the sweet spot for increased write performance. 3000 and 3500 both seem sluggish in feel. 3500 had the best throughout so I was leaning toward it with 3375 but only 3250 seems to bring the crazy speed back to the app drawer. So my current favorites are 3250/750/1/1/1.

Please try with 3500 and 3250 and let me know if you have similar results.

[Steve (Gingerbread Man)]

3250/750/1/1/1 on my n7 will consistently give write speed of 3.5 mb/s in my gnex write was 10.5+ mb/s each time

[Chris (osm0sis)]

Hmm vs. what?

[Steve (Gingerbread Man)]

I was getting around 17 before. Sorry for being so brief before, basicly I took the script from 31/3/13 and added 3250/750/1/1/1 as the deadline numbers. I tested with other io schedulars and they have gone to shit to giving similar results but achieving over 30 mbs read!I haven't tried removing the script and test again as ran out of time before work

[Chris (osm0sis)]

Make sure it's the right values if you're using osmod still. I had to switch back to regular r372 and use init.d since I guess it's just luck which loop wins on each boot. If I make another it will see if 950iosettings exists and not start its own loop if so.

Also I think AndroBench and SD Card Tester are flawed. Want awesome numbers in AndroBench? Raise the write file size to 3mb and the write buffer to 8kb. SD Card Tester on the other hand seems to be unbuffered I/O which explains why it's so prone to choking.

Anyway not sure they're super amazing tunables either. Just an experiment. I got 27-28 consistently for seq read where usually I'd see 26, over 7 for both seq write and rand read where I'd usually see 6 or less, and 0.26 for rand write where it was always 0.24 before. The numbers seemed great on my end, I was more concerned with the feel being off, but if you get bad numbers then maybe we'll call the experiment failed unless you can tweak the 3500/750 for something better.

[Steve (Gingerbread Man)]

My gnex has been fine so far as I can tell it was my nexus 7 giving funny results and I was using the SD tester in antutu. When I get home I should be able to sort it, I don't take my tablet to work as it will get stolen lol The thing is it didn't feel laggy even with it showing 3.5mbs write so the test it self isn't being reliable

[Zach (malaroth)]

OK team we're getting off subject here. We'll let Paul post out the numbers from the tests and when that's said and done we might have some more data to be able to understand which direction to go in. AnTutu and the other benchmark tools don't test very large read and write files.I know there's one out there I've used that you can set the file size and that's where we'll see the lag. I just havE to figure out what app it was. But back to what I was saying, don't distrust these tests just yet, don't be hasty. Let's let them run their course and even if we don't get solid numbers we might get some new ideas. Glad to see everyone still interested in the project! Till later...

[Chris (osm0sis)]

Haha yes. Those were my new ideas and there should never be a bad time to have new ideas or bring them up. ;)

 

AndroBench is the one you can customize the size in, like I said before. You can also customize the buffer which brings drastically different results.

 

I thought my device couldn't do over 0.26mb/s rand write but I raised the buffer from 4kb to 8kb and it does 1.0mb/s easily.

[Paul (pkgnex)]

I've been using AndroBench and AnTuTu, they aren't exactly mirror images of each other, but the data seems to come out similarly.

I agree with Zach's suggestion (of course, I'm biased).  Get Deadline data to me and let's see where the analysis points us.  It may be worthless, or it may point us somewhere new.  Either way, I think we may be able to better frame the discussion after we have some data - especially since Deadline seems to be showing some more statistical effects than CFQ did.

[Chris (osm0sis)]

Oh I agree, I'm running the trials now. Just saying it could be an interesting way to take deadline that we hadn't thought of before. I'm eager to see what the data says. :)

[Paul (pkgnex)]

On my phone alone, it was definitely interesting.  Not sure the data is giving us what we're asking for, but at least it broke some preconceived notions I had based on our collective subjective tweaking.  It will be interesting to see if my device is an anomaly, or if everyone else gets the same results!

[Chris (osm0sis)]

The tests don't seem to include having the read expire more than the write expire at any point, would it even be possible for them to point in that direction?

[Paul (pkgnex)]

Yeah, if either:

1. read expire effect shows that raising it helps a lot AND write expire shows lowering helps a lot

OR

2. the read expire cross-product with write expire shows a strong inverse interaction

Then those would be possible indications that we have everything all mixed up.  We wouldn't get definitive numbers, but this would lead me to recommend doing some additional tests with the numbers the other way around.

[Steve (Gingerbread Man)]

So an update to my situation with the n7, removed the 950iosettings and cleared Franco app data and still only getting 4mbs write with what ever is set as default within r48 with 28mbs read which is slower than before. The only other thing I can think of it I have downloaded a lot of music and all my photos from picasa (nearly 1000) to my device and I have 15gb free memory out of 32gb. Anyway I'm off to bed now catch you guys tomorrow

[Chris (osm0sis)]

Fair enough, run Lag Fix (fstrim), and maybe roll back to r47 for testing since r48 is a little funky. Thanks Steve. :)

[Stu (Khrushy)]

A couple of quick observations I came away with after finishing Paul's tests.

fifo_batch = 4 gave far more consistent read/write results than fifo_batch = 1. 

writes_starved = 2 gave much better *read* performance than writes_starved = 8.

The write/read expire I didn't get too much out of at a cursory glance, so I'll wait for Paul's DOE.

Best results for me were from setting #3 out of the DOE tests, which I've used as a starting point for further dicking around.

Currently I'm at:

250 / 7500 / 1 / 1 / 1

Interestingly, once I dropped the writes_starved to 1, the consistency of results started coming regardless of fifo_batch size, and so 1 gave the best throughput overall.

Edit:

Just tried Os' 3250 / 750 / 1 / 1 / 1 - It's good - a definite improvement over where we had it, but not as good for me as the 250 / 7500 / 1 / 1 / 1 I mentioned above - but the difference is very small between the two.

I'm going to run these two settings for a while casually and see what I notice.

[Chris (osm0sis)]

Pretty much exactly what I saw with the bottom three at 1 as well. :)

 

If you're in the mood for a bit more messing around can you try inverting the expires and see what you can come up with there as well? Logically it would be nice to still favor reads a bit.

 

Edit: Just saw your edit Stu, so hopefully you catch mine too, haha. I still think 3500/750/1/1/1 gives better throughput, maybe give that a spin. Also, Paul will probably scorn me for it but I feel increasing the kernel entropy pool size just slightly with echo 128 > /proc/sys/kernel/random/read_wakeup_threshold; improves the rand read+write performance just slightly, maybe makes the rand write a bit smoother, and more consistent. I'll save most of the talk on that one for the vm, etc. thread but just wanted to mention it. All in AndroBench of course.

 

I'll try your 250/7500 out. Please see if you can come up with something better based on 3500/750 too if you have some time.