bfq Tunables

[Paul (pkgnex)]

Guys,

Feel free to leave BFQ to me until Thursday night, if you want.  In fact, I'm asking you to please trust me with this for a day or two.  That way you can concentrate on the Deadline subjective and final choice, which I think we all agree is more important and which I feel I have less to contribute on (all the contenders feel great to me!).

I do have one or two simple BFQ items "up my sleeve", but based on Zach's max_budget_async_rq find, I now want to try changing EVERYTHING - just me, and just once is fine.  I'll track the data, make a recommendation that you can all try Friday, then we can turn it in to Franco for a weekend/Monday release.  I will not stray from the default values unless I find items that are a significant help.

 

BTW one "up my sleeve" example:  I actually read that if you set max_budget and raising_rt_max_time to 0, BFQ "learns" your device and auto-scales vales.  It doesn't show up with max_budget, but raising_rt_max_time shows actual evidence.  Set it to zero, drive your phone like normal for a few minutes, and come back to check it.  When you left it (after setting it to 0), it immediately goes back to 256, but over time it fluctuates as needed -  after a few minutes of driving, mine is usually at around 1200.  As you continue to use BFQ with the "auto-tune" variables set as such, it feels progressively better, at least to me.  May be hard to benchmark, but hopefully it shows up in the numbers.  If not, I'll let you know.

 

I'm not saying that I don't want help (Zach already has and I welcome that), I just don't want to take our eyes off the deadline prize.

[Chris (osm0sis)]

So far, what we'd worked out in the process of the final cfq tuning was that bfq is best with the following:

 

back_seek_max = 16384

back_seek_penalty = 1

slice_idle = 0

group_idle = 0

 

Paul's handling this until Thursday. Francisco wants anything that's going in the Milestone by Friday.

 

Edit: Edited the OP with Paul's info from the cfq thread, so we know what's up.

[Stu (Khrushy)]

Go crazy mate. I'm excited to see what's up your sleeve ^^

[Paul (pkgnex)]

I'm about half way done with the BFQ testing I want to do, and no surprises so far. Everything stock except slice_idle and the seek_back_penalty being 0 and 1, respectively, which we also did for CFQ and make logical sense.

More to come...

[Paul (pkgnex)]

So here’s what I found for BFQ (data file attached, but it’s kind of a mess).

Leave all as current defaults EXCEPT:

back_seek_penalty = 1 (was 2, but 2 makes no sense for flash memory – we made this same change to CFQ.  The benchmark result was minimal, however.)

slice_idle=0 (was 7, but this REALLY chokes the benchmarks so it needs to be 0)

raising_max_time=0 (this allows BFQ to auto-tune this per your device’s capabilities and was the best setting I found).  NOTE: I am not sure if the default might not already be 0.  The problem is that right after setting, it goes to 256 as a start point, then adjusts from there.  Resetting to BFQ defaults in f.ku shows the same 256, but the behavior does not seem as good after a f.ku reset than it does after manually entering 0.  I recommend having Franco hard-code this to 0, then ensure it actually is auto-tuning (quick check) before releasing to the public.

raising_max_softrt_rate=2000 (was 7000, lowering to 2000 seemed to be a sweet spot that increased benchmarks significantly without hurting the variance).

My methodology was to start with defaults, then turn the next variable in the list down 50% and up 50% and test.  If there was improvement after changing from default, I did some iterations to find a quick “best” value.  I then picked the best value (if it was significant) and moved on to the next variable.  I selected the BFQ default for all parameters except as noted above.  Note that this methodology makes no attempt to find any interaction effects or find any “sweet-spots” except for the variables that changed from default.  Basically, this process just ensured we didn’t miss any big, obvious, low-hanging fruit!  My overall impression is that aside from the 4 variables listed, BFQ is much like CFQ in that most of the variables have only very small effects and would be a phenomenal pain in the ass to do any subjective evaluations with – especially given how many variables there are!

Please give these values a shot for a few hours sometime, and if you all agree they seem generally OK, we can make them the final numbers for Franco’s next update.  Also, I think v5 or v6 doesn’t matter with these based on my past testing, so if the issues some people are having with v6 can’t be ironed out and Franco has to revert to v5, these values are still good.

[Steve (Gingerbread Man)]

Nice work there paul, gonna give this a try on grouper now I'm home

[Steve (Gingerbread Man)]

Haha forgot there is no bfq on grouper :p

[Chris (osm0sis)]

Francisco just added it to manta on his git, so maybe grouper will see it soon too. :)

 

Edit: Err wait that's row.. they didn't have row yet. My bad.

[Paul (pkgnex)]

Gotta admit, I'm underwhelmed with BFQ v6.  Not sure if there is just some lagginess relative to v5, or if our new Deadline is just so fast it makes everything else pale in comparison!

Did anyone every figure out what's up with all the reboots on v6?  I've been running it for 2 days and haven't had it happen... maybe it's my tunables or some other rom-mismatch thing only for certain users.

[Paul (pkgnex)]

BASIC BFQ TWEAK NUMBERS:

(all defaults except change 4 items noted below)

back_seek_penalty = 1

slice_idle=0

raising_max_time=0

? raising_max_softrt_rate=2000

[Stu (Khrushy)]

The reboots are not the tuneables. Still getting random reboots with any combination of tuneables.

Also, I'm finding the results far more variable than just Defaults + back_seek_penalty = 1 and slice_idle=0. They're still good results, just a little more erratic with the seq read. Returning raising_max_softrt_rate to 7000 gives consistently slightly higher reads (only like 1MBps, but hey) for me in Antutu

And yeah, it's clearly not as fast as Deadline.

That said - I still think we should push these values out to Franco for the milestone - any combination that at least has b_s_p = 1 and s_i = 0 is going to drastically improve things over the defaults.

[Paul (pkgnex)]

I agree, Stu.  Although I was getting better AndroBench scores (42.4 vs 41.2) with rmsr=2000 instead of 7000, I know the other two are more important, particularly the two you mention.

No reason to not get them to Franco for the next build.  Having the default slice_idle KILLS BFQ.

[Chris (osm0sis)]

Yup I was thinking Franco could have this "quick tune" of BFQ for M5, but we'd leave it out of 950io since it isn't a "full tune." Does this make sense to everyone? We can call it a franco.Kernel exclusive! :D

[Stu (Khrushy)]

A Franco Exclusive eh?  Are you in marketing Chris?

If you're going to add the bfq settings to 950io then we need to start thinking about whether you're setting it for bfq or cfq (or, you know, at least throw an if statement in there to check before setting them), because currently 950io sets the appropriate cfq tuneables to bfq as well, which is a problem - I've commented out all the cfq related settings from my startups.

If Franco hardcodes them then at least that's not a problem.

[Paul (pkgnex)]

Good catch, Stu.  I forgot about that.

Also - if anyone else wants to try the last parameter in the BFQ settings 2000 (my fav) vs. 7000 (default) and over-ride me, that's cool.  The other ones are more important.

[Chris (osm0sis)]

What? No it doesn't. I've had a massive ifelse in there that checks the scheduler for ages now..?

 

Anyway, like I said, I'd rather leave it out of 950io since it's not a "full tune."

[Stu (Khrushy)]

Why so it does. I wonder how old the version I was looking at is ^^

My apologies Chris, you're already all over it.

[Chris (osm0sis)]

Haha no worries bro. Had to come up with something intelligent to make it set them automatically when the scheduler gets changed. :)

[Chris (osm0sis)]

Hmm.. Not seeing any difference with either r_m_s_r. It's just showing the normal oscillating variation between 34mb/s and 36mb/s on my device.

 

Edit: I just emailed Francisco with our completed deadline and Paul's awesome basic bfq optimizations. :)

 

Edit 2: Okay he got back to me, he'll be checking it out when he has time, but there's some real life stuff outside of his control he's dealing with at the moment. I don't want to say any more, but we're here for you Francisco.

[Steve (Gingerbread Man)]

I wonder if bfq performance will increase when what ever is causing the reboots is fixed? I went to work today with my phone in the official car dock, using Bluetooth and navigation. Removing it from the dock and dropping it by accident even managed to cause a reboot lol

[Chris (osm0sis)]

So weird, I've been on bfq for the majority of 12h now with no problem whatsoever.

Not seeing raising_max_time change yet though.. Still 256.

[Steve (Gingerbread Man)]

Something is broken during the input and saving of the value. Change 256 to 0, press save, exit, re-enter and it turns back to 256. I think the best way to get those little shits to set properly for now is with init.d for now if someone could add the various values into the finalised thread on the script somewhere. I can't really right now as at work and will take for ever on my phone.

[Chris (osm0sis)]

I think Paul said that's normal since it's "autotuning"?

[Steve (Gingerbread Man)]

I'm not sure I don't know what the actual default number is,I just guessed it was 256 as changing it appears to not do anything and the other numbers we have are from the default values. I haven't looked to check if the other values appear to save properlycoukd be worth checking

[Steve (Gingerbread Man)]

OK the other values are saving. Perhaps 256 is supoised to be there? Definatly feels a little more fluid than before. I'm not gonna have much time for testing today as I'm working till late tonight

[Chris (osm0sis)]

I think raising_max_time is already set to 0. I just rebooted and now it's 334! No init.d.

[Paul (pkgnex)]

Sorry I missed this discussion.

Go into f.ku, reset raising_max_time to 0, save, then exit.

Come back immediately and it will be 256, come back later and it will have likely adjusted up further.

I noticed that sometimes leaving the stock BFQ values (changing to cfq, then back to bfq, for instance) would show some of this auto-tuning, but I never saw it go over the 334 you mentioned.  When you do the zero reset, it seems to auto-tune to a much greater degree.  Could just be random chance, though.  Franco may be able to tell what the default input currently is, or just hard-code it to 0 to make sure!

EDIT:  Also, if you manually set to 256 in f.ku, I don't think the auto-tuning is enabled.

[Chris (osm0sis)]

https://github.com/franciscofranco/Tuna_JB_pre1/blob/fk_4.2/block/bfq-iosched.c

The first ones we want are near the top starting at line 68.. Not sure about the others though.. People aren't kidding about bfq being convoluted.

Edit: Found it! There's a second batch of tunable defaults set beginning at line 2927. Weird, I was right; raising_max_time is hardcoded to 0 by default.

[Chris (osm0sis)]

Interesting.. I set it to 0 and this time it went back to 334. Does it continue learning between boots somehow!?

[Steve (Gingerbread Man)]

After a reboot I have 332. Would you agree bfq feels smoother after a reboot? My initial impression is animations don't feel quite as laggy. Having looked again before posting I have 380!

[Steve (Gingerbread Man)]

Had a reboot after updating Worms 2; Armageddon, http://db.tt/dMGYCT4g uploaded last ksmg so we have one of these reports in the correct place.

[Chris (osm0sis)]

Exact same error as everyone else:

Unable to handle kernel NULL pointer dereference at virtual address 00000038

I posted about it in the tuna issues thread as well, for whatever that's worth.

[Paul (pkgnex)]

This one may be different:

[Chris (osm0sis)]

Nah, they're all the same. ;)

[Paul (pkgnex)]

OK.  Just making sure I didn't have something that might help Franco un-screw BFQ.

[Chris (osm0sis)]

I really hope he can figure it out too. I think there's a lot of good information in all of these logs.

 

They've got a specific virtual address "00000038" that's mentioned in all of them, then a detailed memory dump and stack backtrace.

[franciscofranco1990]

I have no idea why they are derping. I'm gonna try something else when I compile the Milestone for Tuna.

[Chris (osm0sis)]

I'd revert to v5 on the milestone for stability reasons, unless you want to let us test your fix out before hand. ;)

[franciscofranco1990]

Yeah, I will do that! 

[Chris (osm0sis)]

Perfect! :D

Can you still add Paul's "quick-tune" ?

back_seek_penalty = 1
slice_idle=0
raising_max_time=0

Basically just optimizes it better for flash media.