Allowing any mix of settings in combination with enabling to run any test from any view saves you a lot of clicks.
This is a very rough mockup and I'm looking to improve it in iterations by feedback. With a bit of polish it will look way more attractive. A usable and attractive test suit adds to the overall impression of Cake.
@mark_story: "As an aside, is there a reason you don't just use CLI runner?"
@Phally: "Slick interfaces make stuff generally more fun to do"