Why is the selection/seeding process so dorked?
stat monkey
2. Computers can do a better job of seeding in order of the best
predictors of performance.
Why is it like this? I am not sure the NCAA fully understands why
they do it this way, but here is my theory:
A truely rational process to select and seed using the best predictors
would put more emphasis on margin of victory during the season,
thereby changing the way teams play. Teams would try to run up the
scores. It happened in football.
Games can be funny this way.
Edward M. Kennedy
Dimineshing returns, anyone? It's not rocket science.
--Tedward
Edward M. Kennedy
> > 1. The Selection Committee has a long procedure with no stated goal
>
> Their goal is to slurp dook.
I thought that was *your* profession.
--Tedward
Donnie Barnes
> 1. The Selection Committee has a long procedure with no stated goal
Last I checked the goal was to fill the 65 team field. I can't remember
them failing to do that.
> 2. Computers can do a better job of seeding in order of the best
> predictors of performance.
Disagreed. Sample size isn't large enough and never will be.
> A truely rational process to select and seed using the best predictors
> would put more emphasis on margin of victory during the season,
> thereby changing the way teams play. Teams would try to run up the
> scores. It happened in football.
>
> Games can be funny this way.
And that would be done all to "fool" the computer into doing something a
sane rational person would *not* have done. So we choose to stay with sane
rational people who have a job nobody envies but all claim they could do
better. Unfortunately most would be doing it with much more tainted
glasses than those on the committee.
They do a fine job, and at the end of the day a great tournament will
happen whether they made three "wrong' choices or not. That's all they
really want is a great tournament.
--Donnie
--
Donnie Barnes http://www.donniebarnes.com 879. V.
Don Del Grande
>1. The Selection Committee has a long procedure with no stated goal
"Select the 34 best at-large teams; seed the teams; and, place the
teams into the championship bracket" - from the 2007 Men's Division I
Basketball Championship Tournament "Principles and Procedures".
>2. Computers can do a better job of seeding in order of the best
>predictors of performance.
>
>Why is it like this? I am not sure the NCAA fully understands why
>they do it this way, but here is my theory:
>
>A truely rational process to select and seed using the best predictors
>would put more emphasis on margin of victory during the season,
>thereby changing the way teams play. Teams would try to run up the
>scores. It happened in football.
Their intent is, first, to choose the top 34 teams that are not
conference champions, and then to try to seed them so the 1s are
better than the 2s, who are better than the 3s, and so on.
Here's the thing: they have to choose the teams that are the top 34
**right now**, as opposed to the best 34 **over the course of the
season**. The same goes with the seeding; who cares how well you
played in November - it's how well you're playing up through mid-March
that counts, since it's how well you play in the last three weeks that
determines who gets the gold trophy.
Don't be so sure that margin of victory isn't taken into account in
some way; I would think a 30-point win would be more representative of
how two teams compare to each other than a 10-point win decided by the
team that was behind fouling a lot.
Then again, how does "10-point margin of victory" tell the difference
between a game like that and Florida-Tennessee (where the closest
Florida got in the second half appears to be nine)?
-- Don
stat monkey
wrote:
> stat monkey wrote:
> >1. The Selection Committee has a long procedure with no stated goal
>
> "Select the 34 best at-large teams; seed the teams; and, place the
> teams into the championship bracket" - from the 2007 Men's Division I
> Basketball Championship Tournament "Principles and Procedures".
>
> >2. Computers can do a better job of seeding in order of the best
> >predictors of performance.
>
> >Why is it like this? I am not sure the NCAA fully understands why
> >they do it this way, but here is my theory:
>
> >A truely rational process to select and seed using the best predictors
> >would put more emphasis on margin of victory during the season,
> >thereby changing the way teams play. Teams would try to run up the
> >scores. It happened in football.
>
> Their intent is, first, to choose the top 34 teams that are not
> conference champions, and then to try to seed them so the 1s are
> better than the 2s, who are better than the 3s, and so on.
>
> Here's the thing: they have to choose the teams that are the top 34
> **right now**, as opposed to the best 34 **over the course of the
> season**.
> The same goes with the seeding; who cares how well you
> played in November - it's how well you're playing up through mid-March
> that counts, since it's how well you play in the last three weeks that
> determines who gets the gold trophy.
Show me where the NCAA or the Selection Committee state that that
is the goal. I see no statement of a goal of any sort.
I would agree that you can infer from their behavior and some of the
procedures that they are not seeding based on the course of the
season.
Cincy in 2000 is a clear example of a team that would have gotten a 1-
seed
based on the course of the season. They were down-graded due to
Kenyon Martin's broken leg in the last game before Selection Sunday.
>
> Don't be so sure that margin of victory isn't taken into account in
> some way; I would think a 30-point win would be more representative of
> how two teams compare to each other than a 10-point win decided by the
> team that was behind fouling a lot.
> Then again, how does "10-point margin of victory" tell the difference
> between a game like that and Florida-Tennessee (where the closest
> Florida got in the second half appears to be nine)?
>
> -- Don
I agree that they might take margin of victory in to accout in some
way, but they
are not good at it.
Kvam and Sokol show with a statistical hypothesis test that their LRMC
computer
ranking would do a better job at giving the Final Four higher seeds.
See the Seed
row in table 5:
http://www2.isye.gatech.edu/people/faculty/Joel_Sokol/ncaa.pdf
Significant at the .01 level and it goes to .007 if you add the 2006
results. Their ranking outperformed the seeding in 3 of the 4
regions in 2006.
Also see:
http://www2.isye.gatech.edu/~jsokol/lrmc/about_lrmc.html
BTW, I am pretty sure that Sagarin Predictor also outperforms the
seeding on this measure. The rankings are similar to the LRMC
rankings.
Now, the Selection Committee should be able to do as well or better
than LRMC. LRMC does not take injuries, illness, or suspensions into
account, so the Committee could improve on it. Instead, they work hard
and end up doing worse.
Kvam and Sokol say that the Selection Committe tends to put too much
stock on the winners of close games. A close game means the two
teams are about even.
Michael Johnson
Re: Why is the selection/seeding process so dorked?
(Don Del Grande) wrote:
Their intent is, first, to choose the top 34 teams that are not
conference champions, and then to try to seed them so the 1s are better
than the 2s, who are better than the 3s, and so on.
Here's the thing: they have to choose the teams that are the top 34
**right now**, as opposed to the best 34 **over the course of the
season**. The same goes with the seeding; who cares how well you played
in November - it's how well you're playing up through mid-March that
counts, since it's how well you play in the last three weeks that
determines who gets the gold trophy.
me:
Unless your Kansas. then you'll have to pay the price for apparently the
next two years for losing the 3rd game of the season to a tourney bound
team. I say that because some are suggesting that since we lost to
bucknell 2 years ago, we dont deserve better than a 3 seed. you have to
give florida a #1 because even though they're playing like shit right
now, they are the national champions.
RCJHGOKU
Constance Reeder
>
> Re: Why is the selection/seeding process so dorked?
>
> (Don Del Grande) wrote:
> Their intent is, first, to choose the top 34 teams that are not
> conference champions, and then to try to seed them so the 1s are better
> than the 2s, who are better than the 3s, and so on.
> Here's the thing: they have to choose the teams that are the top 34
> **right now**, as opposed to the best 34 **over the course of the
> season**. The same goes with the seeding; who cares how well you played
> in November - it's how well you're playing up through mid-March that
> counts, since it's how well you play in the last three weeks that
> determines who gets the gold trophy.
> ------------------
>
> me:
> Unless your Kansas. then you'll have to pay the price for apparently the
> next two years for losing the 3rd game of the season to a tourney bound
> team.
If you had a couple of RPI top-25 wins in the last two months, and
then got screwed, your bile might make sense.
I guess when you are a Kansas fan, you expect to get high seeds based on
your historical rep as an elite program, but oddly don't think that a
bad rep based on recent performance is warranted either.
Go figure.
--
Function in chaos, finish in style. -- Unknown
stat monkey
>
> - Show quoted text -
According to this report, history is nothing in the process:
http://www.kget.com/sports/playoffs/story.aspx?content_id=2f9dda2b-8fee-443c-8a2f-8471b933082f
20 sports writers were invited to participate in a mock selection
committee process in February. It's produced some interesting
articles. Try google news for "mock selection process"
Andrew
news:27666-45...@storefull-3256.bay.webtv.net...
> I say that because some are suggesting that since we lost to
> bucknell 2 years ago, we dont deserve better than a 3 seed. you have to
> give florida a #1 because even though they're playing like shit right
> now, they are the national champions.
No one is suggesting that, you just seem to have serious reading
comprehension problems...
Don Del Grande
> Don Del Grande wrote:
>> stat monkey wrote:
>> >1. The Selection Committee has a long procedure with no stated goal
>>
>> "Select the 34 best at-large teams; seed the teams; and, place the
>> teams into the championship bracket" - from the 2007 Men's Division I
>> Basketball Championship Tournament "Principles and Procedures".
>Show me where the NCAA or the Selection Committee state that that
>is the goal. I see no statement of a goal of any sort.
It depends on your definition of "goal".
You're right about one thing: nowhere does it define how a committee
member should determine what teams are "best". For all I know, some
use season-long play, and others give more weight to recent play.
-- Don
stat monkey
wrote:
The procedure for this year includes this principle:
"The committee shall select the best 34 available teams to fill the at-
large berths, regardless of conference affiliation."
http://www.ncaasports.com/basketball/mens/story/9183455
I don't recall seeing a list of principles in the procedure for
previous years.
There is no similar seeding principle, even for the preliminary S-
curve that could be based on a pure best ordering.
They say that each committee member may define best using his own
approach.
"While the various elements of the RPI are important in the evaluation
process, the tournament bracket each year is based on the subjectivity
of each individual committee member to select the best at-large teams
available and to create a nationally balanced championship."
Since "best" is not defined, you can't really define a performance
criteria to show that they are failing; that they are not seeding as
good (by their own measure) as a computerized process like the LRMC
ranking or Sagarin predictor. Analysis does show that some of the
seed upsets evidenced in the teams that reach the Final Four are
really just seeding errors that would be eliminated if they used the
LRMC or Sagarin Predictor seeding. And, some of the principles of
seeding force them to not follow the ideal seeding anyway, like not
putting any of the top 3 teams in a conference in the same bracket,
etc. The committee creates an initial seeding called the S-curve
prior do addressing all the constraints, but they don't release it to
the public.
You can't help but think that the S-curve is heavily influenced by the
RPI with grossly unperformed as measured by its within-region ranking
the Final Four in the Kvam and Sokol paper. It was the worst
performer of all they tested with a p-value of 0.004 vs LRMC, even
worse than the seeding. That implies that the Selection Committe may
improve a bit on the RPI during the seeding process.
James Gibson
>
> The procedure for this year includes this principle:
>
> "The committee shall select the best 34 available teams to fill the at-
> large berths, regardless of conference affiliation."
>
> http://www.ncaasports.com/basketball/mens/story/9183455
>
> I don't recall seeing a list of principles in the procedure for
> previous years.
>
> There is no similar seeding principle, even for the preliminary S-
> curve that could be based on a pure best ordering.
>
> They say that each committee member may define best using his own
> approach.
>
> "While the various elements of the RPI are important in the evaluation
> process, the tournament bracket each year is based on the subjectivity
> of each individual committee member to select the best at-large teams
> available and to create a nationally balanced championship."
>
> Since "best" is not defined, you can't really define a performance
> criteria to show that they are failing; that they are not seeding as
> good (by their own measure) as a computerized process like the LRMC
> ranking or Sagarin predictor.
This doesn't surprise me at all, but I'm not sure that using the
predictive ratings is really what we're after. I guess if it was
truly just the best 34 teams, then predictive ratings would be the way
to do. I actually don't see that as the total goal. I see using the
34 best teams, primarily using W-L as the main criterion, with a few
ulterior motives: punishing teams from power conferences for soft non-
conference scheduling, and rewarding teams that have done well who
haven't had a chance to play these teams. These two criteria
certainly have a poor effect on the predictive element of seeding.
On a smaller scale, you'd probably get more accurate results for the
seeding of the ACC tournament if you used George's ratings instead of
the conference standings. But I don't think that would be the way to
go, and I think a similar thinking is used when picking the NCAA field.
Don Del Grande
Don Del Grande wrote:
>> stat monkey wrote:
>> > Don Del Grande wrote:
>> >> stat monkey wrote:
>> >> >1. The Selection Committee has a long procedure with no stated goal
>>
>> >> "Select the 34 best at-large teams; seed the teams; and, place the
>> >> teams into the championship bracket" - from the 2007 Men's Division I
>> >> Basketball Championship Tournament "Principles and Procedures".
>> >Show me where the NCAA or the Selection Committee state that that
>> >is the goal. I see no statement of a goal of any sort.
>>
>> It depends on your definition of "goal".
>>
>> You're right about one thing: nowhere does it define how a committee
>> member should determine what teams are "best". For all I know, some
>> use season-long play, and others give more weight to recent play.
>
>The procedure for this year includes this principle:
>
>"The committee shall select the best 34 available teams to fill the at-
>large berths, regardless of conference affiliation."
>
>http://www.ncaasports.com/basketball/mens/story/9183455
>
>I don't recall seeing a list of principles in the procedure for
>previous years.
The same text appears in Appendix E of the 2006 tournament handbook,
and, except for the number 34, in the 2005 tournament handbook as well
(both times in Appendix E, under "Principles For Selecting At-Large
Teams).
>There is no similar seeding principle, even for the preliminary S-
>curve that could be based on a pure best ordering.
I assume the reason for this is, they have to allow for the ability to
change a team's seed by one row up or down (which could be up to seven
places in the S-curve) - for example, no region can have its 5 and 8
(or 6 and 7) seeds come from the same conference. Also, a pure
S-curve seeding may violate some other principles - for example, a
team's top three seeded teams have to be placed in different regions.
-- Don
stat monkey
But, using preditive rankings would tend to downgrade
a team for a soft non-conference scheduling, all else being
equal.
If you go to true punishment (beyond what predictive
rankings would do), then you are using the selection/seeding
policy to manipulate non-conference scheduling policy.
That seems like bad idea to me. Perhaps it's an unintended
consequence if it is happening now. Mostly, the process
is not well thought out.
> and rewarding teams that have done well who
> haven't had a chance to play these teams. These two criteria
> certainly have a poor effect on the predictive element of seeding.
>
> On a smaller scale, you'd probably get more accurate results for the
> seeding of the ACC tournament if you used George's ratings instead of
> the conference standings. But I don't think that would be the way to
> go, and I think a similar thinking is used when picking the NCAA field.
I don't see that. The ACC Tournament uses standings.
Lots of sports use standings to seed tournaments: pro baseball, etc.
But other sports, like the Olympics, Boxing, use current proformance
estimates in the selection process at least.
The NCAA process seems to be based more current
performance than standings. Of course, there are no
standings to use. But they say that they use injuries in
the seeding process, and clearly they do based on what
they did to Cincy in 2000. So, they are more current
performance based.
But if they go to the best predictive model, then margin of
victory would have more impact on selection and seeding,
and it would have a quantifiable impact, so that would lead
to teams running up scores.
I think the best thing to do would be to use the best predictive
model that does not use margin of victory or limits it to some
threshold like 15 points, for the selection process. They still
would have to tweak the seeding process for injuries
sometimes (but be very conservative about it) and for all
the constaints in the policies.
stat monkey
>
> - Show quoted text
I wonder if they could do better while still not violating the other
principles?
I think they could, but there is the problem that express use of
margin of
victory in the process would probably lead to teams trying to run up
scores during the season.
I am not sure if they could do better with no use of margin of
victory and without violating the constraints. No statistical study
has answered that question, to my knowledge.
jsh
"stat monkey" <tada...@yahoo.com> wrote:
>
> The procedure for this year includes this principle:
>
> "The committee shall select the best 34 available teams to fill the at-
> large berths, regardless of conference affiliation."
>
> http://www.ncaasports.com/basketball/mens/story/9183455
>
> I don't recall seeing a list of principles in the procedure for
> previous years.
>
> There is no similar seeding principle, even for the preliminary S-
> curve that could be based on a pure best ordering.
>
> They say that each committee member may define best using his own
> approach.
>
You would think by now that some former Selection Committee member would
have come out with a tell-all book or article that dishes the dirt on,
or at least lays out in more detail, the actual process.
Does anyone know if such information exists?
-- jsh
Edward M. Kennedy
> > The procedure for this year includes this principle:
> >
> > "The committee shall select the best 34 available teams to fill the at-
> > large berths, regardless of conference affiliation."
> >
> > http://www.ncaasports.com/basketball/mens/story/9183455
> >
> > I don't recall seeing a list of principles in the procedure for
> > previous years.
> >
> > There is no similar seeding principle, even for the preliminary S-
> > curve that could be based on a pure best ordering.
> >
> > They say that each committee member may define best using his own
> > approach.
> >
>
> You would think by now that some former Selection Committee member would
> have come out with a tell-all book or article that dishes the dirt on,
> or at least lays out in more detail, the actual process.
>
> Does anyone know if such information exists?
I've read about it over the years. There is criteria each
is supposed to take into account, and a process for selecting
the teams. I believe the seeding is done second, though some
could be done as they go (everyone picks the same top 2 teams).
A scorecard of sorts for each team is provided to each member,
and it reflects the specified criteria and maybe some other
stuff, I forget. The process is somewhat like a series of run-
off votes. The first half of the at large bids are quickly
chosen. The last five take a while.
The seeding is done largely in order the teams were picked
in, with the exception of automatic bids and rules about how
many same conference teams in a bracket or region. The lowest
at large seed is around 11 or 12, a reflection of how weak
the bottom ~15 conferences are.
For late tournaments like the ACC tournament, they draw up
contingency plans depending on who wins. I think they kinda
screw this up sometimes.
The non-scholarship Ivy League is 21st out of 32 conferences
according to Sagarin, Pretty impressive, actually.
--Tedward
Constance Reeder
They have been very forthcoming about the process all the way along, and
Jerry Palm always has had on his (now for-pay) site a good description.
This year they invited the press to a mock selection committee session.
There were a number of articles written about it, all of which confirmed
the exact things committee members have been saying for years:
1. RPI is not used directly.
2. They don't care what conference gets how many.
3. They don't try to make marquee matchups based on
old rivalries.
Here is one account:
http://www.statesman.com/sports/content/sports/stories/other/02/13/13bracket.html
--
I am convinced that life is 10% what happens to me and 90%
how I react to it. And so it is for you... we are in charge
of our attitudes. -- Charles Swindoll
jsh
jsh <hug...@uiuc.edu> wrote:
To follow up on my own post, I do realize we now get a post-mortem from
a committee member during the selection show, but it always seems
pre-scripted and is often mainly defensive posturing in response to
criticism of specific selections.
The fact that people are still so unclear on exactly which factors are
considered, and to what extent, seems somewhat unnecessary. I won't deny
that the mystique surrounding a closed-door meeting of this type can add
to the overall interest, but I still would prefer a little more
transparency as regards the selection criteria.
-- jsh
Edward M. Kennedy
> The fact that people are still so unclear on exactly which factors are
> considered, and to what extent, seems somewhat unnecessary. I won't deny
> that the mystique surrounding a closed-door meeting of this type can add
> to the overall interest, but I still would prefer a little more
> transparency as regards the selection criteria.
Beat up on a bunch of mid-majors out of conference and
get to .500 in conference, and you're in. Schedule some
good OOC teams to hide your strategy.
--Tedward
Constance Reeder
They don't hide a thing. Here is a good article:
http://sportsillustrated.cnn.com/2005/writers/luke_winn/03/09/inside.bkc/1.html
They have discussed the whole process in excruciating detail over the
years.
I have read all this stuff, and feel like I understand how the committee
and the process operates. Maybe that's why I dismiss all the conspiracy
theory stuff people put out...
--
There comes a time when you should stop expecting other people to make
a big deal about your birthday. That time is age 12. -- Dave Barry
Donnie Barnes
> To follow up on my own post, I do realize we now get a post-mortem from
> a committee member during the selection show, but it always seems
> pre-scripted and is often mainly defensive posturing in response to
> criticism of specific selections.
I always found it funny just how defensive they are in that post mortem,
too.
jsh
Constance Reeder <cons...@duxmail.com> wrote:
> >
> > The fact that people are still so unclear on exactly which factors are
> > considered, and to what extent, seems somewhat unnecessary. I won't deny
> > that the mystique surrounding a closed-door meeting of this type can add
> > to the overall interest, but I still would prefer a little more
> > transparency as regards the selection criteria.
>
> They don't hide a thing. Here is a good article:
>
> http://sportsillustrated.cnn.com/2005/writers/luke_winn/03/09/inside.b
> kc/1.html
>
Thanks, that was interesting.
Edward M. Kennedy
> > To follow up on my own post, I do realize we now get a post-mortem from
> > a committee member during the selection show, but it always seems
> > pre-scripted and is often mainly defensive posturing in response to
> > criticism of specific selections.
>
> I always found it funny just how defensive they are in that post mortem,
> too.
Lots of money is at stake, and there's always a few schools
going batshit about how they got screwed. It's as political
as, well, politics.
--Tedward
stat monkey
> On 2007-03-02, jsh <hug...@uiuc.edu> wrote:
>
>
>
>
>
> > jsh <hug...@uiuc.edu> wrote:
>
> >> In article <1172843467.422400.39...@s48g2000cws.googlegroups.com>,
>
> They have discussed the whole process in excruciating detail over the
> years.
>
> I have read all this stuff, and feel like I understand how the committee
> and the process operates. Maybe that's why I dismiss all the conspiracy
> theory stuff people put out...
>
> --
>
> There comes a time when you should stop expecting other people to make
>
> - Show quoted text -
That's a good article. I think jsh is talking about the specifics of a
given year. We know they use a first cut seeding without contraints
(the S curve), but we don't know what the S-curve was for any given
year, for instance.
Don Del Grande
Sounds like you're describing the NCAA's printed "principles and
procedures" (which, up until the late 1990s or so, was available only
to the schools themselves - sorry for the long URL):
http://www1.ncaa.org/membership/champadmin/basketball/d1_men/handbook/principles_and_procedures_for_bracketing.pdf
>For late tournaments like the ACC tournament, they draw up
>contingency plans depending on who wins. I think they kinda
>screw this up sometimes.
They had the same problem back when the old Pac-10 tournament was
played so the final ended after the announcements were made. Assuming
one team had to win its way in, it was usually, "Either Team Y (from
the Pac-10) or Team Z (from another conference) gets the spot,
depending on whether or team Team Y beats Team X in the Pac-10 final",
although at least once they did give a team two different seeds
depending on whether or not it won its final game.
-- Don