Why isn't the computer ranking the most accurate?

[ERCougar]

I have no doubt that the Vegas poll is the most accurate. But there would be a problem with incentives if the BCS actually used it. There would certainly be an incentive to put two teams against each other that would generate more betting action rather than necessarily the best two teams. Yes, this would be limited to some degree by reputational effects but I think the Vegas poll would have significant room to manipulate the poll to their advantage (at least among the top three or four teams). This is not to suggest there are no incentive problems with the other polls. The coaches poll is a nightmare from an incentive point of view.

Agree on all counts. I don't think the Vegas poll should be used to rank or seed teams in a playoff, for both the reason you cited and that, as I mentioned earlier, there should be some reward for performance on the field through the whole season. Thus, Florida may be the best team on the field, but we shouldn't ignore their loss to Ole Miss.

I also agree the coaches poll potentially has all sorts of bias, the AP less so, and I guess the Harris is a mix. No computer poll is perfect, so my ideal situation would be a mix of Harris, AP, and several computer polls to seed teams for a playoff. It would be interesting to see the confidence intervals on the computer ratings.

[pelagius]

It would be interesting to see the confidence intervals on the computer ratings.

I will do it if I have a chance to do a little programming tonight. The intervals are, of course, different for each team and it makes sense to do it in terms of rankings. IOW, find the range of rankings where the hypotheses tests of ranking equality are statistical insignificant (technically this should be done taking into account a multiple hypotheses test inference problem but we won't worry about that). Not exactly the same as a typical 95% confidence interval but it makes more sense in context of a computer poll.

[pelagius]

A It would be interesting to see the confidence intervals on the computer ratings.

For an example I did Utah

W/L Computer Model

           Confidence Interval
Est. Rank      Low    High
4                1      45

MOV Computer Model

           Confidence Interval
Est. Rank      Low    High
13               5      38

As you can see, there is a lot of sorting on noise when it comes to computer models. On the other hand, its not like the human polls aren't doing their share of sorting on noise.

[ERCougar]

For an example I did Utah

W/L Computer Model

           Confidence Interval
Est. Rank      Low    High
4                1      45

MOV Computer Model

           Confidence Interval
Est. Rank      Low    High
13               5      38

As you can see, there is a lot of sorting on noise when it comes to computer models. On the other hand, its not like the human polls aren't doing their share of sorting on noise.

Is this based on your own computer models?

[pelagius]

Is this based on your own computer models?

Ya, those are mine; those are the only one I can do confidence intervals for but I think they are fairly representative of most W/L and MOV based models. The other models definitely would not have more statistical precision than mine.

[ERCougar]

Ya, those are mine; those are the only one I can do confidence intervals for but I think they are fairly representative of most W/L and MOV based models. The other models definitely would not have more statistical precision than mine.

Interesting. Is your MOV model straight up MOV or does it have some sort of discount for large MOV's?

[pelagius]

Interesting. Is your MOV model straight up MOV or does it have some sort of discount for large MOV's?

That one was RAW anadjusted MOV. I just did those two because they represent the two extremes. An adjusted MOV is easy to implement, though. When I do adjust I do something like the following:

Adjusted MOV = MOV if MOV <= 21
              = 21 + (MOV-21)^0.5 if MOV > 21

When you do that the Utes move from 13 to 12 [but the confidence interval would be basically unaffected].


Someday, I will clean up my code (and move it all to perl for portability) and make it available to anybody who wants it.