The Letdown - Let's do some research!

[medinabuckeye1]

I agree with @bwarbiany and I think his post in defense of @OrangeAfroMan 's point is very well stated.  

The problem, as I see it, with what @utee94 and others are advocating is that it is highly subjective and we could spend the entire off-season arguing about which teams were sufficiently disappointed to belong in the group and which were not.  In contrast, OAM's definition avoids that entire debate by laying out a very narrow and very specific definition.  I like that.  

Now, as I see it, the major problem with OAM's definition is that even after going back ~30 years we are still dealing with a fairly small sample-size which leaves the possibility that the result could be just random noise.  

On the overall debate, I agree with OAM.  

First, it is indisputable that teams do not always play their "best game".  You can look at any conference or team for proof but I'll use tOSU because they are my team.  Looking at the whole seasons it would be silly to argue that Iowa-2017 and/or Purdue-2018 were better than tOSU in those respective years.  In spite of that, Ohio State lost to those teams.  Why?  Well, IMHO it is mostly because S*&T happens.  However, there are mental/emotional influences in a game played by college kids.  

I think OAM's data make a strong case.  It isn't THAT big of a gap but on the other hand a lot of these teams were playing opponents that were MUCH worse than they are.  

[utee94]

OAM set his criteria narrowly, that is true. He did so because he believes there is a qualitative emotional difference between a "final loss" and a "September loss".
It honestly makes sense. Ohio State, for example, had time to emotionally process the loss to Purdue. They knew it might be considered a damning loss by the committee, and continued their season. They ended up accomplishing some of their goals--beating Michigan, winning the CCG, and although they missed out on the CFP, they ended up in the Rose Bowl. And then, to top it off, they had an additional emotional goal--to send Meyer out on a high note.
Michigan, on the other hand, did not have a damning loss. They had a tough-fought close road loss to Notre Dame, a team that ended up undefeated and was a shoo-in for the CFP. All they had to do to get to the CFP was knock off Ohio State, and then they'd have as close to a lay-up in a CCG as you can get with Northwestern, and they'd have been in. But they lost. So instead of going to the CCG, instead of their consolation prize being the ROSE Bowl on NYD, they ended up losing and facing Florida in a rematch of the Citrus Bowl that nobody cared about. So much so that quite a few players even sat out.
Now, does that mean that OAM's criteria is a complete theory of quantifying "disappointed teams"? Probably not. But that doesn't invalidate that his criteria itself is meaningful.

If the entire point is to derive a correlation between disappointment and let-down losses, then excluding entire classes of disappointed teams only serves to weaken the hypothesis, and any conclusion that might be drawn from it.
Again I'll go to the 2008 Texas example because it's an important one.  That Texas team didn't lose a final game, it lost a tie-breaker without having any control at all.  And they watched their hated archrival-- that they'd already beaten in the regular season by double digits on a neutral field-- take away every single one of their postseason goals, effectively at the flip of a coin.  That was an unbelievably bitter outcome for that team.  I don't think this year's Georgia or Michigan could have been ANY more bitterly disappointed than that 2008 Texas team was.  
So eliminating entire classes of teams that should be considered, renders the data set incomplete.  And therefore, any conclusions drawn, are already faulty before we've even begun the experiment.
I get it, we're arguing opinions here, but being told by OAM that his view is the only acceptable one is way out of line, and I reject that.  If we're here for debate, great, let's debate it.
Otherwise, it's just as  Kris said-- OAM is sticking his fingers in his ears, closing his eyes as tight as possible, and ignoring all of the other potential factors that affect these classes of "disappointed" teams.

[betarhoalphadelta]

If the entire point is to derive a correlation between disappointment and let-down losses, then excluding entire classes of disappointed teams only serves to weaken the hypothesis, and any conclusion that might be drawn from it.  

That's fair. But there's a degree to which that's hard. If you're trying to figure out "disappointment", you have to realize that there are different kinds of disappointment. 
I'd wager that Ohio State 2017 or 2018 disappointment is a lot different than Michigan 2018 or Georgia 2018 disappointment. In both cases OSU was the conference champion in a strong P5 conference and was excluded. But in neither case was that disappointment all that much of a surprise to them. They knew going into the CCG that winning the game was in their control, but that the CFP was out of their hands. 
Conversely, Michigan and Georgia were both in a position where a win would pretty much undoubtedly put them in [although for Michigan they'd have needed to beat NU too], and a loss excluded them. Basically the outcome WAS within their control, and their own inability to prevail destroyed their shot at it. And again, unlike OSU, both teams also didn't hit their secondary goal of a conference championship. UM by not making it to the CCG, and UGA by losing the CCG. 

Again I'll go to the 2008 Texas example because it's an important one.  That Texas team didn't lose a final game, it lost a tie-breaker without having any control at all.  And they watched their hated archrival-- that they'd already beaten in the regular season by double digits on a neutral field-- take away every single one of their postseason goals, effectively at the flip of a coin.  That was an unbelievably bitter outcome for that team.  I don't think this year's Georgia or Michigan could have been ANY more bitterly disappointed than that 2008 Texas team was.

Again, there's a question of motivation. I don't think that Texas team was necessarily "disappointed", I think it's far more likely they were f'ing angry. It's the very fact that they believed they were every bit as qualified--probably more so due to H2H--as Oklahoma. They wanted to prove a point. It's a different feeling to be excluded because you didn't measure up on the field, rather than believing that you never got your shot and the team that got their shot should have been you. 
Do you honestly think that Michigan or Georgia this year had that sort of fire? That they wanted to prove that they were legitimate CFP contenders, and that they were unfairly excluded? Did Michigan have any argument that they should have been in over ND or over OSU [who didn't even get in]? Did Georgia have any argument they should have been in over Alabama? 
No, Michigan and Georgia were excluded because they lost. Texas was excluded because a team they beat by double digits won an irrelevant beauty contest. 

[utee94]

And beyond all of that, getting a near 50/50 result from a sample size that is already not statistically significant, isn't telling us anything, anyway.

This year Georgia will end up 2-3 against teams that will finish in the top 15.  Their "let-down" opponent Texas will end up 2-1 against the same group of teams.  Texas had a couple of bad losses for sure, but this is hardly representative of an opponent that had no chance at winning, unless Georgia experienced a "let-down."  Texas was completely capable of beating Georgia under any circumstances.

Michigan will end up 0-3 against teams finishing in the top 15.  Their "let-down" opponent Florida will end up 1-2 against that same group of teams.  Again, Florida had a bad loss, but the fact that Michigan couldn't beat a single Top 15 team doesn't make me feel like it was a "let-down" that caused their loss.  They just weren't good enough to beat ANY top 15 teams.  

[utee94]

That's fair. But there's a degree to which that's hard. If you're trying to figure out "disappointment", you have to realize that there are different kinds of disappointment.
I'd wager that Ohio State 2017 or 2018 disappointment is a lot different than Michigan 2018 or Georgia 2018 disappointment. In both cases OSU was the conference champion in a strong P5 conference and was excluded. But in neither case was that disappointment all that much of a surprise to them. They knew going into the CCG that winning the game was in their control, but that the CFP was out of their hands.
Conversely, Michigan and Georgia were both in a position where a win would pretty much undoubtedly put them in [although for Michigan they'd have needed to beat NU too], and a loss excluded them. Basically the outcome WAS within their control, and their own inability to prevail destroyed their shot at it. And again, unlike OSU, both teams also didn't hit their secondary goal of a conference championship. UM by not making it to the CCG, and UGA by losing the CCG.
Again, there's a question of motivation. I don't think that Texas team was necessarily "disappointed", I think it's far more likely they were f'ing angry. It's the very fact that they believed they were every bit as qualified--probably more so due to H2H--as Oklahoma. They wanted to prove a point. It's a different feeling to be excluded because you didn't measure up on the field, rather than believing that you never got your shot and the team that got their shot should have been you.
Do you honestly think that Michigan or Georgia this year had that sort of fire? That they wanted to prove that they were legitimate CFP contenders, and that they were unfairly excluded? Did Michigan have any argument that they should have been in over ND or over OSU [who didn't even get in]? Did Georgia have any argument they should have been in over Alabama?
No, Michigan and Georgia were excluded because they lost. Texas was excluded because a team they beat by double digits won an irrelevant beauty contest.

Neither you nor I have ANY way to know what you're suggesting.  I'm not willing to split those hairs, because we don't know.  I actually disagree with you and think Texas was more bitterly disappointed than Georgia or Michigan could have been, due to the fact that they lost everything to their archrival, but that doesn't really matter at all because neither of us can ever actually know.
Which is precisely why JUDGING various classes of "disappointment" completely invalidates the experiment.

[betarhoalphadelta]

And beyond all of that, getting a near 50/50 result from a sample size that is already not statistically significant, isn't telling us anything, anyway.

This year Georgia will end up 2-3 against teams that will finish in the top 15.  Their "let-down" opponent Texas will end up 2-1 against the same group of teams.  Texas had a couple of bad losses for sure, but this is hardly representative of an opponent that had no chance at winning, unless Georgia experienced a "let-down."  Texas was completely capable of beating Georgia under any circumstances.

Michigan will end up 0-3 against teams finishing in the top 15.  Their "let-down" opponent Florida will end up 1-2 against that same group of teams.  Again, Florida had a bad loss, but the fact that Michigan couldn't beat a single Top 15 team doesn't make me feel like it was a "let-down" that caused their loss.  They just weren't good enough to beat ANY top 15 teams.  

To be fair, this is a stronger critique than that of OAM's criteria. Being 1.5 games under .500 for these teams isn't exactly the most statistically strong statement.
However, I'd counter with a few things:

• I haven't gone through the data, but I would believe that even AFTER losing that final game, the teams OAM considers "disappointed" were more highly ranked than their bowl opponents. 
• Thus, the "disappointed" teams shouldn't actually be measured against .500, they should be measured against the theoretical win percentage expected based in the ranking differential, which I'm sure is above .500.
• To extend farther, it would be best to look at a more granular differential than the very noisy W/L record. Such as based upon either ranking or S&P, what is the expected scoring differential in these games and compare to actual. 

Perhaps #3 might be most important. W/L is a VERY noisy statistic. Score differential gives a lot more granularity, and might help. The game I'm most familiar with here, the 1998 Alamo Bowl, is a good example of that. KSU was ranked #3 in the BCS and Purdue came in unranked. I think on basically any measure, the game SHOULD have been a comfortable win for KSU. In the end KSU lost a really hard-fought back and forth game. Had they won by 4 points instead of losing by 3 points--i.e. if Brees hadn't completed that TD pass with 0:30 left in the 4th, all of a sudden OAM's analysis would have been 13-14 instead of 12-15. But I'd still argue that a 4-point win would be considered a letdown for KSU, even though they would have ended the day with a victory...

Now, all of these things dramatically complicate the testing, and would be a HELL of a lot more work. I don't know how many people are willing to go in and do all that work, so perhaps we'll never get to a truly statistically significant answer. 

But I do think that 12-15 as a record suggests that perhaps there's something there. You can deny that it's enough evidence to PROVE anything, but doesn't it give you at least some suspicion? 

[utee94]

To be fair, this is a stronger critique than that of OAM's criteria. Being 1.5 games under .500 for these teams isn't exactly the most statistically strong statement.
However, I'd counter with a few things:

• I haven't gone through the data, but I would believe that even AFTER losing that final game, the teams OAM considers "disappointed" were more highly ranked than their bowl opponents.
• Thus, the "disappointed" teams shouldn't actually be measured against .500, they should be measured against the theoretical win percentage expected based in the ranking differential, which I'm sure is above .500.
• To extend farther, it would be best to look at a more granular differential than the very noisy W/L record. Such as based upon either ranking or S&P, what is the expected scoring differential in these games and compare to actual.

Perhaps #3 might be most important. W/L is a VERY noisy statistic. Score differential gives a lot more granularity, and might help. The game I'm most familiar with here, the 1998 Alamo Bowl, is a good example of that. KSU was ranked #3 in the BCS and Purdue came in unranked. I think on basically any measure, the game SHOULD have been a comfortable win for KSU. In the end KSU lost a really hard-fought back and forth game. Had they won by 4 points instead of losing by 3 points--i.e. if Brees hadn't completed that TD pass with 0:30 left in the 4th, all of a sudden OAM's analysis would have been 13-14 instead of 12-15. But I'd still argue that a 4-point win would be considered a letdown for KSU, even though they would have ended the day with a victory...

Now, all of these things dramatically complicate the testing, and would be a HELL of a lot more work. I don't know how many people are willing to go in and do all that work, so perhaps we'll never get to a truly statistically significant answer.

But I do think that 12-15 as a record suggests that perhaps there's something there. You can deny that it's enough evidence to PROVE anything, but doesn't it give you at least some suspicion?

To be fair, BOTH of the critiques I've raised are equally strong, and here's why--
To answer your final question, my response is: I can't really believe there's something there, unless we're considering all of the members of the class of "disappointed" teams.  Until we are, then this is a futile exercise.
(Although it's likely still futile even if we DID include all of the suggestions made by Marq, and me, and others.  Such is the nature of statistical significance. )

[betarhoalphadelta]

Neither you nor I have ANY way to know what you're suggesting.  I'm not willing to split those hairs, because we don't know.  I actually disagree with you and think Texas was more bitterly disappointed than Georgia or Michigan could have been, due to the fact that they lost everything to their archrival, but that doesn't really matter at all because neither of us can ever actually know.
Which is precisely why JUDGING various classes of "disappointment" completely invalidates the experiment.

Which is why you create an objective rule where you don't have to subjectively judge each team's disappointment. Which is exactly what OAM did:

Letdown team:  any team that is in line to play for the national championship if it wins its last game of the season (pre-bowl), but loses said game.
Playoff era:  top 4 team or ranked 5th and playing a top 4 team in its CCG/season finale.
BCS era:  top 2 team or ranked 3rd and playing a top 2 team in its CCG/season finale.
Pre-BCS era:  #1 team only, or #2 and playing the #1 team in its CCG/season finale.

It's true that this is more narrow than you want, and excludes SOME teams that you'd consider disappointed, like 2008 Texas. 
But you can't sit here and accuse OAM of cherry-picking. He laid out a very clear objective rationale of teams that know that they're basically in with a win and out with a loss, who lost. 
That doesn't include all classes of "disappointment". But the class of disappointment he considered at least suggests that there's something there worth further study.

[betarhoalphadelta]

To answer your final question, my response is: I can't really believe there's something there, unless we're considering all of the members of the class of "disappointed" teams.  Until we are, then this is a futile exercise.

Well, there's another question, then. You say "I can't really believe there's something there."

Do you affirmatively believe there's nothing there? I.e. that the "letdown" theory is bunk?
Or are you agnostic on the question?

[utee94]

Which is why you create an objective rule where you don't have to subjectively judge each team's disappointment. Which is exactly what OAM did:
It's true that this is more narrow than you want, and excludes SOME teams that you'd consider disappointed, like 2008 Texas.
But you can't sit here and accuse OAM of cherry-picking. He laid out a very clear objective rationale of teams that know that they're basically in with a win and out with a loss, who lost.
That doesn't include all classes of "disappointment". But the class of disappointment he considered at least suggests that there's something there worth further study.

Where did I suggest he was cherry-picking?  
I asserted that his definition is so narrow that it precludes a truly representative sample.  And therefore the experiment is faulty before it begins.  

[utee94]

Well, there's another question, then. You say "I can't really believe there's something there."

Do you affirmatively believe there's nothing there? I.e. that the "letdown" theory is bunk?
Or are you agnostic on the question?

I'm either agnostic, or I think there might be something to it.  KSU in 1998 is probably the best example IMO.
But Georgia and Michigan this past season are extremely poor examples IMO, because both had already proven to be completely capable of losing to teams of the same caliber (top 15 ranking) as those they faced in their bowl games.

[utee94]

If we include all of the "disappointed teams" and the result flips from 12-15 to 22-19 or something like that, does that flip your view on whether it's a "real thing" or not?

Should it?

[betarhoalphadelta]

Where did I suggest he was cherry-picking?  
I asserted that his definition is so narrow that it precludes a truly representative sample.  And therefore the experiment is faulty before it begins.

Sorry, I shouldn't say you accused him of cherry-picking. Someone in this thread or the Bowl SOC suggested that his criteria basically just confirmed his bias, which I thought was an unfair accusation. And you were complaining that his criteria didn't include that wider range [such as 2008 Texas] and was incomplete.

My apologies.  

I'm either agnostic, or I think there might be something to it.  KSU in 1998 is probably the best example IMO.
But Georgia and Michigan this past season are extremely poor examples IMO, because both had already proven to be completely capable of losing to teams of the same caliber (top 15 ranking) as those they faced in their bowl games.

If you're agnostic, I can accept that. I think other folks, such as Badge and Kris, are not agnostic but actively believe the theory is bunk. That once you strap on that helmet, "letdown theory" goes out the window. I wasn't sure whether you were in that group or not; based on how vociferously you were debating OAM, I may have lumped you in there.

I'm agnostic on this as well. I wouldn't say that OAM has proven his case by any means. But his analysis provided enough data to be intriguing. Which is about all one might expect on a message board.

It's not like he was saying that more carries equals lower YPC or something silly like that ;-)

[utee94]

Heh.  I'm a scientist and an engineer.  I simply want the process-- the experiment-- to be as precise as possible.  

I'll admit that OAM's dismissive nature occasionally rubs me the wrong way and leads me to debate things more... vociferously... than I might otherwise.