By The Numbers: How Good a Coach is Bronco?

It’s the crazy time of the college football calendar when the coaching meat-market is in full swing. It’s also the time of year where the fans of more stable programs get itchy, wondering if maybe another coach could take their team the promise land. BYU fans are not immune to the wandering eye and there has been quite a lot of talk recently about whether Bronco Mendenhall should be replaced. The argument for replacing Bronco generally boils down to the assertion that Bronco is underperforming because he has not gotten BYU to a BCS bowl due to frequent early or mid-season losses. The counter-argument is usually phrased like this: Bronco has a better winning percentage than LaVell, five 10-win seasons, seven straight bowl trips, and five bowl wins, so quit the whining. In the interest of full disclosure, I am a fan of Bronco’s coaching and of him personally, so I am by no means unbiased in this, which is why I decided to check the only unbiased source available - cold, hard stats.

Being a numbers guy, I decided to use some statistical machinations and some advanced metrics from the very smart folks at Football Outsiders. Specifically, I am using their overall, offensive, and defensive S&P+ ratings which measure the effectiveness and efficiency of a team, adjusting for the pace of the game and the quality of the opponent (for a more detailed description, go here). Using the S&P+ ratings for the past 7 years along with the results of the games played those years (note for simplicity I’ve dropped games against 1-AA opponents), I was able to calculate the expected points per game an offense with a given S&P+ rating should score against a defense with a given S&P+ rating and vice versa. I then use those to compute an expected margin of victory (MOV) for a given game. The actual MOV minus the expected MOV for a given game gives a measure of how many points the coach added to his team's MOV that day, or what I call the Points Added. Of course any number of things can effect a single game, but by averaging over several seasons, some trends emerge, which I attribute to the head coach. Because the team’s S&P+ ratings (which cover a whole season) go into the calculation, my Points Added statistic is really not a measure of the coach’s ability to assemble a quality team, but rather a measure of how well they coach their team in each game. Things like recruiting and offensive or defensive schemes shouldn’t impact these ratings too much.

In order to get a feel for how good of a coach Bronco is relative to his peers, I’ve chosen to compare him with 3 other coaches - Boise State’s Chris Petersen, Dennis Erickson formerly of Arizona State, and Utah’s Kyle Whittingham. Petersen and Whittingham were chosen because they’ve done what Bronco hasn’t in getting their non-AQ teams to undefeated seasons and BCS bowls. If I were to play psychologist, I would say that the two biggest reasons for the impatience with Bronco are Whittingham’s Sugar Bowl win over Alabama and Petersen’s Fiesta bowl win over Oklahoma. Finally, Erickson was chosen as an example of a not-so-successful coach who is still not a total failure so we can see what that looks like. If Bronco were to be fired, it would be for similar reasons to why Erickson was shown the door. For all four coaches I used games they coached from the 2005 through 2011 seasons, including bowl games.

First, let’s look at what fraction of each coaches’ games showed a Points Added statistic in one of eight touchdown-sized bins. Negative values mean a team’s margin of victory was smaller (or more negative) than it should have been. Positive values mean a team did better than it should have.

Clearly Chris Petersen is a man among boys here. His teams consistently do better than they should. In fully a third of his games his coaching is worth more than three touchdowns. Bronco fares well in this comparison, coming in a distant second. He is clearly worth positive points more often than he is worth negative, but by nowhere near the margins Petersen enjoys. Hot on Bronco’s heels is Whittingham who is just as likely to cost his team a single touchdown as he is to benefit his team one. Bringing up the rear is Erickson who cost his teams points more often than he provided them. Interestingly, while Bronco comes in 2nd overall, he is last in the +21 or more category, which indicates that generally Bronco doesn’t blow out opponents as often as other coaches do. That answers at least one question - overall Bronco is slightly better than Whittingham and his Sugar-bowl winning Utes, better than the not-quite-successful-enough Erickson, and not as good as Petersen.

Now let’s break it down even further by look at how each coach has done against the bad teams and the good teams. For my cut-off line I’ve chosen to use the average overall S&P+ rating of 205 as the dividing line. A S&P+ of 205 corresponds to teams like Piit, Utah, San Diego State, and Northwestern this season, so I call teams better than that above average and teams worse than that below average.

Once again Petersen is the gold-standard - his coaching is worth more than two touchdowns in 70% of the games he plays against bad teams. Petersen doesn’t just beat bad teams, he demolishes them. Again Bronco is a distant second with a strong showing in the +0 to +14 point range and the lowest percentage of blow-outs, meaning that Bronco beats bad teams but doesn’t hurt anyone’s feelings doing it. Whittingham is 3rd and Erickson is 4th - no surprises here.

If we now turn to how each coach does against good teams, things stay roughly the same for Petersen and Erickson, but now Whittingham and Bronco come out almost exactly even, with Bronco more likely than Whittingham to be worth positive two touchdowns or less and the opposite for the more than +14 points category.

Finally, to look at how Bronco does against the best teams, let’s turn to games against teams with S&P+ ratings over 230. During Bronco’s tenure as head coach that includes TCU 5 times, Utah 3 times, Boston College twice, and Notre Dame, Oklahoma, Florida State, and Nevada once each - a small sample size but it’s all we have.

Again, Petersen is clearly on top, only this time Bronco comes in third behind Erickson (who was apparently pretty decent in big games), but still better than Whittingham. Interestingly, Bronco, Whittingham, and Erickson all cost their teams points more often than they add points, so none of them are really that great against top competition.

So, what does it all mean? Statistically Bronco Mendenhall is clearly outperforming a coach like Dennis Erickson, even when adjusting for strength of schedule, and he narrowly but consistently outperforms Kyle Whittingham. Chris Petersen leaves him in the dust, but that’s why every major coaching search in the nation starts with Petersen’s name at the top of the list. Clearly Bronco is not at his best in big games, but he does out-preform his peers for the most part. Does that mean BYU should start looking for a new head coach? Your views on that probably depend on what kind of coach you think BYU can get to replace him.