CFL.ca Staff

TORONTO -- The CFL Power Rankings have been one of the most talked about and controversial features on CFL.ca since their introduction at the start of the 2010 season.

Many fans have expressed outrage at the rankings, while others have vowed never to pay attention to them again! Still, there have been other fans that have asked questions about the CFL Power Rankings – how we came up with the formula and how to interpret its results.

We hope the following is able to address many of your questions you may have had about the CFL Power Rankings.

Q: The Power Rankings were developed using something called “Regression Analysis”.  What does this mean?

A: A formal definition of “regression analysis” would be, “a mathematical equation to describe the statistical relationship between one or more predictors and a response variable and to predict new observations.”

This sounds complicated, but like most things mathematical, it really isn’t as bad as it sounds. All this definition is saying is that there is a measurable relationship between certain things. 

For example, you may think that there is a relationship between a family’s weekly grocery bill and the size of a family. In fact, if you had the time, you might conduct some research to determine if this was the case. Let’s assume that you look at 15 different families and plot their weekly grocery bill against family size:



As you might suspect, the weekly grocery bill tends to increase as family size increases. This “upward trend” could be represented by a straight line, as follows:



This straight line is represented by a formula (in this case, our weekly grocery bill is estimated to be equal to $53.30 + $40.40 x Family Size). This formula is called the regression equation.

The key to understand is that this formula is only an estimate. In fact, the reason that none of those blue dots happen to be on the red line is because there are other things, besides family size, that impact a family’s grocery bill - things like, age of family, family income and family/store location.

Q: Why did you choose to use Quarterback Rating, Rushing Yards, Field Goals Missed and Sacks Taken in your formula, but not statistics like Fumbles Lost and Penalties?

A: The great thing about a regression analysis is that it can estimate two important things for us. Let us explain starting with our grocery example above.

As you can see, family size tells us an awful lot about a family’s weekly grocery bill, even if it doesn’t tell us everything. A regression analysis can tell us how much a variable impacts or “predicts” a response. In our grocery example above, family size predicts 69.5% of a family’s weekly grocery bill.  That is, there is 31.5% of a family’s weekly grocery bill that is explained by factors other than family size (at least in our example above). 

So, a regression analysis tells us how predictive a certain model is.

A regression analysis can also tell us which specific variables we choose to add actually improve the predictability of our model. For example, if we chose to, we could have included other variables like age of family, family income and store location as variables in the above equation. The regression analysis would then tell us which of these variables actually improved our model and which had no impact whatsoever.

This is what we did with our CFL Power Rankings. We included every statistic measured by the CFL to see which were the most predictive. It turns out that the most predictive were Quarterback Efficiency Rating, Rushing Yards, Field Goals Missed and Sacks Taken.

Q: You said that your model has 75% predictability. I’ve done some calculations and determined that your power rankings are only working at 50%. Is your model flawed?

A: Without seeing your calculations, it’s difficult to answer your question completely. However, we may have unintentionally misled some of our fans when we said our model has 75% predictability. When we use the term “predictability” in statistics, we really mean how much of a result the model is able to explain. For example, in our grocery scenario above, our model has a 69.5% predictive rate.

Q: You said that you only used 2009 results to develop your model. Shouldn’t you have gone back further, like 10 or 15 years?

A: That’s a valid criticism. I think if we were to do it again, we would go further back, which would almost certainly change the equation. That said, 2009 did provide us with 144 observations (72 games x 2 teams) from which to draw our findings. Of course, more observations are almost always better in statistics. For example, in our grocery example above, we only had 15 observations. If we had secured 100 or 1,000 or even 10,000 observations we would likely get a more robust equation.

Q: Why don’t the power rankings consider wins and losses?

A: The approach that we have taken presumes that wins and losses are a function of the variables that we have included.

In other words, a team that has a strong quarterback, that is able to rush the ball, that doesn’t miss field goals and protects its quarterback is more likely to score more points and, as a result, win the game.

Q: Why doesn’t your model consider defence?

A: It does. We actually perform the regression equation twice for each “score”. The first calculation determines a “gross” result for a team. From this result we will subtract the score we determined for the opposing team, to arrive at a net score. So, the effectiveness of a team’s defence is considered when we measure the QB Rating of the opposing team and subtract it to arrive at our result.