Probability and Bill James Pythagorean
Eight games is about five percent of 162. In baseballic terms, a .550 team is eight games better than a .500 team. What does it mean when a model of winning percent says one team is eight games better than another team? Bill James' Pythagorean expectation estimates team winning percent using team runs scored and runs allowed. Below we'll study what is meant when the Bill James model says one team is better than another.
The Bill James Pythagorean is an expected winning percent (EWP) calculator, which relies on the following formula using a team's runs scored and runs allowed figures:
EWP = (runs scored)2 / ((runs scored)2 + (runs allowed)2).
An interesting example of how this formula works also shows that the formula, while accurate, isn't terribly precise. The 2012 Baltimore Orioles won 93 games. That team scored 712 runs while allowing 705 runs. Intuitively, a 712-705 combo does not translate a recored of 93 and 69. The EWP works out to
7122 / (7122 + 7052)
= 506,944 / 1,003,969
That's a record of 82 wins, nowhere near 93 wins achieved with the .574 rate played.
A study of all 162-game Major League seasons shows that the relationship between winning percent and Pythagorean EWP is normally distributed. Specifically, given a team's EWP, their real winning percent is normally distributed with the EWP as the mean. The standard deviation of the distribution is four games.
The following image is a histogram illustrating the bell curve properties of a normal distribution.
Click image for larger version.
Each column represents a bucket containing team seasons. Each bucket represents an error range of one half of one percent. There is a bucket for seasons that are between 2.5 and 3.0 percent better than than their EWP. Note that the buckets with the most team seasons are near zero percent error. The more a team's record diverges from its EWP, the more rare that team is. For example, the vertical bar over .030 which ascends to 70 on the vertical axis means that 70 times in the study teams exceeded the EWP figure by an amount ranging from .026 to .030. The bar over .065, indicating errors ranging from .061 to .065, only has two team seasons. The EWP formula is very rarely off by that much.
This leads to the questions. What if the Oakland Athletics had an EWP of .620 and the Baltimore Orioles had an EWP of .588, what is the probability that Oakland has a better record1?
That works out to be a difference of .032 or 3.2 percent or just over five games. Let's call it a five game difference. This five game difference is over the course of a whole season.
Despite what appears to be a subtantial difference, the EWP only tells us that Oakland is the better team 81 percent of time give that difference in EWP. The gory details of this calculation are available upon request. They were made under the assumption that winning percent is a normally distributed random variable with standard deviation four games from EWP, using the Bill James Pythagorean model of EWP.
1 As of September 23, before any pitches were thrown, Oakland had scored and allowed runs to generate a .620 EWP, Baltimore a .588 EWP, using the Bill James formula.