What Is Standard Deviation?

A mean on its own can hide almost anything. Standard deviation is the number that tells you how much it is hiding.

The problem it solves

Two teams each average 50 points a game.

• Team A: 48, 50, 49, 51, 52
• Team B: 20, 80, 30, 75, 45

Identical means, completely different teams. One is reliable, the other is a coin toss. The mean cannot tell them apart; standard deviation can.

Team A has a standard deviation of about 1.4. Team B has about 24. If you had to bet on one scoring close to 50 next week, that pair of numbers is the whole answer.

What it measures

The typical distance between a value and the mean. Roughly: if you picked a value at random, how far from average would you expect it to be?

Small standard deviation means values huddle close to the mean. Large means they spread out.

It is expressed in the same units as the data. Heights in centimetres give a standard deviation in centimetres. Salaries in pounds give one in pounds. This is why it can be read directly against the data, and why it cannot be compared between datasets measuring different things.

How it is calculated

Four steps:

1. Find the mean.
2. For each value, subtract the mean and square the result.
3. Average those squared differences. That is the variance.
4. Take the square root. That is the standard deviation.

For 2, 4, 4, 4, 5, 5, 7, 9:

• Mean: 5
• Differences: −3, −1, −1, −1, 0, 0, 2, 4
• Squared: 9, 1, 1, 1, 0, 0, 4, 16, totalling 32
• Variance: 32 ÷ 8 = 4
• Standard deviation: 2

So values sit about 2 away from the mean on average.

Why square and then unsquare

Averaging the raw differences gives zero every single time, because the amounts above and below the mean cancel exactly. That is not a coincidence or a quirk of this example, it is the definition of the mean.

Squaring removes the signs so they stop cancelling. It also weights large deviations more heavily than small ones, which is usually what you want: one value a long way out is more informative than several slightly out.

Taking the square root at the end returns the answer to the original units, so it can be read against the data directly rather than as an abstract quantity.

The 68, 95, 99.7 rule

For data roughly shaped like a bell curve, which covers a great many natural measurements:

• About 68 percent of values lie within one standard deviation of the mean
• About 95 percent within two
• About 99.7 percent within three

If adult male height has a mean of 175 cm and a standard deviation of 7 cm, then roughly two thirds of men are between 168 and 182 cm, and about 95 percent between 161 and 189 cm.

This is what makes standard deviation genuinely useful rather than merely descriptive. It converts a single number into a statement about how common or unusual any particular value is. A man of 196 cm is three standard deviations out, which places him in roughly the top 0.15 percent.

Sample versus population

One technical detail that changes the arithmetic slightly.

When you have every value in the group you care about, divide by the count, as above. When you have a sample and want to estimate the wider population’s spread, divide by one less than the count instead.

Dividing by a smaller number produces a slightly larger answer, which corrects for the fact that a sample tends to understate the spread of the population it came from. Spreadsheet functions distinguish these, usually as STDEV.P and STDEV.S, and picking the wrong one is a common and usually minor error.

Reading it in practice

Test scores. A mean of 60 with a standard deviation of 5 is a tightly bunched cohort where 65 is a strong result. The same mean with a standard deviation of 20 is a wide spread where 65 is unremarkable.

Investment returns. Standard deviation is the standard measure of volatility. Two funds averaging 7 percent a year, one with a standard deviation of 4 and one with 18, are entirely different propositions despite the identical average.

Manufacturing. Quality control is largely the business of keeping standard deviation small. The mean can be perfect while a wide spread produces unusable parts at both ends of the range.

Response times. A service averaging 200 milliseconds with a large standard deviation is delivering a much worse experience than the average suggests, because the slow requests are the ones people notice.

Two cautions

It assumes the mean is meaningful. For heavily skewed data, where the median is the better centre, standard deviation is similarly distorted. Quartiles and the interquartile range describe the spread better in those cases.

It is sensitive to outliers, for the same squaring reason that makes it useful. One extreme value can inflate it substantially, so a large standard deviation is always worth investigating: it may reflect genuine spread, or a single unusual point, or a data entry error.

Comparing across datasets

Because it carries units, standard deviation cannot be compared between different kinds of data directly. A standard deviation of 3 means nothing without knowing whether the values are in millimetres or millions.

Dividing it by the mean gives the coefficient of variation, a unitless ratio that can be compared across anything. A coefficient of 0.05 is tight regardless of what is being measured; 0.8 is very scattered. It is the right tool when the question is which of two very different processes is more consistent.