Comparing two groups has a t-test, but comparing three or more groups at once needs a different tool, and running several separate t-tests instead is a real statistical trap, not just extra work. This works through one-way ANOVA’s F-statistic, degrees of freedom, and what a significant result actually tells you, with a full worked example.
In this guide
Why not just run several t-tests?
One-way ANOVA (analysis of variance) compares the means of three or more groups at the same time, in a single test. Running a separate t-test on every pair of groups instead inflates the overall chance of a false positive: each individual test carries its own 5 percent Type I error risk, and running many of them on the same data stacks those risks up across the whole analysis. Comparing test scores across four teaching methods would need six separate pairwise t-tests; ANOVA tests all four at once.
The F-statistic
The F-statistic is the ratio of variance between the group means to variance within the groups themselves. A large F-statistic means the groups’ means are spread out much more than the natural variability inside each group would suggest, pointing toward a real difference between groups.
Worked example
Three groups: (85,90,88,92,87), mean 88.4; (78,82,80,85,79), mean 80.8; (92,95,91,97,94), mean 93.8. Comparing how much these three means vary against each other, relative to the spread within each group, gives F = 30.91 and a p-value of 0.000018, far below the common 0.05 threshold, strong evidence of a real difference somewhere among the three groups.
Contrast that with three groups that genuinely do not differ, (50,52,49,51,50), (51,49,52,50,51) and (49,51,50,52,50): the F-statistic comes out to only 0.05, nowhere near large enough to suggest a real difference.
Degrees of freedom and the critical F value
| Degrees of freedom | Formula | Why |
|---|---|---|
| Between groups | k minus 1 | The grand mean across all groups is already fixed |
| Within groups | N minus k | One df lost per group for estimating that group's own mean |
For 3 groups with 5 observations each (15 total), that gives 2 degrees of freedom between groups and 12 within groups. At a significance level of 0.05, the critical F value there is 3.89, the worked example’s F of 30.91 clears that easily.
The null and alternative hypothesis
The null hypothesis is that all of the group population means are equal. The alternative hypothesis is deliberately broad: at least one group’s population mean differs noticeably from all the others, without specifying which particular group or groups. The decision rule follows the same pattern as any other hypothesis test: reject the null if the F-statistic exceeds the critical value, or if the p-value is below alpha.
What a significant result does not tell you
A significant one-way ANOVA only proves that at least one group’s mean differs from the others, it does not identify which particular group or groups. Finding out which pairs actually differ needs a post-hoc test, such as Tukey’s HSD, run afterward, which makes all the pairwise comparisons while still controlling the overall false-positive rate that running many raw t-tests would have inflated.
A marketing team comparing customer satisfaction across five store locations, finding a significant ANOVA result, should run a post-hoc test next, not stop the analysis there and not assume every store differs from every other.
ANOVA’s assumptions
ANOVA assumes the groups being compared have roughly equal variances: the within-group variance estimate is pooled across all groups, which is only reliable if their true variances are similar. It does not require data within each group to be exactly normally distributed, but it works best when each group is roughly normal, or the sample sizes are large enough, and tends to be fairly robust to mild departures.
ANOVA with exactly two groups
One-way ANOVA is a direct generalisation of the independent two-sample t-test. With exactly two groups, the two approaches agree completely, down to the F-statistic being the t-statistic squared.
Common mistakes
- Running separate t-tests on every pair instead of one ANOVA. That inflates the overall Type I error rate across the analysis.
- Treating a significant ANOVA as identifying which groups differ. It only shows a difference exists somewhere; a post-hoc test is needed to pin down where.
- Concluding a non-significant ANOVA proves the groups are equal. It only means there was not enough evidence of a difference, not proof there is none.
- Ignoring the equal-variances assumption. Pooling within-group variance only makes sense if no single group is secretly far more variable than the rest.
- Running a post-hoc test without a significant overall ANOVA first. That defeats the purpose of controlling the combined error rate in the first place.
Cheat sheet
| Question | Answer |
|---|---|
| What is one-way ANOVA used for? | Comparing 3+ group means at once |
| Why not just run multiple t-tests? | It inflates the overall Type I error rate |
| What does the F-statistic represent? | Between-group variance / within-group variance |
| Degrees of freedom between groups (k groups)? | k minus 1 |
| Degrees of freedom within groups (N total, k groups)? | N minus k |
| What is ANOVA's null hypothesis? | All group means are equal |
| What does a significant ANOVA tell you? | At least one group differs, not which one |
| What finds which specific groups differ? | A post-hoc test, e.g. Tukey's HSD |
| What does ANOVA assume about the groups? | Roughly equal variances, approximate normality |
| How does ANOVA relate to a 2-group t-test? | Identical result; F equals t squared |
Try it yourself
Quick check
Three groups: (20,22,19,21,23), (25,27,24,26,28), (30,32,29,31,33). Without running the exact calculation, would you expect the F-statistic to be small or large, and why?
Show answer
Large. The three groups’ means (21, 26 and 31) are spread five points apart from each other, while the values inside each group only vary by about plus or minus 2, so the between-group spread is much larger than the within-group spread. Running the real calculation confirms it: F = 50.0, strong evidence of a real difference.
Frequently asked questions
What is one-way ANOVA used for?
One-way ANOVA compares the means of three or more groups at the same time, in a single test, extending the idea behind a two-sample t-test.
Why not just run a separate t-test on every pair of groups?
Each t-test carries its own Type I error risk, so running many of them on the same data inflates the overall chance of a false positive across the whole analysis. ANOVA avoids this by testing all groups together once.
What does the F-statistic in ANOVA measure?
The F-statistic is the ratio of variance between the group means to variance within the groups. A large F-statistic suggests the groups’ means differ more than ordinary within-group noise would explain.
If an ANOVA result is significant, do you know which groups differ?
No. A significant result only shows that at least one group’s mean differs from the others. A post-hoc test, such as Tukey’s HSD, is needed to identify specifically which groups differ.
What assumptions does one-way ANOVA make?
ANOVA assumes the groups have roughly equal variances, since their within-group variability is pooled into one estimate, and it works best with approximately normal data or large enough sample sizes.
How does one-way ANOVA relate to a two-sample t-test?
With exactly two groups, ANOVA and an independent two-sample t-test give identical results, and the F-statistic equals the t-statistic squared.
Test yourself
Timed questions on ANOVA Basics, with an explanation for every answer.