A Bonferroni correction calculator adjusts the significance level when you run several tests at once. Enter the number of tests and your overall level, such as 0.05, and you get the stricter level that each test must meet. Add your p-values, and the calculator multiplies each by the number of tests and tells you which stay significant. It also shows the Šidák level and the chance of a false alarm without any correction. It runs in your browser.
How to use the Bonferroni correction calculator
- Enter the number of tests.
- Enter the overall significance level, or leave it at 0.05.
- Add your p-values if you want them adjusted.
- Press Calculate.
Why a correction is needed
Every test carries a small chance of a false positive. If you run many tests, the chance of at least one false positive grows. With independent tests at 0.05, the chance of at least one false positive is 1 − 0.95^m, which is our own working. With 20 tests, it is about 64 percent, so a significant result among them is no surprise. A correction keeps the overall chance of a false positive at your chosen level.
The correction
The Wikipedia article on the Bonferroni correction describes the method as testing each individual hypothesis at a significance level of α ÷ m, where α is the desired overall level and m is the number of hypotheses. Its example is a trial that tests 20 hypotheses with an overall α of 0.05, for which the corrected level is 0.05 ÷ 20 = 0.0025.[1] The calculator gives this number. Each test must then have a p-value of 0.0025 or less to count as significant.
Adjusted p-values
The page also gives the other form: each p-value is multiplied by the number of tests, with any value above 1 reduced to 1, and the significance level is left as it is. The two forms lead to the same decisions.[1] For three tests with p-values of 0.001, 0.01 and 0.04, the adjusted values are 0.003, 0.03 and 0.12, so at 0.05 the first two stay significant and the third does not. The calculator lists them with the decision.
What the method controls
The page states that the Bonferroni correction rejects the null hypothesis for each p-value that is at most α ÷ m, which controls the family-wise error rate, the chance of at least one false positive, at no more than α. The proof uses Boole's inequality. It also says that the guarantee does not need any assumptions about the dependence of the p-values or about how many of the null hypotheses are true.[1] This is why the method is so widely used.
The Šidák level
The page mentions that the Holm–Bonferroni method and the Šidák correction are always at least as powerful as the Bonferroni correction, and that they do not control the expected number of type I errors per family.[1] It does not give the formula for the Šidák level, so we use the standard one, 1 − (1 − α)^(1 ÷ m), which assumes independent tests. For 20 tests at 0.05, it is 0.002561, a little looser than the Bonferroni level of 0.0025. The two are very close for small α.
The cost: lower power
The page notes that the Bonferroni correction can be conservative when there are many tests, or when the test statistics are positively correlated. It also says that multiple-testing corrections increase the chance of type II errors when the null hypotheses are false, which means that they reduce the statistical power.[1] A real effect may fail to pass the stricter level. If you test hundreds of hypotheses, as in genetics, other methods that control the false discovery rate are often preferred.
When to use it
Use the correction when you run a small number of tests, such as comparing several groups or checking several outcomes, and any false positive would be costly. Decide the number of tests before you look at the results, and count every test that you ran, including those that you did not report. If the tests are planned in advance, a higher level may be given to the most important one, which the page also describes.[1]
A worked example
A researcher compares a new treatment with a control on five outcomes and finds p-values of 0.012, 0.030, 0.200, 0.004 and 0.045. At an overall level of 0.05, the Bonferroni level is 0.05 ÷ 5 = 0.01, so only the outcome with p = 0.004 is significant. Equivalently, the adjusted p-values are 0.06, 0.15, 1, 0.02 and 0.225, and only 0.02 is below 0.05. Without the correction, four of the five would have looked significant, and the chance of at least one false positive among five independent tests at 0.05 would be about 22.6 percent. Enter these values to see the same results.
Alternatives
Methods such as Holm's step-down procedure test the smallest p-value at α ÷ m, the next at α ÷ (m − 1), and so on, and they are at least as powerful as Bonferroni while giving the same guarantee. When there are very many tests, controlling the false discovery rate, the expected share of false positives among the significant results, may be more useful. This calculator covers the original, simple method.
Common mistakes
- Counting only the significant tests. Count all the tests that you ran.
- Dividing the p-values. Divide the level, or multiply the p-values.
- Using it for hundreds of tests. It becomes too strict.
- Forgetting the cost in power. Real effects may be missed.
What next
Use this often? Add Bonferroni C to your Home screen.
Frequently asked questions
What is a Bonferroni correction calculator?
A tool that lowers the significance level of each test, or raises each p-value, when you run several tests at once.
What is the Bonferroni correction formula?
Test each hypothesis at α ÷ m, where m is the number of tests. For 20 tests at 0.05, the level is 0.0025.
How do I adjust a p-value?
Multiply it by the number of tests, and cap the result at 1. Compare it with the original α.
What does the correction control?
The family-wise error rate, the chance of at least one false positive, at no more than α. It needs no assumption about dependence.
What is the drawback?
It is conservative and lowers statistical power, especially with many tests or correlated statistics.
Sources
Was this tool helpful?
What went wrong?
We only receive the reason you pick, never what you typed into the tool.
Thanks for your feedback.