Confidence Level -- Are you Suuuure?

Published on 15 March 2026 at 17:30

When scientists run an experiment, they want to know whether their results are real or if they might have happened just by chance.

The significance level is the rule scientists set to help decide that.

Think of it as a cut-off point for how much randomness we're willing to accept.

 

The most common threshold: p < 0.05

In many scientific studies, researchers use 0.05 (or 5%) as the significance level.

 

This means:

If the probability that the result happened by chance is less than 5%, scientists consider the result statistically significant.

In simple terms:

p < 0.05 → unlikely to be random → result is considered meaningful

p > 0.05 → could easily be random → result is not considered significant

 

Imagine a study testing whether using comics helps students learn science better than textbooks alone.

If the results give a p-value of 0.03, that means:

  • There is a 3% probability the difference between the groups happened just by chance.

Because 3% is less than 5%, the result is considered statistically significant.

So researchers would say:

The comic-based learning method had a statistically significant effect on student learning.

 

Important thing to remember

Statistical significance does NOT mean something is large, important, or useful in the real world.

It only means the result is unlikely to be caused by random variation in the data.

 

That’s why scientists also look at effect size, sample size, and study design before drawing strong conclusions.

 

Add comment

Comments

There are no comments yet.