What is Statistical significance? Definition and examples

Statistical significance is the confidence that a difference between test variants is real rather than random noise.

Example

A result at 95% confidence means there's roughly a 5% chance you'd see a difference this large if the variants actually performed the same.

Why does Statistical significance matter for e-commerce?

Stopping a test the moment one variant pulls ahead is the most common testing mistake; wait for the planned sample size.

How does Posta use Statistical significance?

Posta's agents use Statistical significance when reviewing your accounts and recommending changes. See the creative testing framework page for how it fits into your workflow.