Statistical significance
Also called p-value, significant difference
A statement about how often chance alone would produce a difference at least as large as the one you found, if there were really no difference.
A p-value is that frequency. p = 0.007 means a gap this size would turn up in under one comparison in a hundred when nothing is going on. By convention p below 0.05 is called significant, a threshold with no special mathematical standing: it is a habit, and a useful one only when it is fixed before you look.
Significance is not the size of the effect, not the probability that you are right, and not evidence that something matters. A trivial difference becomes significant with enough responses, and an important one stays non-significant on a small sample. Read the percentage-point difference first, and decide in advance what size would change your decision.
“Not significant” does not mean the groups are the same. It means you cannot distinguish them yet, which is a different and much weaker claim than equality.
Worked example
120 of 400 against 156 of 400 is 30% against 39%. The chi-square is about 7.2 with one degree of freedom, and p is about 0.007.
Where it goes wrong
Running many tests and reporting only those below 0.05 increases the risk of false positives. Report how many tests you ran and how you handled multiple comparisons.