Choosing a Statistical Test

Every "which statistical test should be used?" question on an AQA paper is answered the same way: by working through three questions in order. You never have to guess, and you never memorise a list. You interrogate the study.

1. Difference or correlation? Does the study compare conditions to look for a difference, or does it measure two variables to look for a correlation / association?
2. Related or unrelated? If it is a difference, is the experimental design related (repeated measures or matched pairs) or unrelated (independent groups)?
3. Level of measurement? Are the data nominal, ordinal or interval?

Answer all three and exactly one test is left standing. The spec (4.2.3.3) requires you to know when each named test is used, not to calculate them — the only calculation you must be able to perform is the sign test, which is covered separately.

The two factors AQA names explicitly are level of measurement and experimental design. Add the difference-versus-correlation split and you have the full decision.

Miss any one of the three questions and you can land on the wrong test even when the other two answers are right, so treat all three as non-negotiable.

The first split decides which half of the map you are in.

A test of difference is used when a study takes an independent variable with two conditions and compares a dependent variable between them — for example, recall scores in a quiet room versus a noisy room. You are asking "do these two sets of scores differ?"

A test of correlation / association is used when a study measures two variables for each participant and asks whether they are related — for example, hours of sleep and reaction time. Nothing is manipulated into conditions; you are asking "as one variable changes, does the other change with it?"

Feature	Test of difference	Test of correlation / association
What the study does	Compares two conditions or groups	Measures two variables per participant
Typical wording	"compared", "difference between"	"relationship", "association", "as X rises"
Example	Recall in noise vs. silence	Sleep hours vs. reaction time

A giveaway for correlation is that each participant provides two scores that are not conditions of an IV — a co-variable pair. If the study manipulates conditions and compares them, it is a difference.

The word "association" belongs with nominal data specifically: Chi-Squared can test an association between two categorical variables, which is why it sits in the correlation/association column as well as the unrelated-difference column.

This question only applies to tests of difference. Correlations do not have a related/unrelated split.

A related design links the two sets of scores. There are two forms:

• Repeated measures — the same participants take part in both conditions, so each person gives a pair of scores.
• Matched pairs — different participants, but each one is paired with a partner matched on key variables (age, IQ, and so on). The pairing links the scores.

An unrelated design is independent groups: different, unpaired participants take part in each condition, so the two sets of scores come from separate people with no link between them.

Design	Who takes part	Counts as
Repeated measures	Same people in both conditions	Related
Matched pairs	Paired people, one per condition	Related
Independent groups	Different, unpaired people	Unrelated

The trap is matched pairs. Because it uses different people, students often call it unrelated. It is related — the deliberate pairing links each score to a partner.

Get this wrong and, for ordinal data, you will reach for Mann-Whitney when Wilcoxon is correct (or the reverse), so pin the design down before naming the test.

This grid is the heart of the topic. Read down the left for the level of measurement, across the top for difference (related or unrelated) versus correlation/association, and the cell is your test.

Level of measurement	Test of difference — RELATED (repeated measures / matched pairs)	Test of difference — UNRELATED (independent groups)	Test of correlation / association
Nominal	Sign test	Chi-Squared	Chi-Squared (association)
Ordinal	Wilcoxon	Mann-Whitney	Spearman's rho
Interval	Related t-test	Unrelated t-test	Pearson's r

Reading it by row:

• Nominal data are frequency counts in named categories (yes/no, pass/fail). A related difference uses the sign test; an unrelated difference or an association uses Chi-Squared.
• Ordinal data are ranked positions where the intervals are not equal (ratings, ranks). Related difference → Wilcoxon; unrelated difference → Mann-Whitney; correlation → Spearman's rho.
• Interval data have equal, measurable intervals (time in seconds, temperature). Related difference → related t-test; unrelated difference → unrelated t-test; correlation → Pearson's r.

A common mnemonic for the six named parametric-and-rank tests is "Carrots Should Come Mashed With Swede Under Roast Potatoes" — Chi-Squared, Spearman's, Coefficient (Pearson's) across the top three, then Mann-Whitney, Wilcoxon, and the t-tests filling the grid.

Note the special status of Chi-Squared: it is the only test that appears twice, because with nominal data it can test both an unrelated difference between groups and an association between two variables. It requires the data to be independent — each participant must fall into one cell only, never counted twice.

Because level of measurement decides the row, mislabelling the data changes the test. Learn the three levels precisely.

Level	What it is	Example
Nominal	Data in named categories; you count frequencies	Number of people who chose tea vs. coffee
Ordinal	Data ranked in order; intervals are not equal	Ratings of confidence from 1 to 10
Interval	Equal, fixed intervals on a public scale	Reaction time in milliseconds

The dividing lines that matter for the exam:

• Nominal is a headcount. If the "data" are how many people fall into each category, it is nominal, and you are looking at Chi-Squared or the sign test.
• Ordinal has order but uneven gaps. Rating scales, Likert responses and rankings are ordinal. The gap between a 6 and a 7 is not guaranteed to equal the gap between a 2 and a 3.
• Interval uses a standardised unit. Seconds, centimetres, degrees and counts of correct items on a standardised test are interval.

When data could be read as ordinal or interval, choosing the rank-based test (Wilcoxon, Mann-Whitney, Spearman's rho) is the safer, more conservative choice. Rating-scale data are treated as ordinal, not interval.

Fix the level first, and half of the grid is already ruled out before you even think about the design.

Choosing the test is one skill; reading the result is another the spec assesses. Once a test produces a calculated value, you compare it against a critical value from a statistical table (using the number of participants, whether the hypothesis is one- or two-tailed, and the significance level, usually $p\le 0.05$).

The direction of the comparison is not the same for every test, and this catches people out.

Rule	Tests it applies to
Calculated value must be equal to or greater than the critical value	Chi-Squared, Spearman's rho, Pearson's r, related t-test, unrelated t-test
Calculated value must be equal to or less than the critical value	Sign test, Wilcoxon (T), Mann-Whitney (U)

The memory hook is the letter: the "less than" tests are the ones without an R in their statistic — the sign test, Wilcoxon's T and Mann-Whitney's U. The "greater than" tests carry an R sound — Chi-Squared, Spearman's, Pearson's, and the t-tests compared via their R-tables.

For the sign test, Wilcoxon and Mann-Whitney the calculated value must be equal to or less than the critical value for the result to be significant. Getting this direction backwards flips a significant result into a non-significant one and loses the interpretation marks.

If the result meets its rule, you reject the null hypothesis and accept the alternative; if not, you retain the null. Always state the significance level and whether the hypothesis was one- or two-tailed when you report the outcome.

Apply the three questions to three studies.

Scenario 1. A researcher measures the same participants' stress ratings (on a 1–10 scale) before and after a mindfulness course, and tests whether stress changed.

1. Difference or correlation? Difference — two conditions compared.
2. Related or unrelated? Related — repeated measures, same people twice.
3. Level of measurement? Ordinal — a 1–10 rating scale.

→ Related difference, ordinal data = Wilcoxon.

Scenario 2. A study records whether each participant passed or failed a driving test, comparing a group who took lessons with an app against a group who used a traditional instructor.

1. Difference or correlation? Difference — two groups compared.
2. Related or unrelated? Unrelated — different, unpaired people in each group.
3. Level of measurement? Nominal — pass/fail category counts.

→ Unrelated difference, nominal data = Chi-Squared.

Scenario 3. A researcher measures each participant's number of hours of revision and their exam mark (percentage), testing whether the two are related.

1. Difference or correlation? Correlation — two variables measured per person, nothing manipulated.
2. Related or unrelated? Not applicable — correlations have no design split.
3. Level of measurement? Interval — hours and percentages are equal-interval scales.

→ Correlation, interval data = Pearson's r.

Notice how each scenario is decided purely by answering the three questions in order. Once you trust the process, no study can trip you up.

1. Choosing a test without checking all three factors

The most common failure is naming a test after answering only one or two of the questions. You need the difference/correlation split, the design (for differences), and the level of measurement. A right answer on two of three still lands you in the wrong cell of the grid.

2. Confusing related and unrelated designs

Repeated measures and matched pairs are both related; independent groups is unrelated. Matched pairs is the classic trap — it uses different people but the pairing links the scores, so it is related. Mislabelling the design swaps Wilcoxon for Mann-Whitney (or the related t-test for the unrelated t-test).

3. Using a t-test on ordinal data

The t-tests are for interval data only. Rating scales and rankings are ordinal, so a difference on ordinal data uses Wilcoxon (related) or Mann-Whitney (unrelated), and a correlation uses Spearman's rho — never Pearson's r or a t-test.

4. Forgetting Chi-Squared needs nominal, independent data

Chi-Squared works on nominal frequency counts, and the data must be independent — each participant counted in one cell only. If a study reuses the same participants across categories (so the counts are not independent), Chi-Squared is not appropriate.

5. Getting the "greater than vs. less than" significance rule backwards

For the sign test, Wilcoxon (T) and Mann-Whitney (U) the calculated value must be equal to or less than the critical value to be significant. For Chi-Squared, Spearman's rho, Pearson's r and the t-tests it must be equal to or greater than. Reversing the direction turns a significant finding into a non-significant one.

6. Treating "association" and "difference" as the same thing for Chi-Squared

Chi-Squared can test either a difference between unrelated groups or an association between two variables, and the hypothesis wording tells you which. Do not assume it is always a "difference" test — read whether the study is comparing groups or looking for a relationship between two categorical variables.