2Research methods

Choosing the right test

Which test to use depends on the level of measurement, the number of groups, whether they are related, and whether parametric assumptions hold.

Practice

12 questions

Level

Core

Reading time

3 min

Test yourself on this topic

Comparing groups

For differences between groups or conditions on a continuous outcome. For two independent groups, the t-test in Welch’s version does not assume equal variances (Delacre, Lakens and Leys, 2017).

The rank-based tests suit ordinal data and small, skewed samples, and they ask a different question. Mann-Whitney U asks whether scores in one group tend to be higher than in the other; it compares medians only when the two distributions have the same shape and spread (Fagerland, 2012).

Table 1. Tests for a continuous outcome, by design.

Design	Parametric	Rank-based
Two independent groups	Independent t-test (Welch’s)	Mann-Whitney U
Two related conditions	Paired t-test	Wilcoxon signed-rank
Three or more independent groups	One-way ANOVA	Kruskal-Wallis
Three or more related conditions	Repeated-measures ANOVA	Friedman

Two independent groups: Parametric: Independent t-test (Welch’s); Rank-based: Mann-Whitney U

Two related conditions: Parametric: Paired t-test; Rank-based: Wilcoxon signed-rank

Three or more independent groups: Parametric: One-way ANOVA; Rank-based: Kruskal-Wallis

Three or more related conditions: Parametric: Repeated-measures ANOVA; Rank-based: Friedman

Sources: Field (2024), checked 24 Sept 2026; Delacre, Lakens and Leys (2017), checked 13 Oct 2026.

Worked example:One service, three questions

A service runs anxiety groups and individual therapy. It records each client’s anxiety score at referral and at discharge, and whether they were discharged recovered.
Do group and individual clients differ at discharge? Two independent groups and a continuous score: Welch’s t-test.
Do scores fall from referral to discharge? The same people measured twice, so the conditions are related: a paired t-test.
Is recovery more common in one format? Recovered or not is a category, so the counts make a 2 × 2 table for the chi-square test of independence.

Sources: Field (2024), checked 24 Sept 2026; Delacre, Lakens and Leys (2017), checked 13 Oct 2026.

Categorical outcomes

When the outcome is a category, such as recovered or not, the data are counts. To compare groups on it, or to test whether two categorical variables are related, use the chi-square test of independence.

Examining relationships

For association rather than difference:

Two continuous variables, linear: Pearson’s r.
Ordinal data, or a monotonic but non-linear relationship: Spearman’s rho.
Predicting a continuous outcome from one or more predictors: linear regression.
Predicting a binary outcome: logistic regression.

Parametric assumptions

Parametric tests make four assumptions. Each one, and how to check it:

Interval or ratio data. Check the level of measurement when choosing the outcome: a count or a time qualifies, a category does not (Describing data sets out the four levels).
Approximately normal sampling distributions. Look at a histogram or Q-Q plot of the residuals; in a large sample the central limit theorem makes this less of a worry.
Homogeneity of variance, for comparisons between groups. Compare the groups’ standard deviations. Welch’s t-test does not need it, which is why Delacre, Lakens and Leys (2017) recommend it by default.
Independent observations, which every test here assumes. Check the design: repeated measures, or clients grouped within therapists or services, break it and need a test or model built for that structure.

Common mistakes

Running multiple t-tests instead of an ANOVA, inflating the Type I error rate.
Using an independent t-test on repeated measures, which discards the pairing and loses power.
Reaching for a non-parametric test at the first sign of skew, when the sample is large enough for the CLT to apply.

Cookies on DClinPsy Prep