Week 25: The McNemar Test

Ready for more nonparametric tests? Today we’re talking about the McNemar test!

When Would You Use It?
The McNemar test is a nonparametric test used to determine if two dependent samples represent two different populations.

What Type of Data?
The McNemar test requires two categorical or nominal data.

Test Assumptions

  • The sample of subjects has been randomly chosen from the population it represents.
  • Each observation in the contingency table is independent of other observations.
  • The scores of the subjects are measured as a dichotomous categorical measure with two mutually exclusive categories.
  • The sample size is not “extremely small” (though there is debate over what constitutes an extremely small sample size).

Test Process
Step 1: Formulate the null and alternative hypotheses. For the McNemar test, the data are usually displayed in a contingency table with the following setup:


Here, Response 1 and Response 2 are the two possible outcomes of the first condition. Response A and Response B are the two possible outcomes of the second condition. Cell a represents the number of people in the sample who had both Response 1 and Response A, cell b represents the number of people in the sample who had both Response 1 and Response B, etc.

The null hypothesis of the test claims that in the underlying population represented by the sample, the proportion of observations in cell b is the same as the proportion of observations in cell c. The alternative hypothesis claims otherwise (one population proportion is greater than the other, less than the other, or that the proportions are simply not equal).

Step 2: Compute the test statistic, a chi-square. It is computed as follows:


Step 3: Obtain the p-value associated with the calculated chi-square. The p-value indicates the probability of a difference in the two cell counts equal to or more extreme than the observed difference between the cell counts, under the assumption that the null hypothesis is true.

Step 4: Determine the conclusion. If the p-value is larger than the prespecified α-level, fail to reject the null hypothesis (that is, retain the claim that the cell proportions for cell b and cell c are equal). If the p-value is smaller than the prespecified α-level, reject the null hypothesis in favor of the alternative.

The example for this test comes from a previous semester’s STAT 217 grades. In the semester in question, the professor offered the students a “bonus test” after their midterms. This was done by allowing the students to essentially re-take the midterm given in class, but doing so on their own time and using all the resources they wanted to. A (small) fraction of the points they would earn on this bonus test would be added to their actual in-class test points.

I wanted to determine if the proportion of students who passed the lab test and failed the bonus test was equal to the proportion of students who failed the lab test but passed the bonus test, using n = 109 students and α = 0.05.

H0: πpass/fail = πfail/pass
Ha: πpass/fail ≠ πfail/pass

The following table shows the breakdown for the four possible outcomes in this case.




Since our p-value is smaller than our alpha-level, we reject H0 and claim that the proportions for cells b and c are significantly different.

Example in R
Since the calculations for this week’s test are quite easy, it’s probably faster to do them by hand than use R!

What sayest thou? Speak!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: