7
Writing scores are lower than other scores and seem less
consistent with other scores
Writing scores are generally lower than other scores—on average performance at the same
percentile is associated with a writing score which is 3–4 points lower than the ACT Composite
or English scores. However, reading scores on average have consistently been the highest score
across groups and such variations are common across different aggregate scores on many
standardized tests. It is not just the lower scores but the larger than expected gap between writing
scores which has led to questions from the field. So, let’s discuss this gap in more detail.
Each test score includes some level of imprecision—and every observed test score is comprised of
both a true score, reflecting an individual’s actual skill or knowledge and the expected variation from
that score.
9
The standard error of measurement (SEM) is a metric for reporting the extent that a
typical observed score varies from the true score. The SEM is used to calculate a score range that
represents an individual’s true score with a specific level of precision. For example, if a student were
to take the same test repeatedly, with no change in the student’s knowledge or skill, some scores
would be slightly higher or slightly lower than the student’s true score. In this hypothetical example,
the small differences between the standard deviation of the student’s observed scores and the
student’s highest or lowest score are known as the SEM. The SEM for ACT test scores is about
1 point for the ACT Composite and about 2 points for English, math, reading, and science, but the
SEM for the writing test is about double that, at about 4 points on the 1–36 scale. The SEM can be
used to aid in interpreting test scores as follows:
Given a student’s observed score of X, there is a two-out-of-three chance that the student’s true
score is between the lowest and highest score in the range. For the ACT, that information would be
based on the reported score.
• A score of 20 on the ACT Composite would indicate that there is a two-out-of-three chance that
the student’s true score would be between 19 and 21.
• A score of 20 on ACT math, English, reading or science would indicate that there is a two-out-of-
three chance that the student’s true score would be between 18 and 22.
• A score of 20 on ACT writing would indicate that there is a two-out-of-three chance that the
student’s true score would be between 16 and 24.
Therefore, the writing test does have significantly greater variation than other scores because it is a
single task, evaluated by raters using a 6-point interval scale, while other ACT tests are comprised of
40 to 75 questions.
This is no different from the former writing test, but the lower reliability associated with the old
writing score was not as evident because scores (on the 2–12 scale) were not as easily compared
to scores on the 1–36 scale. There have been anecdotal reports of large differences between
the score students have received on ACT writing and their ACT English or ACT Composite score.
Differences of 10 or more points have been reported. So the question is how frequently are such
differences occurring and does such a difference indicate there is a problem with the scale or
norms? Figure 2a presents the percent of observed difference scores between students’ writing
scores and their English, reading, or the Composite scores. The difference scores were obtained by
subtracting students’ English, reading, or Composite scores from their writing scores. It shows that
9
See Harvill, L. M. Standard Error of Measurement (1991), National Council on Measurement in Education, as one of many sources for
explaining this issue in more detail.