Grading the Test

Ever since the new SAT writing test, featuring a proctored off-the-cuff writing sample, was launched in 2005, it has found few fans. Students weren't crazy about having to write a high-stakes essay under time pressure on a randomly assigned topic, or the fact that the new test extended a three-hour college-entrance exam by another 45 minutes. Meanwhile, college admission offices were reluctant to put much weight in a new test of unknown value that hadn't been formally validated.

This week, the nonprofit College Board, which publishes the SAT, gave its critics one less reason to beat up on the writing test. Based on the results of their 2008 national validity study, which analyzed the grades of approximately 150,000 students who completed their first year of college at 110 four-year institutions during the spring of 2007, the SAT writing test proved to be just slightly more predictive of college freshman grades than the math and critical reading components of the college entrance test. (For those who appreciate adjusted correlations, the new writing test earned a correlation coefficient of 0.51, compared to 0.47 for the math test and 0.48 for the critical reading test. A correlation of 1.0 is the highest possible value, and anything over 0.5 is considered a large correlation.)

The new study also validated the recently overhauled SAT, of which the writing test was but one component. The new SAT, introduced along with the writing test in March 2005, put increased emphasis on higher level math, critical reading skills and grammar. Overall, the changes made the SAT less of an aptitude test and more of an achievement test. The SAT validity study concluded that the new three-part test was "not substantially" different, in terms of its reliability, than the old two-part, three-hour SAT which gave only math and verbal scores.

While College Board officials were quick to describe the results as proving that the SAT is an "excellent predictor of how students will perform in their first year of college," they also had to concede that the validation study revealed that a student's high-school grade-point average is a bit more predictive of college freshman-year grades than the SAT. However, the study results also confirmed that the SAT is better at forecasting the success of most minority students during their freshman year. Consideration of both sets of data by college admission offices will predict freshman grades significantly better than either GPA or SAT scores viewed alone, said College Board President Gaston Caperton. The College Board, which publishes the SAT, used its inhouse research staff to conduct the study.

The decision to overhaul the SAT was made in 2001 after the president of the University of California system threatened to stop requiring applicants to submit standardized test results, out of concern that they were having a negative effect on high-school curricula. Specifically, he worried that too many high school students were spending too much time focused on obscure analogies, which made up a significant part of the old SAT verbal test, rather than spending their time writing or critically discussing literature.

Fearful of losing such a huge market or inspiring a copycat trend, the College Board worked with UC officials to overhaul the SAT. The new test introduced a new writing section (multiple-choice questions on grammar and usage, as well as the new student-produced essay, scored by at least two different graders). The verbal section was renamed the "critical reading" test. The analogy questions were eliminated and emphasis switched to comprehension of long and short readings from a broad range of disciplines. The new math section included more advanced math problems. A perfect score was boosted from 1600 to 2400, with a total of 800 possible from each section.