GRADING STUDENT LEARNING:
BETTER LUCK NEXT TIME
By Peter Ewell

Imagine receiving a report card that contained letter grades for your child's conduct and preparation for school but nothing about performance in English or Math. Policymakers and the public will face a similar situation in this inaugural issue of Measuring Up. The National Center's decision to award all states an Incomplete in the area of student learning is wise for many reasons. But including this "subject" in the first place highlights a significant gap in our ability as a nation to say something meaningful about what or how much our students learn in college.

Why Fifty Incompletes?

Photo Credit: Dennis Brack, Black Star Peter Ewell is Senior Associate at the National Center for Higher Education Management Systems.

The decision not to award a letter grade in student learning is the right one because there are no common benchmarks that would allow meaningful state-to-state comparisons. This is not to say that individual states know nothing about student learning at higher levels within their own borders. Nor does it mean that there has been no interest in academic achievement at the national level.

Since about the mid-1980s, states have been seriously concerned about examining what students learn in college. But how the states act on this concern varies substantially. Most approach the task indirectly by asking each public college and university to administer a locally designed or locally chosen assessment and to report on what they find. Often this is done in loose partnership with regional accrediting bodies, all of which require colleges and universities to undertake some kind of student assessment.

Fewer than ten states administer a common test to large numbers of college students—and these states do so for different reasons. Some, like Florida (and to some extent, Texas and South Dakota), want to ensure that all students have the necessary knowledge and skills to progress through the system. Others, like Tennessee (and to some extent Missouri and Wisconsin), want to collect data in order to take stock of—and sometimes to reward—institutional performance. Still others examine students in only one area of study (for example, Georgia, in writing) or test students in one postsecondary sector (for example, community and technical colleges in West Virginia or state universities in California).

Such variations in scope and purpose mean that states employ very different methods when they assess college students, if they do so at all. And because states can mandate testing only in the public colleges and universities that they operate, they have no authority to assess students enrolled at private institutions. This range of variation doesn't mean that the states are doing nothing—though some are, in truth, doing nothing—but that the many different things they may be doing cannot be aggregated or compared.

We have seen periodic national interest in establishing common benchmarks for collegiate learning. The National Education Goals, which were proposed in 1990, included a provision that graduates of U.S. colleges in 2000 "will have increased markedly in their ability to think critically, communicate effectively, and solve problems." A four-year effort to design a national assessment system to measure these outcomes produced many good ideas and even a few workable prototypes. But consensus was hard to achieve about how and whether to proceed with what would have been an expensive and controversial enterprise, and the U.S. Department of Education proposal to undertake the project was never funded.

A few existing data sources provide some basis for estimating collegiate proficiency. For example, the National Adult Literacy Survey (NALS) conducted in 1992 can be used to estimate some higher-level literacy skills for college graduates. Aggregated results of the 1992 NALS, in fact, are included in the Educational Benefits grade in Measuring Up 2000. Also, large numbers of college students take statewide or nationwide examinations (professional licensure and professional school admissions tests, tests for aspiring K-12 teachers, and graduate school admissions exams). None of these, of course, were designed as national benchmarking tools, and the students who take them are neither broadly representative nor comparable across states. But this has not stopped some commentators from trying—with appropriate caveats—to make sense of them.

Why Is This Hard?

National data on academic achievement have been available for K-12 students for many years through assessments like the National Assessment of Educational Progress (NAEP), and most states use standardized achievement tests to examine primary and secondary school students, either for benchmarking purposes or to certify progress. Why haven't we done the same for "grade 16"? There are at least four reasons.

First, there is relatively little consensus about what the core outcomes of a college education ought to be. Institutions of higher education differ by design in both clientele and mission, and policymakers are aware of and largely support such differences. True, all colleges and universities offer "general education" courses, which are supposed to be teaching similar skills and knowledge. But except for having acquired communications skills and a basic set of quantitative abilities, graduates in different majors at different institutions arguably ought to look different. Coming to an agreement about performance standards for core generic abilities like "critical thinking" and "problem-solving" is thus a formidable task for both educators and policymakers.

Second, performance on any college-level exit assessment depends a lot on the abilities that students had when they arrived on campus. This means that "outcomes measures" for many institutions say more about how selective their admissions policy is than about what students learn while attending them. Admittedly, this difficulty is less troublesome when we look at state-level outcomes. But in most states, debates about whether to assess college students' achievement have focused on measuring institution-level performance—sometimes for high-stakes purposes like performance funding. As might be expected, many colleges and universities have little interest in going down that path.

A third challenge is how to create assessment instruments that can measure the abilities that constitute successful performance for college graduates. Educators and employers agree that the requisite abilities are too complex to be measured by multiple-choice tests. An appropriate assessment would require students to write extensively, solve open-ended problems, and perform real-life tasks. We now know a lot about how to create these kinds of tests, but commercial test-makers have been understandably reluctant to do so until there is a demand for them—and the first two difficulties have up to now restricted demand. As a result, the inventory of standardized tests designed for large-scale assessments of the outcomes of higher education is quite limited. Perhaps more strikingly in the light of a burgeoning "assessment movement" in higher education, most of these tests are over ten years old. This means that adequate assessments need to be created largely from scratch, at considerable expense, and no state has yet been willing to foot the bill.

Finally, the few states that have statewide testing programs find it hard to create conditions under which students will do their best. This problem has also been encountered in K-12, but a more generalized culture of student compliance tends to mitigate its effects. Motivating young adults and older returning students to show up for an examination that does not affect their coursework—let alone motivating them to try hard when they do—is not an easy task.