By Peter Ewell

And Paula Ries

December 2000

National Center for Public Policy and Higher Education

National Center Report #00-5

© 2000 by the National Center for Public Policy and Higher Education

Material may be duplicated with full attribution.

Foreword

Assessing Student Learning Outcomes was commissioned by the National Center for Public Policy and Higher Education as a supplement to Measuring Up 2000: The State-by-State Report Card for Higher Education.

This report, prepared by Peter Ewell and Paula Ries of the National Center for Higher Education Management Systems (NCHEMS), provides information on state efforts to assess student learning outcomes and track students as they graduate or transfer from one institution of higher education to another. The findings in this report informed Peter Ewell’s article in Measuring Up 2000, "Grading Student Learning: Better Luck Next Time."

Joy Gambino, a student intern at the National Center for Public Policy and Higher Education, contributed research related to student tracking and information systems in place in the states.

The National Center welcomes the comments of readers.

Vice President

The National Center for Public Policy

and Higher Education

In the Spring of 2000, the National Center for Higher Education Management Systems (NCHEMS) was asked by the National Center for Public Policy and Higher Education to conduct a survey of statewide efforts to assess student outcomes in conjunction with Measuring Up 2000: The State-by-State Report Card for Higher Education. The purpose of this project was to gather descriptive information on a state-by-state basis regarding two issues:

Accordingly, this report is divided into two major sections. The first summarizes states’ efforts at assessing collegiate learning outcomes. (For a detailed discussion of activities in individual states, see Appendix B.) The second section summarizes states’ student tracking efforts. The student learning outcomes information was collected from telephone surveys of State Higher Education Executive Officers (SHEEO) agency and statewide system staff conducted by NCHEMS. (See Appendix A for a copy of the interview protocol used.) Information about student tracking capabilities was gathered from the SHEEO survey and through survey work done by National Center staff.

The following cross-cutting themes emerged from the surveys regarding states’ efforts to collect student learning outcomes, and their student tracking efforts:

Findings: Student Learning Outcomes

For the purposes of this study, student learning outcomes are defined as communication (e.g., reading, writing, speaking), problem-solving and critical thinking skills. States’ assessment activities designed to measure these skills fall into six broad categories: 1) a common statewide test; 3) mandated statewide assessment with local choice of nationally-normed tests; 3) states in the process of developing a common approach to outcomes assessment; 4) state mandated assessments using locally developed or locally chosen instruments with a mandate that each institution report on its performance; 5) state mandated assessment with local instruments (as above) with no reporting requirements; and 6) no visible state requirement for assessing student learning outcomes. In all cases, these requirements apply only to public institutions.

Table 1 and Figure 1 summarize the states’ student learning outcomes assessment activities.

Table 1

Figure 1

1. Common Statewide Tests

Only six states currently collect state-level data on learning outcomes using a common statewide test: Arkansas, Florida, Georgia, South Dakota, Tennessee and Texas. Of these states, Florida, Georgia and Texas use a state-developed instrument. The other three use nationally normed instruments to assess student learning. Both South Dakota and Tennessee utilize incentive funding to encourage institutions to conduct assessments.

The following section briefly describes the assessment efforts in these six states.

Arkansas. In 1994, the Legislature mandated that all baccalaureate degree-seeking students take a Junior Rising Exam using the ACT Collegiate Assessment of Academic Proficiency (CAAP) when they accumulated between 45 and 60 credit hours, in order to assess their general education skills. Institutions received funding when they demonstrated success in teaching courses in the State Minimum Core (as evidenced by student test scores). In 1997, the Legislature discontinued performance-based funding as a consideration for institutional budget allocations. To date, however, the CAAP exam is still required of all rising juniors.

Florida. A 1983 state law requires that all students in public institutions must take the College Level Academic Skills Test (CLAST), in order to receive an Associate of Arts degree or continue on to upper division courses. CLAST comprises four tests in reading, writing, math and English, and was developed by the Florida Postsecondary Education Planning Commission.

Georgia. The Regents’ Testing Program (RTP) requires all sophomores enrolled in a baccalaureate or associate program to pass a common exam in order to earn a degree. The test consists of two parts, reading and an essay, and is administered each semester at all System institutions. Passing the test became a requirement for graduation from undergraduate degree programs (associate and baccalaureate) in 1973.

South Dakota. In 1997, the South Dakota Board of Regents adopted nine state policy goals for higher education, with one focusing on improving academic performance. In order to measure institutions’ progress toward this goal, a series of assessments is given to freshmen, sophomores and seniors. As of spring semester 1998, satisfactory performance on the CAAP was required of all students seeking baccalaureate degrees from the South Dakota Unified System of Higher Education. Baccalaureate degree-seeking students become eligible to sit for the exam after completing 48 pass credit hours, and associate degree-seeking students become eligible after passing 33 credit hours.

Tennessee. There are three types of student learning outcomes assessment: general education foundation testing, general education outcomes, and major field assessments. Only the first, foundation testing in general education, uses nationally normed tests, either the ACT COMP or College BASE. All undergraduates working toward an Associate of Arts or baccalaureate degree in either traditional degree or career training programs must take the exam, although institutions graduating more than 500 students a year may petition to test a representative sample of students. At least 30 percent of an institution’s students must be tested (and no fewer than 500). Results are used in the state’s performance funding scheme in which institutions are rewarded for high performance.

Texas. The Texas Academic Skills Program (TASP) was created by the Legislature in 1987 as an early assessment and academic support program for all students entering Texas public institutions of higher education. The first TASP test was administered in March 1989. Students are required to take the TASP assessment to determine if they have the reading, writing and mathematics skills to be successful in college. Students must pass all three sections of the TASP test before they can take upper-level courses beyond 60 semester credit hours and before they can graduate from a certificate, associate or baccalaureate degree program.

3) Mandated Statewide Assessment with Local Choice of Nationally Normed Tests

One state, Missouri, has mandated statewide assessments, but institutions are free to choose their own instrument as long as it is nationally normed. The state provides incentive funding to encourage institutions in their assessment efforts. Oklahoma mandates student learning assessment and encourages institutions to use nationally normed instruments, but institutions are allowed to develop their own.

Missouri. Public institutions are required to assess their programs every four years at state expense. No single instrument is required for student assessment, but the Missouri Coordinating Board does collect locally administered assessment data, and reports the proportion of students achieving certain scores on nationally normed exams. Institutions whose students score at certain levels on these exams or pass licensure or certification exams are given additional funds.

Oklahoma. Every public institution must evaluate students at four levels: 1) entry-level assessment (basic skills); 3) mid-level (general education competencies); 3) exit-level (program outcomes); and 4) student satisfaction regarding their educational experience. Mid-level assessment is designed to assess the basic skills competencies gained by students in the college general education program. Instruments used for mid-level assessment include locally developed as well as standardized testing instruments such as the ACT CAAP, the Riverside College Base Academic Subjects Examination (BASE), and the Test of Adult Basic Education (TABE).

3) States in the Process of Developing Outcomes Assessment

Eight states are considering or are actively in the process of developing outcomes assessments: Colorado, Connecticut, Kentucky, Massachusetts, New Mexico, Rhode Island, Utah and Virginia. Of these, Colorado and Utah appear to be the furthest along in these efforts.

Colorado. 1996 legislation created a comprehensive, comparative accountability initiative, the Quality Indicator System (QIS), which included new outcomes measures, and required explicit benchmarks of performance for all previously established quality indicators. An amendment called for assessing competency in functional skills and basic literacy for all sophomores as well as using a nationally normed standardized exam for all public institution graduates. The Colorado Commission on Higher Education has piloted ETS’ Academic Profile (AP) as a potential sophomore exam. Additionally, in 1999, the Commission required all institutions to submit a graduation-year assessment plan by fall 2000 for measuring skills and knowledge in the major. The Commission also is requiring that institutions either use ETS’ Major Field Exams or justify why they are not doing so.

Connecticut. The Connecticut Department of Higher Education created a Performance Measures Task Force and appointed members to represent the range of interests in the state. The Task Force has developed performance indicators, and has proposed definitions for the various institutional types in the state. Some examples include: 1) employer satisfaction with University of Connecticut graduates; 3) Connecticut State University graduates demonstrating competence in critical/analytical/logical thinking, effective writing, effective communication, use of scientific and quantitative skills, and life-long learning; and 3) upon completion of general education, Connecticut Community College students will demonstrate reading, writing and oral communication skills, an understanding of artistic and literary expression, ability to locate, analyze, synthesize and express ideas logically, and an understanding of social issues. The Task Force currently is developing instruments, identifying potential data sources, and creating timelines for assessment activities.

Kentucky. During the 1990s Kentucky legislators mandated the development of accountability indicators to be measured through a collaborative effort between the Kentucky Council on Higher Education and the public institutions of higher education. Additionally, an objective of the Strategic Plan for Kentucky Higher Education 1996–2000 requires institutional programs to "identify and evaluate students’ educational outcomes relative to program priorities within institutional missions." The council is working on identifying a methodology and appropriate instrument to use in assessing student learning outcomes, and is in the process of developing key indicators, some of which will include foundational skills, alumni satisfaction, civic engagement, preparation of teachers, and undergraduate student experience.

Massachusetts. The Massachusetts Board of Higher Education has created a task force to explore an exit assessment to be administered across the system. There is also a common writing assessment that is scored by institutional faculty. The board also collects a great deal of student unit record data on admissions information (for four-year institutions only) such as high school grade point average, ACT/SAT score, and ACUPLACER data on reading and math placement scores.

New Mexico. A 1990 law required the Commission on Higher Education to submit an annual report card to the governor and Legislature, highlighting each institution’s learning assessment programs and steps taken to improve learner outcomes. In 1999, the report card law was repealed and replaced with an annual Accountability Report. The commission is currently in the process of developing consistent indicators, some of which focus on individual institutional missions. The commission is discussing student learning outcomes as potential indicators. There are ongoing assessment activities at individual campuses.

Rhode Island. The Board of Governors identified five goals and priorities for higher education in 2000–2003, including quality assurance and outcomes assessment. Several initiatives include reviewing national and local assessment models to determine best practices, developing systemwide performance indicators that will evaluate outcomes, and incorporating NEASC accreditation learning outcome standards into institutional and system assessment plans.

Utah. The Utah State Board of Regents sponsored a pilot evaluation of general education using the CAAP in 1999–2000. The results of the pilot have led members of the Regents General Education Task Force to conclude that Utah institutions will be better served by developing statewide general education tests that are directly connected to course content and objectives, and to Regent-approved competencies. Thus far, systemwide faculty committees for writing, mathematics and American Institutions have identified competencies and are in the process of developing online assessments for students in these courses.

Virginia. The Governor recently convened a Blue Ribbon Panel on Higher Education, and student learning outcomes was one of the issues explored. Learning outcomes likely will be assessed as part of institutional effectiveness within a year, but it is not yet sure how this will be done. Institutions will be able to determine their expectations for their students and identify reliable and valid instruments to use. The state will require certain comparable measures regarding graduation rates, licensure, etc. The Panel also identified math, science, critical thinking, communication and technology as important skills, with the latter possibly becoming a statewide standard.

4) State-Mandated Assessment with Local Instruments and Reporting Requirements

Eight states have mandated assessment programs, but institutions are responsible for choosing or developing their own instruments: Hawaii, Iowa, Kansas, Louisiana, Maryland, Nevada, New York and North Carolina. These mandates also require assessment reports.

Hawaii. In 1996 the Legislature required the University of Hawaii to seek ways to measure and demonstrate the effectiveness of its programs and services, not only for accountability purposes, but also to inform improvement efforts. In response to this legislation, the Board of Regents devised policies that identified the purpose, guidance, commitment and policy base for institutional accountability and educational assessment processes for the regular and systematic assessment of programs, campuses and the University of Hawaii system. This policy also emphasizes that the university has purposely decentralized assessment activities. Each campus reports assessment information to the board.

Iowa. The Iowa Board of Higher Education requires institutions under its jurisdiction to assess outcomes for every program. The primary purpose is to improve student learning, teaching, and to provide an outcomes assessment component to ongoing program review efforts. Assessment varies by institution and program, with faculty taking the lead in identifying assessments and selecting or developing instruments.

Kansas. The Board has been the primary initiator of assessment in the state, but institutions develop their own assessment programs, and each program must assess basic skills, general education and student learning in the major. All public colleges and universities have created their own assessment plans based on institutional role and mission, which were approved by the Board in 1989. The plans must create and identify expectations for baccalaureate degree students in three areas: basic skills, general education and the major field of study. Assessment results are reported annually to the Board.

Louisiana. The Legislature required the Louisiana Board of Regents to identify general education requirements in English, math, humanities, natural sciences, social sciences and the arts for all public colleges and universities. "Recognizing the diverse missions of colleges and universities, and the lack of consensus regarding the best testing procedure, the Board of Regents leaves to each management board and campus the determination of what is the ‘appropriate’ testing procedure." However, many institutions use the ACT CAAP exam data as measures of general education outcomes (although not all do).

Maryland. A new accountability system adopted by the Maryland Higher Education Commission in 1996 gave the governing boards of the public campuses primary responsibility for monitoring student learning outcomes. The public campuses agreed to provide the commission with a report on their progress in improving student learning, instructional effectiveness, and curriculum every three years, beginning in 1998. The commission reviews institutional reports to determine whether assessment findings resulted in actions designed to enhance the instructional process, and reports to the General Assembly on the progress institutions are making toward improving student learning outcomes.

Nevada. The Board of Regents of the University and Community College System of Nevada adopted an assessment policy in 1989 requiring each campus to develop a plan of regular student assessment. Each campus assumes responsibility for developing the assessment processes and procedures to be used based on their mission, and must utilize multiple assessment approaches. Each campus president reports biennially on the results of the institution’s assessment activities. All institutions also must address assessment and student learning outcomes during their regular accreditation self-study reports and site visits.

New York. The State Education Department has mandated institution-based assessment for decades. The regulatory mandate affects all public, postsecondary institutions in the two major systems—SUNY and CUNY—as well as independent colleges and universities and degree-granting proprietary institutions. It is designed to foster assessment efforts within institutions but is not intended to make comparisons across institutions. SUNY just concluded a Task Force on student learning outcomes which recommended developing systemwide tests in a number of fields. For placement in and exit from remediation, CUNY is using the ACT ASSET tests in reading and writing, plus a written essay. Additionally, a Proficiency Examination has been developed by the CUNY faculty and outside consultants. It will assess students’ competence in academic reading, writing and critical thinking, and is in place as of Fall 2000. Students who enroll as first-time freshmen, beginning Fall 1999, and transfer students, beginning Fall 2000, must take and pass the test before moving to upper-division work or graduating with an associate degree.

North Carolina. Although there is no direct measure of student learning, the University of North Carolina (UNC) System surveys its graduating seniors, first-year alumni and their employers every year, in order to track students’ self-reported skill gains. Employers and alumni also are asked how well they were prepared for their jobs, and students in postgraduate programs of study are asked how well their undergraduate programs prepared them for advanced study. The UNC data are fed back to institutions for improvement purposes and are used in the Universities’ Performance Program Budgeting System.

5) State-Mandated Assessment with Local Instruments: No Reporting Requirements

Many states have a systemwide or state mandate for assessment but institutions are free to choose their own assessment instruments. However, the reporting requirements or financial consequences to these assessment activities are minimal. Illinois is typical of states in this category.

Illinois. In February 1999, the Illinois Board of higher education adopted "The Illinois Commitment: Partnerships, Opportunities, and Excellence." One of the Commitment’s six goals includes holding college students to higher expectations for learning and holding Illinois colleges and universities accountable for the quality of academic programs and the assessment of learning. One of the specific charges to institutions is to systematically assess student learning and to use the results to improve programs. There is not yet a mechanism in place to assess college student learning outcomes on a statewide basis.

While many states do not have a statewide mandate for assessing collegiate learning outcomes, some may require institutional data for statewide indicators (e.g., graduation and retention rates) and in other states, institutions may be engaging in their own assessment activities. Two states, Oregon and Washington, have developed competency-based admission requirements. Details of the efforts of all 50 states with respect to student outcomes assessment are described in Appendix B.

Findings: Student Tracking

 

This section describes the states’ efforts at tracking student persistence, completion and post-graduation status (e.g., whether graduates are working or are enrolled in further postsecondary education).

Persistence and Completion

The information discussed in this section was compiled by a National Center staff survey of Web sites of higher education coordinating boards, departments of education, and public systems. When information was not available on the Web site, appropriate individuals at a given agency were contacted by the National Center, either via e-mail or telephone.

Almost one-quarter of states have comprehensive systems in place to measure persistence and completion among students enrolled at public institutions: Arizona, Arkansas, Illinois, Indiana, Kentucky, New Jersey, New Mexico, New York, North Carolina, Oklahoma, Tennessee and Wisconsin. These state data systems are characterized by:

• Tracking cohorts of students that are full-time freshmen.

• Student persistence is tracked at least from the first to second years of study.

• Community college student persistence and completion is followed, although not every state monitors transfer rates.

• Six-year graduation rate data are collected (except for Illinois which follows its students for seven to ten years).

Slightly more states (38 percent) report completion data to the U.S. Department of Education obtained from their constituent institutions, but do not maintain their own tracking systems: Alabama, Alaska, Connecticut, Idaho, Iowa, Maine, Michigan, Minnesota, Nevada, New Hampshire, North Dakota, Oregon, Rhode Island and Wyoming.

Table 3 displays states’ data collection efforts according to this taxonomy.

Table 3

State-Level Data on Persistence and Completion

State	Description of the cohort	Calculation of the statistics	Contact
Alabama		Does not track graduation/persistence/transfer rates	Sandra Woodley, Director of Financial Affairs, AL Commission on Higher Education 334-343-3104
Alaska	IPEDS	IPEDS	Steve Merkel, University of Alaska Institutional Research and Budget Division, 907-474-7148
Arizona	First-time full-time students.	Tracks number of freshmen returning for their sophomore year. Tracks how many students graduate within six years. Tracks the four-year completion rate of community college students who enter AZ universities at the junior level or above. Also calculates a participation rate of community college students based on the total population aged 18 or older in the state. Additionally, several community colleges maintain employer satisfaction surveys of their graduates. In terms of transfer student success, AZ calculates the number and percentage of upper-division Arizona public university students who (1) have any community college credits and (3) have 13 or more community college credits.	Web site: Don Puyear puyear@stbd.cc.az.us (603) 355-4037, ext. 306. Can also be contacted for more information on community college data.
Arkansas	Cohorts of first-time freshmen beginning in 1993; statistics reported by gender, race and age.	Students in the cohort are tracked by enrollment patterns over a period of six years. Retention is reported as the percent of students still enrolled after year one through year six. A cumulative graduation rate also is computed for each of the six tracking years.	Steve Floyd, Deputy Director for Academic Affairs, AR Dept of HE 114 East Capitol, Little Rock, AR 73301 501-371 3031 stevef@adhe.arknet.edu
California	First-time baccalaureate degree seekers. Community college transfers also are tracked at Cal State and UC.	Numerator is the number of regularly admitted freshmen who graduated or continued within five years. (Another numerator is the number of community college transfers who graduate within three years). The denominator is the total number of students in the cohort.	SHEEO SURVEY
Colorado	First-time full-time degree seekers.	Numerator is the number of graduates and students still enrolled (and for community colleges, transfers to four-year institutions). Denominator is the cohort minus deaths, total disabilities and missionary service.	SHEEO SURVEY
Connecticut		Does not track graduation/persistence/transfer rates.	Dr. Jan W. Lyddon, Chief Research and Information Officer, Dept of HE 860-947-1843 jLyddon@commnet.edu
Delaware	Numerator is the number of graduates in 4,5,6 years. Denominator is the number of incoming full-time freshmen from 1991.	Tracks four-, five- and six-year completion rates.	Cynthia Kirkpatrick DE HE Commission 303-577-3340 ckirkpatric@state.de.us
Florida	First-time part- and full-time students.	Numerator is the number of baccalaureate degree earners, and the denominator is first-time students from the fall term and previous summer term who continued into the fall term, not restricted to full-time students; excludes non-degree-seeking students (e.g., special students).	SHEEO SURVEY
Georgia	Since 1987: Calculates a five-, six- and seven- year graduation rate statistic. First-time full-time freshmen.	Institution-specific graduation rates are calculated by dividing the number of graduates at the institution by the number of first-time full-time freshmen in 1987. The system-specific rate uses a numerator which includes those students who graduated from any institution in the system.	Web site
Hawaii	First-time full-time baccalaureate degree seekers.	Numerator is the total number of baccalaureate degree earners, and the denominator is the total number of students in the cohort. Hawaii does not edit the cohort for mission duty or death.	SHEEO SURVEY
Idaho		Does not track graduation/persistence/transfer rates.	Jerry Engstrom Management Information Officer, Idaho State Board of Education jengstro@osbe.state.id.us
Illinois	Cohorts of first-time full-time freshman since 1987 have been monitored. Seven- to ten- year completion rates are monitored. Focus is on time-to-degree rather than first to second year persistence.	Maintains a "Shared Enrollment and Graduation Information System" that includes students at all public community and four-year colleges. Tracking system monitors persistence, completion and time to degree. Students transferring out of the public system are considered "drop-outs."	Primary contact: David Smith, Assistant Director of Policy Studies, IBHE 317-557-7336 Smith@ibhe.state.il.us. Also contact Ann Bragg Associate Director of Academic Affairs, Board of Higher Education bragg@ibhe.state.il.us 317-557-7357
Indiana	Numerator is the number of students completing a degree within four and six years. Denominator is the number of first-time full-time freshmen beginning their studies in the fall term.	Since 1987, state law has mandated persistence and completion reporting. Prior to this mandate, the first cohort tracked was 1984-1985 freshmen who were monitored after four and six years. In the report presented in 1997, the cohort is fall 1990 degree-seeking freshmen. The degree level intention is based on the declared major in the freshman year. Also tracks associate degree students who began in 1993. A full-time student is defined as a student attempting 13 or more credits in the first year; a part-time student as one taking less than 13 credits. A similar procedure is used for associate degree-seeking students who are tracked after three and six years.	Jeff Webber, Manager of Information Research, Indiana Commission for Higher Education 317-464-4400 JeffW@che.state.in.us
Iowa		Only reported at the institutional level. Reports to IPEDS six-year rates.	Keith Greiner Keith.Greiner.csac.state.ia.us
Kentucky	Cohorts began with 1988 first-time freshmen.	Persistence rates reflect the percentage of full-time degree-seeking freshmen who either graduated, transferred to another public institution or were still enrolled at their original institution through the end of the designated tracking period. Baccalaureate students are tracked for six years, community college students are tracked for three years. At the technical colleges, non-cohort completion rates are calculated by dividing the total number of diploma and certificate completers by the total number of students exiting the program for any reason (transfer students are omitted from the analysis). Data are reported to the Legislature as part of the annual status/accountability report.	Connie Schumake Director of Data Collections connie.shumake@mail. state.ky.us
Maine		Only reported at the institutional level.	Gayla Shaw gshaw@maine.edu
Maryland	Tracks cohorts of students from 1977 to 1997.	Tracks the number and percentage of students who were retained at their original public four-year campus, and those who graduated from any senior public college or university in the state.	Monica Randall, Maryland Higher Education Commission Policy Analysis and Research Division, 410-974-3971 ext. 103
Massachusetts	Retention is defined as the rate at which full-time freshmen return to continue their education after one year of study. Community college persistence rates are defined as the percentage of students who return after completing one semester. Graduation rates are based upon completion of a degree within six years of enrollment.	Data are collected on persistence and completion, and are contained in the 1997 "Report on Public Higher Education." Data are collected for both two- and four-year institutions.
Michigan		No cohort data. Only reported at the institutional level.	Rhonda Burke, Higher Education Consultant, Community College Services Unit 517-335-0403 burkerp@state.mi.us
Minnesota	IPEDS	IPEDS
Mississippi	Nine cohorts (1985–1993) for each of the eight institutions in the state are represented in the MIS. Cohorts are tracked for six years.	Data are collected by the Institutions of Higher Learning Management Information System (MIS) on six-year graduation rates.	Dr. Milton Baxter, State of MS Board of Trustees of State Institutions of Higher Education 601-433-6396
Nevada	IPEDS	IPEDS	SHEEO Survey
New Hampshire	IPEDS	IPEDS	Patricia Edes New Hampshire Postsecondary Education Commission 603-371-3695
New Jersey	First-time full-time baccalaureate degree seekers.	Performance funding based on improved graduation rates involves three measures for the universities: overall graduation rate of freshman cohort; graduation rate for students needing remediation; and average time to degree. For the community colleges the goal is based on five measures: graduation and/or transfer rate of full cohort; success rate of full cohort (persistence); graduation rate of students needing remediation; success rate for remedial students; and average time to degree.	Phil Beardsley, SHEEO
New Mexico	Cohort data of first-time full-time freshmen beginning in fall 94 and fall 95 are used in the analysis.	In the 1999 "Student Persistence Report," data are presented on the persistence, transfer and degree completion of students. Data are disaggregated by type of institution and by enrollment intensity.
New York	HEDS tracks full-time undergraduate students’ graduation rates by program level, and tracks their ultimate employment status. Students entering community colleges in fall of 1993, and baccalaureate students entering in the fall of 1990 are the most recent cohort analyzed.	Maintains a "Higher Education Data System" (HEDS) that tracks enrollment and graduation rates. A 1996 report documents four-year graduation rates for associate degree students and six-year graduation rates for baccalaureate students. Data are presented for public and private institutions in the state, and are reported by disability status and race/ethnicity. Graduation rates do not track transfer students. Rates are computed as the number of entrants who graduated from the reporting institution within the time periods, as a percentage of all entrants (matriculating in the fall term).	Mary Jane Crotty Senior Statistician NY State Education Department Office of Research and Information Systems 518-486-3760 mcrotty@mail.nysed.gov
North Carolina	Retention rates are based on first-time full-time freshmen at UNC institutions. Completion data tracks cohort entering in 1991 and graduating in 1997. College going rates are monitored from 1981-83.	Based on a 1999 "Report on Retention, Graduation, and Time to Degree" presented to the Planning Committee of the UNC General Administration. UNC system calculates freshman to sophomore year retention rates, and four- and six-year graduation rates based on cohorts of students. Also tracks the UNC college going rates of in-state high school students.	George Antonelli antong@ga.unc.edu
North Dakota		No consistent reporting practice for graduation rates.	SHEEO SURVEY
Oklahoma	Beginning in 1985: Calculates persistence and completion and transfer rates for cohorts of students in the OK system.	Tracks student transfer and progress through two- and four-year institutions, and throughout the OK system. At the two-year institutions, graduation rates for both associate and baccalaureate degrees are calculated for two through six years. At the four-year and comprehensive institutions, graduation rates for baccalaureate degrees are calculated for two through six years. The persistence rate is calculated by taking the number of students retained the following fall plus the number of graduates, divided by the number in the original cohort. The graduation rate is the number of graduates, divided by the original cohort.	Dr. Debra Stuart, Associate Vice Chancellor for System Research 405-534-9170 dstuart@osrhe.edu
Oregon	IPEDS	IPEDS	SHEEO SURVEY
Pennsylvania		In the process of developing computer programs to calculate the graduation rate of 1993 cohort of first-time full-time undergraduates.	Devki Khanna PA Department of Education 717-783-6788 dkhanna@state.pa.us
Rhode Island		No systemwide data collection or reporting practice for graduation rates.	Diane Reedy. Forwarded by William R. Ferlad Rhode Island Office of Higher Education 401-333-6560 X133 dreedy@etal.uri.edu
South Carolina	First-time full-time degree seekers.	Numerator is the initial cohort of graduates plus transfer-in graduates. The denominator is first-time full-time cohort plus students transferring in with full-time status.	SHEEO SURVEY
South Dakota	Since fall 1993.	Tracks first-time degree-seeking freshmen who entered a regental university over four and five years. In addition to baccalaureate degree completion, SD tracks freshman to sophomore persistence and transfer rates.	Carol Stonefield Director of Information and Institutional Research SD Board of Regents 605-773-3455 carols@ris.sdbor.edu
Tennessee	Cohort is of 1993 first-time freshmen graduating by 1999.	Has published an annual report card since 1997 which monitors six-year graduation rates at Tennessee Board of Regents institutions. Additional measures, historically published in the TN Higher Education Commission Report, also show six-year graduation rates using the same calculation.	George Malo 615-366-4444 gmalo@TBR.state.tn.us
Texas	First-time full-time freshmen.	Numerator is the number of first-time full-time students who graduated, and the denominator is the number of students in the cohort.	SHEEO SURVEY
Vermont	Definitions in line with NCAA.	Retention and graduation data are based on entering cohorts of first-time freshmen students (most of whom are full-time). Beginning in 1978, cohorts are tracked for their four-, five-, and six-year graduation rates. Beginning with the 1986 freshman class, students have been monitored after each semester for first- to second-year retention rates. Rates are not broken down by residence, gender or race/ethnicity. Vermont State Colleges did not reply to e-mail.	Fred Curran Office of Institutional Studies 358 Waterman Building The university of Vermont Burlington VT 05405-0160 fcurran@zoo.uvm 803-656-0693
Virginia	Captures only full-time, program placed (matriculated), first-time students.	Graduation measure: This measure reflects graduation in six years within an institution.
Washington	First-time full-time baccalaureate degree seekers.	Numerator is the number of first-time full-time baccalaureate recipients, and the denominator is the number of students in the same cohort.	SHEEO SURVEY
West Virginia	First-time full-time students.	Numerator is the first-time full-time graduates (plus transfers who graduate). Denominator is all first-time full-time freshmen who began in fall; includes non-degree students.	SHEEO SURVEY
Wisconsin	Graduation rates are based on a six-year time period for first-time freshmen. Graduation rates for cohorts of fewer than 30 students are not calculated. The graduation rate takes the number graduating in six years and divides it by the beginning enrollment.	Has published an annual accountability report since 1993 that includes several cohort measures of persistence and completion. One measure calculates six-year graduation rate for students beginning and completing their education at the same UW institution, and for those who complete their course of study at any UW institution. Another measures four-year graduation rates by residency and institution. Part-time and full-time cohorts of students are tracked at the system and institutional levels. A new measure of first- to second-year retention is planned for 2000.	Jonathan Keller University of Wisconsin System Administration, Office of Policy Analysis and Research 608-363-3038 jkeller@uwsa.edu
Wyoming	Has established in the past records of first-time full-time, degree-seeking freshman cohort. Results will be available for future strategic plan reporting.	Using IPEDS data currently, but legislation is moving through the Wyoming Congress that will mandate common data elements for a state-level-management information system (MIS). To date, the Wyoming Community College Commission has reported on student access, linking student enrollment in community colleges to demographic trends. Additionally, the WYCC has coordinated a common Transfer Survey, Continuing Student Survey, Graduate Survey, and Employer Survey.	Steven Butler Senior Policy Analyst Wyoming Community College Commission sbutler@commission. wcc.edu
NCAA	First-time full-time baccalaureate Students entering an institution.	Tracks six-year graduation rate at 1,041 four-year member institutions. Numerator is the first-time freshmen entering six years prior to study date. Denominator is the total number of students who graduate from the institution within six years. Graduation rate does not account for transfers.	Maria DeJulio responsible for the collection and reporting of graduation rates. mdejulio@ncaa.org; Todd Petr NCAA Director of Research 317-917-6333

 

Post-Graduation Student Status

This information was gathered through interviews of SHEEO agencies and state higher education systems by NCHEMS. (See Appendix C for detailed discussions of post-graduation tracking efforts.)

States’ efforts at tracking their graduates fall into four broad categories: 1) state-level tracking with a common instrument or method; 3) state-level tracking, but without a common instrument; 3) one sector tracks its graduates; and 4) no state-level tracking (although many institutions conduct their own graduate surveys). Three states, Ohio, Utah and Virginia are in the process of developing graduate tracking systems. In many of the remaining states, individual campuses survey their alumni, but do not report their results to state agencies or systems. Table 3 and Figure 3 summarize the states’ student tracking efforts.

The most common type of data obtained is student post-graduation status, such as: a) employment status; b) postsecondary enrollment status; c) satisfaction with the undergraduate program/experience; and d) employer satisfaction with graduates. The most common method used to collect these data is self-report surveys. A handful of states link state unemployment insurance records with institutional student record data, with Florida and Texas leading the pack in these efforts.

 

Table 3

STATE	ACTIVITY	DESCRIPTION
Alabama	4	Individual institutions conduct alumni surveys
Alaska	4	Individual institutions conduct alumni surveys
Arizona	4	Individual institutions conduct alumni surveys
Arkansas	4	Individual institutions conduct alumni surveys
California	4	Individual institutions conduct alumni surveys
Colorado	3	State-level tracking, no common instrument
Connecticut	4	Individual institutions conduct alumni surveys
Delaware	3	Community colleges track their graduates
Florida	1	State-level tracking; link with state DOL files
Georgia	4	Individual institutions conduct alumni surveys
Hawaii	1	State-level tracking, common instrument
Idaho	3	Community colleges track their graduates
Illinois	1	State-level tracking, common instrument
Indiana	4	Individual institutions conduct alumni surveys
Iowa	1	State-level tracking, common instrument
Kansas	4	Individual institutions conduct alumni surveys
Kentucky	1	State-level tracking, common instrument
Louisiana	1	State-level tracking, common instrument
Maine	4	Individual institutions conduct alumni surveys
Maryland	3	State-level tracking, no common instrument
Massachusetts	3	State-level tracking, no common instrument
Michigan	3	State-level tracking, no common instrument
Minnesota	4	Individual institutions conduct alumni surveys
Mississippi	3	Community colleges track their graduates
Missouri	1	State-level tracking, common instrument
Montana	4	Individual institutions conduct alumni surveys
Nebraska	4	Individual institutions conduct alumni surveys
Nevada	4	Individual institutions conduct alumni surveys
New Hampshire	3	Community colleges track their graduates
New Jersey	4	Individual institutions conduct alumni surveys
New Mexico	1	State-level tracking, common instrument
New York	3	State-level tracking, no common instrument
North Carolina	3	UNC conducts alumni surveys
North Dakota	1	State-level tracking; link with state DOL files
Ohio	1	OBR is working on linking with state DOL database
Oklahoma	1	State-level tracking; link with state DOL files
Oregon	1	State-level tracking, common instrument
Pennsylvania	4	Individual institutions conduct alumni surveys
Rhode Island	4	Individual institutions conduct alumni surveys
South Carolina	3	State-level tracking, no common instrument
South Dakota	1	State-level tracking; link with state DOL files
Tennessee	1	State-level tracking, common instrument
Texas	1	State-level tracking; link with state DOL files
Utah	1	Board is working to link enrollment data with workforce department
Vermont	4	Individual institutions conduct alumni surveys
Virginia	4	Individual institutions conduct alumni surveys
Washington	3	Community colleges track their graduates
West Virginia	1	State-level tracking, common instrument
Wisconsin	3	The UW system surveys its graduates
Wyoming	3	Community colleges track their graduates

 

 

Figure 3

1) State-level Tracking with a Common Instrument

States in this category use a common instrument (such as the ACT OPINION) to survey their graduates, link institutional enrollment data with state employment data files to determine graduates’ status (such as working or enrolling in graduate school), or both.

Florida. The Florida Education and Training Placement Information Program (FETPIP) is a comprehensive data collection system that obtains follow-up data on former students. Social security numbers, obtained from postsecondary institutions, are linked with unemployment insurance files. FETPIP also conducts employer opinion surveys.

Hawaii. The university conducts an alumni outcomes survey and a graduating senior survey, both of which are organized around the system’s strategic plan (e.g., access, distance learning, graduation, exam performance for nursing, teaching, law, medical, etc.). Responses also address student satisfaction and transfer articulation. The surveys are conducted every three years.

Illinois. Every three years a cohort of graduates is followed up at one, five and nine years after graduation. They are surveyed about current employment, how closely it is related to their baccalaureate major, whether they are pursuing (or have pursued) additional degrees, and satisfaction with their academic preparation.

Iowa. The Board of Regents has just begun surveying graduates to see where they are employed (in or out of state) or whether they are in graduate school. These data are used primarily for economic development planning activities.

Kentucky. The Council is developing a new statewide alumni survey. The core questions, to be used by all institutions, concern: 1) satisfaction with instruction, curriculum, preparation for work, etc.; 3) current enrollment status in postsecondary education ; and 3) employment status and the degree to which that employment is related to the college major. Institutions will have the option of adding their own questions.

Louisiana. The Louisiana Board of Regents uses the ACT OPINION survey to obtain data about student satisfaction on a statewide basis, with different versions for two- and four-year institutions. They also are fielding two other surveys, one for non-returning students and one for alumni outcomes, both of which are ACT products.

Missouri. The two-year system follows up on its graduates 180 days after graduation, for Perkins legislation, to determine whether students are working in the fields for which they were trained. The two-year and four-year public institutions with professional/technical programs periodically conduct a sample alumni survey using ACT’s "Counseling for High Skills" instrument which reports students’ perceptions of how well they were trained for their career or field.

New Mexico. All the four-year institutions conduct alumni surveys, which have common questions, but this is not coordinated by the Commission on Higher Education. The community colleges are working with the labor department to link their data with the unemployment insurance system to track their graduates.

North Dakota. The University system follows up on its graduates with assistance from the Follow-up Information on North Dakota Education and Training (FINDET), a consortium of several state agencies. As part of the new accountability effort, the system is also considering assessing both graduate and employer satisfaction.

Oklahoma. An annual employment survey is conducted by linking Oklahoma system records with state employment records to track what proportion of graduates are employed in which industries.

Oregon. The Board collects data on employment, further education, personal income, satisfaction with education, etc., for graduates one and five to ten years after graduation. Bachelor’s graduates and advanced degree recipients are surveyed separately. The Board is in the planning phase of developing a survey of employers.

South Dakota. Along with other state agencies, the university system participates in the South Dakota Follow-up Program managed by the state’s Department of Labor. The Board provides information on graduates for each fiscal year, and enrollments as of October. The Department of Labor tracks graduates using unemployment, federal sources and driver licenses, and collects data on wages and benefits by surveying South Dakota employers.

Tennessee. Institutions survey either alumni or currently enrolled students in alternating years using the ACT Student Opinion Survey (there is one version for four-year institutions and one for two-year institutions) or the ACT Alumni Survey. The former assesses enrolled students’ perceptions of the institution’s programs, services and environment, while the latter compiles employment history and students’ perceptions of the impact of their educational programs and experiences. Two-year institutions must survey their graduates annually to determine the number working in the fields for which they were trained.

Texas. Public community and technical colleges, and four-year institutions that award associate degrees, are required by Perkins legislation to document post-exit results achieved by "completers" and "leavers." The Coordinating Board contracts with the Texas State Occupational Information Coordinating Council (SOICC) to accomplish this. Participation in SOICC’s student tracking program is not required for public universities and four-year institutions that do not offer associate degrees, health sciences institutions, proprietary and private four-year colleges and universities. However, because several large public universities participate (notably the University of Texas and Texas A&M systems), approximately 75 percent of all of Texas’ postsecondary completers and leavers are included in the state’s automated students and adult learner follow-up.

West Virginia. Institutions conduct an undergraduate satisfaction survey. The state also collects data on whether graduates are working in-state, either working or pursuing further education, and average income. Data are reported in the aggregate (not broken down by institution) in the report card.

3) State-Level Tracking, Without a Common Instrument

States in this category require institutions to conduct follow-up assessments, but allow them to choose the instrument.

Colorado. The accountability program requires that institutions provide evidence of after-graduation performance in employment, professional advancement or achievement in graduate/professional school. Institutions also are required to assess student/alumni satisfaction with their educational experience.

Maryland. Institutions report to the Higher Education Commission on two common indicators: admission of undergraduates to post-baccalaureate study, and employment rates of graduates.

Massachusetts. Individual campuses survey graduates and send the Board of Higher Education aggregate data such as the proportion of graduates employed, etc. The community college system uses common/comparable questions, but the four-year institutions do not.

Michigan. All colleges conduct placement follow-up surveys to obtain information regarding the number of students who graduated, how many continued their education, are employed, etc. The Department is considering linking their database with Unemployment Insurance data to get better results. The community colleges collect graduation rates, further postsecondary education, unemployed/employed, etc. to satisfy federal Perkins requirements.

New York. The State Education Department does not track graduates, but individual institutions do. SUNY system administration conducts an Alumni Survey every three years. CUNY’s two-year technical programs have to track "completers" per federal Perkins requirements. Additionally, there are plans to survey four-year graduates to see what they are doing regarding employment or further postsecondary education.

South Carolina. There is no statewide survey of alumni, but institutions are required to follow up on their graduates and include these data in their annual assessment report.

3) One Sector Tracks its Graduates

In this category, only one sector or system tracks its graduates.

Delaware. The community colleges collect self-report data on graduates’ employment and transfer status.

Idaho. Under federal Perkins regulations, the vocational/technical system is required to track their graduates 180 days after graduation to see whether they are working in the fields in which they were trained, are in the military, are unemployed or are not seeking employment. If a program is "unproductive" (i.e., doesn’t graduate students) or there are no jobs in a particular training area, that program may be phased out.

Mississippi. The community and junior colleges collect data on community and junior college students who transfer to the university system, as well as transfer-student performance versus native-student performance. Since the State Department of Education allocates Perkins III funds, it tracks students upon completion of a technical or career program.

New Hampshire. The community and technical college system tracks their graduates to determine whether they are working full- or part-time, and collects salary data. The results are used in a mandated annual report to their board and the governor.

North Carolina. Every two years graduating seniors are surveyed regarding whether they feel well prepared for employment and/or graduate school. The data are used as baseline evidence of institutional improvement and are fed back to all institutions.

Washington. The community colleges track their students into the workforce and collect data on employment status and wages using a direct linkage to SOICC-maintained wage record data.

Wisconsin. The University of Wisconsin system surveys its graduates.

Wyoming. Six months after graduation, two-year graduates are surveyed to see whether they are working, what their income is, and to determine their level of satisfaction with student services as well as self-reported gains in areas like communication and computer literacy.

Summary

 

Progress is being made across the states toward collecting more information on what students are learning during college, and what they are doing after college:

• Nearly one-half (44 percent) of the states (or state systems) require institutions to conduct some type of student learning assessments, either with a common test, nationally normed tests, or locally developed or selected tests.

• One-quarter (34 percent) of the states have comprehensive data systems to track student persistence and completion.

• Almost one-third (30 percent) of the states regularly survey their graduates using a common statewide instrument.