Technical Guide
Documenting Methodology,
Indicators, and Data Sources
For
Measuring Up 2002:
The State-by-State Report
Card for Higher Education
October 2002
The National Center for Public Policy and
Higher Education
National Center Report #02-8
© 2002 by The National Center
for Public Policy and Higher Education.
Foreword............................................................................................................................... v
Introduction........................................................................................................................... 1
I. Scoring and Grading State Performance ........................................................................... 2
II. Graded Performance Categories....................................................................................... 6
A. Preparation........................................................................................................... 7
B. Participation......................................................................................................... 21
C. Affordability......................................................................................................... 25
D. Completion.......................................................................................................... 33
E. Benefits............................................................................................................... 40
F. Learning............................................................................................................... 51
III. Non-Graded Measures.................................................................................................... 52
About the National Center for Public Policy and Higher Education........................................... 59
|
T |
his Technical Guide provides complete information on the indicators presented in Measuring Up 2002: The State-by-State Report Card for Higher Education. Measuring Up 2002 is available in full at www.highereducation.org.
The Technical Guide describes all indicators in Measuring Up 2002, explains the methodology used to measure the indicators, and lists the sources of data. The Technical Guide was compiled by Mikyung Ryu, policy analyst at the National Center. William Doyle, senior policy analyst at the National Center, provided a review.
The National Center welcomes the comments of readers.
Joni Finney
Vice President
The National Center for Public Policy and Higher
Education
|
T |
his Technical Guide describes the methodology and concepts used to measure and grade the performance of the 50 states in the higher education arena.
Part I presents the methodology for grading states and provides information on data collection and reporting. Part II explains the indicators that comprise each of the graded categories. Specifically, this section details the construction of each indicator—its scope, source, and computations. Part III provides data sources for non-graded information, some of which is provided in the print edition of Measuring Up 2002, some of which is posted on the on-line version of the report card. The non-graded information provides a context for understanding graded state performance.
Data spreadsheets for Measuring Up 2002 can be downloaded from the National Center’s Web site at www.highereducation.org, under “Follow-Up Documents.”
The term higher education is used consistently throughout the Measuring Up series. In this context, higher education refers to the postsecondary education and training offered by accredited degree-granting colleges and universities that are eligible for Title IV federal financial aid. Private for-profit institutions run by employers to provide specific job-related training are not included. Unless otherwise noted, the indicators used in Measuring Up 2002 refer to the postsecondary education and training through the bachelor’s degree.
Each of the graded categories contains a number of relevant indicators. These indicators, culled from nationally comparable data, represent variables that explain, in part, statewide variation in category performance.
State performance on different indicators is compared through an indexing method in which raw scores for each indicator are scaled to the median value of the top five performers. This median of best performance is the benchmark for all other states. Each state’s raw scores are indexed to (that is, divided by) the benchmark scores on every indicator in every category.
Once indexed, each state score is multiplied by a predetermined “weight” that accounts for the indicator’s relative importance in predicting category performance. The value of each weight was determined by existing quantitative research documenting the significance of these variables as a measure of category performance. Although some indicators are weighted more heavily than other indicators in a category, the sum of all assigned weights totals 100%. At the introduction of each category in Part II, the specific weights assigned to each indicator are described.
In practice,
once the value of each indexed indicator is multiplied by the appropriate
weight, the weighted indexed values are totaled. From these totaled scores, the
single best performer in the category is identified. The best performer’s
overall score in the category is then set to 100 and the overall scores of all
other states are indexed to this. The exception to this process occurs when the
best performer’s score is over 100. In that case, the best score is set to 100
and all other states are indexed to 100. The result is the category index score to which alphabetic grades are assigned for
each state. The following grading scale is used.
|
Grading Scale |
|||||
|
93 and above |
A |
80–82 |
B– |
67–69 |
D+ |
|
90–92 |
A– |
77–79 |
C+ |
63–66 |
D |
|
87–89 |
B+ |
73–76 |
C |
60–62 |
D– |
|
83–86 |
B |
70–72 |
C– |
Below 60 |
F |
Measuring Up 2002 presents information about whether state performance has improved from 1998 to 2000, which are the most common data years represented in Measuring Up 2000 and Measuring Up 2002. In identifying performance improvement, only those indicators that are comparable in both report card editions are selected (see sidebar).
First,
it is determined whether a state’s updated raw score for each indicator in a
category improved by at least one percent since Measuring Up 2000.
Second, for each indicator showing improvement of at least one percent in its raw score, predetermined “progress points” are awarded. Progress points equal the weights of each indicator as defined in Measuring Up 2000. The underlying rationale is that improvement observed in each compared indicator is taken into account to the extent that each of these indicators is important in predicting overall performance in the category. If the weights of all compared indicators do not total 100% because not all indicators are compared, the weights of each compared indicator increase in proportion to the original weights. For example, if a state’s raw score improved in an indicator whose original weight was 10% and the totaled weights of all compared indicators in the category are 90%, then the weight is adjusted to 11% (10 divided by 90). Therefore, the state receives progress points of 11% in this indicator.
Finally, the earned progress points are added (that is, the weights of all the indicators in which the state’s raw scores improved by at least one percent are added). If these totaled weights are greater than 50%, then the state’s performance in the category is considered as “improved” since Measuring Up 2000; that is, with the weights of the indicators taken into account, the state has improved on the “majority” of measures. The same procedure is followed for all five categories.
This method applies only when data are available for a state for both years. For a state with missing data for certain indicators, performance improvement is determined based on all available data for the state, after adjusting the indicator weights proportionately based on the original size of weights.
The Measuring Up series includes the most recent data available. Unfortunately, collecting agencies often require months to analyze and disseminate reports to the public. Additionally, in some cases, data are not systematically collected each year. Finally, it is possible that future assessments or studies have not received authorization or funding for subsequent data collection. For one or more of these reasons, state results on the report card’s indicators may lag behind recent changes or incompletely capture the most recent initiatives that state policymakers have implemented.
Like Measuring Up 2000, Measuring Up 2002 uses reliable and comparable state-by-state data. Despite the scientific survey methods used to collect this data, information cannot always be reported reliably for each state because many national surveys do not oversample populations from every state. Thus, estimates of the behaviors, characteristics, or educational performance of the populations in small states are often not captured adequately by nationally drawn random samples. In cases of nationally administered surveys such as the National Assessment of Educational Progress (NAEP), states are given the option to participate in a state oversample and may decline to take part.
Measuring Up 2002 includes some indicators in which data are missing for more than one state; such indicators were retained because they reflect salient policy issues and signal the necessity to expand formal data collection to all 50 states.
To adjust for missing data, several strategies were considered and discarded. Choosing to assign a zero value to states that did not report data on specific indicators presumes the lowest possible performance. Alternatively, relying on the mean value of all states’ performance presumes similarity between states that are in fact quite distinct. Calculating a grade using only available data distorts the weighting method applied because indicators with data become more important than those without data in the calculation of the overall grade, regardless of their overall influence in determining category performance.
Consequently, where no comparable data exist to gauge a state’s performance on an individual indicator, a technique known as imputation is applied. This strategy calculates the weighted mean value of state performance on indicators within a given category for which data are available for the state and applies that value where data are otherwise missing. This technique is applied to every state with missing data, using the weighted mean score of the state’s own performance on other indicators. Imputing in this way presumes the state does neither better nor worse on an indicator for which it is missing data than it does on highly correlated indicators within the same category.
In cases where some states did not participate in the most recent survey, although they had previously participated, the report card applies the latest data available principle. This means that, to calculate the final grade, states’ raw scores on each indicator are derived from the most recent survey that they participated in, not necessarily the most recently administered survey. Therefore, for several states, Measuring Up 2002 uses the scores from the previous report card. In Part II, where data availability is noted for each indicator, the states are identified for which Measuring Up 2002 uses previous data.
An estimate derived from a sample rather than the entire population can vary, depending on different sample populations. Standard error is a measure of the variability among all possible samples. The accuracy of an estimate decreases with larger standard error. When state estimates are produced with a large standard error, the estimates are unlikely to be precise. Therefore, the report card applies the rule that an estimate with a standard error of 10 percentage points or greater is not reliable and is considered as missing data. In such cases, the state score is based on data used in the previous report card, or on an imputed value if previous data are unavailable.
Migration of students and college graduates is a critical component of state performance in many of the categories. Although indicators in the educational benefits and participation categories would do well to adjust for migration, this type of detailed analysis is simply not possible at the state level, given current practices of data collection. Unless otherwise noted, population changes resulting from inter-state migration cannot be accounted for in this report card due to data limitations.
|
S |
ix overall categories are used in the Measuring Up series to gauge state higher education performance: preparation, participation, affordability, completion, benefits, and learning. As in Measuring Up 2000, all states again receive an Incomplete in learning because states lack information on the educational performance of college students that would permit systematic state-by-state comparisons.
In Measuring Up 2002, four new indicators have been introduced and a few existing indicators are now calculated with more refined methodologies. The new indicators and revised methodologies are described in this section.
The following pages detail each of the performance categories, describing the indicators and the indicator weights used to calculate states’ grades. Also presented is a comprehensive catalogue of data sources, indicating collecting agency and periodicity.
|
Preparation: Indicators and Weights |
|
|
Indicator |
Weight |
|
Cluster 1: High School Completion |
20% |
|
18- to 24-year-olds with a high school credential |
20% |
|
Cluster 2: K–12 Course Taking |
40% |
|
9th to 12th graders taking at least one upper-level math course |
10% |
|
9th to 12th graders taking at least one upper-level science course |
15% |
|
8th grade students taking algebra |
10% |
|
12th graders taking at least one upper-level math course |
5% |
|
Cluster 3: K–12 Student Achievement |
40% |
|
8th graders scoring at or above “proficient” on the national assessment exam in math |
4% |
|
8th graders scoring at or above “proficient” on the national assessment exam in reading |
4% |
|
8th graders scoring at or above “proficient” on the national assessment exam in science |
4% |
|
8th graders scoring at or above “proficient” on the national assessment exam in writing |
4% |
|
Low-income 8th graders scoring at or above “proficient” on the national assessment exam in math |
4% |
|
Number of scores in the top 20% nationally on SAT/ACT college entrance exam per 1,000 high school graduates |
10% |
|
Number of scores that are 3 or higher on an Advanced Placement subject test per 1,000 high school juniors and seniors |
10% |
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
This measure uses the following calculation:
§ Numerator: Number of 18- to 24-year-olds in the population holding a high school credential.*
§ Denominator: Total population ages 18 to 24, excluding those still enrolled in high school or currently pursuing alternative certification.
* High school credential includes a high school diploma or alternative certification such as a General Educational Development (GED) diploma.
This indicator measures the extent to which the young adult population in the state is minimally certified to participate in the workforce and participate in postsecondary education. This indicator is not a calculation of the percentage of students graduating from high school in four years. Given the drop-out and re-entry patterns of many students, a simple calculation of high school graduation rate would fail to capture their eventual completion.
This indicator pools three years of the most current data, 1998 to 2000, to obtain a large enough sample size to make reliable state estimates and to account for aberrations in any single year of data. Using this method, data are available for all 50 states.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Washington, D.C.: Council of Chief State School Officers, 2002, p. 43.
This indicator measures the percentage of public high school students in the state in grades 9 to 12 who took one or more math courses at levels 2 through 5 during the 1999–2000 school year. These math courses include geometry, algebra 2, integrated math 3, trigonometry, pre-calculus, calculus, and AP calculus.
Although high school course taking in humanities is also important to students’ preparation, neither the Council of Chief State School Officers (CCSSO) nor any other organization collects this type of comparative data from the states.
Data are available for 32 states—including Delaware and Michigan, for which the latest data available method was applied (that is, their previous data, from Measuring Up 2000, were used because they did not participate in the most recently administered survey).
States for which data are unavailable are: Alaska, Arizona, Colorado, Florida, Georgia, Hawaii, Illinois, Kansas, Maine, Maryland, Montana, New Hampshire, New Jersey, Pennsylvania, Rhode Island, South Carolina, Virginia, and Washington.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Washington, D.C.: Council of Chief State School Officers, 2002, p. 44.
A separate but similar indicator to math course taking, science course taking measures the extent to which students in the state were enrolled in one or more of the following science courses during the 1999–2000 school year: all courses in chemistry or physics, second-year biology, AP biology, second-year earth science, or other advanced science courses.
Although high school course taking in humanities is also important to students’ preparation, neither the CCSSO nor any other organization collects this type of comparative data from the states.
Data are reported for 32 states—including Delaware and Michigan, for which the latest data available method was applied (that is, their previous data, from Measuring Up 2000, were used again because they did not participate in the most recently administered survey).
States for which data are unavailable are: Alaska, Arizona, Colorado, Florida, Georgia, Hawaii, Illinois, Kansas, Maine, Maryland, Montana, New Hampshire, New Jersey, Pennsylvania, Rhode Island, South Carolina, Virginia, and Washington.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Washington, D.C.: Council of Chief State School Officers, 2002, p. 46.
This indicator measures the percentage of public school 8th grade students in the state who took algebra 1 during the 1999–2000 school year.
Data are available from 30 states—including Delaware, Michigan, and New York, for which the latest data available method was applied (that is, their previous data, from Measuring Up 2000, were used again because they did not participate in the most recently administered survey).
States for
which data are unavailable are: Alaska, Arizona, Colorado, Florida, Georgia,
Hawaii, Illinois, Iowa, Kansas, Maine, Maryland, Montana, New Hampshire, New
Jersey, Pennsylvania, Rhode Island, South Carolina, Texas, Virginia, and
Washington.
Calculations based on unpublished data provided by Science and Math Indicator Project Team at Council of Chief State School Officers. The data are from the same source that was used for: Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Washington, D.C.: Council of Chief State School Officers, 2002.
This newly added indicator measures the percentage of public high school seniors in the state who took at least one advanced math course during the 1999–2000 school year. In recent years, much attention has been paid to the problem of America’s high school seniors. The indicator provides a current status check on how many high school students maintain academic rigor during their last year. Students tend not to take academically demanding courses after college acceptance and thus their preparation for postsecondary education or the workforce may decline. In order to ensure that more students are ready for a successful transition, state policies requiring more rigorous course enrollment throughout all high school years are suggested.
The indicator uses the following calculation:
§ Numerator: Number of public high school seniors enrolled in math courses at levels 2 through 5* during the 1999–2000 school year.
§ Denominator: Number of public high school seniors enrolled for the 1999–2000 school year.
* These courses include geometry, algebra 2, integrated math 3, trigonometry, pre-calculus, calculus, and AP calculus.
Data are reported for 16 states. Many states are missing data because they declined to participate in the survey, or they did not report course enrollments by grade level.
States for which data are unavailable are: Alaska, Arizona, Colorado, Delaware, Florida, Georgia, Hawaii, Illinois, Iowa, Kansas, Kentucky, Louisiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, Nevada, New Hampshire, New Jersey, New York, Ohio, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, Tennessee, Virginia, and Washington.
National Assessment of Educational Progress. The Nation’s Report Card, Mathematics 2000. Washington, D.C.: U.S. Department of Education. http://nces.ed.gov/nationsreportcard/mathematics/ (accessed 2/15/02).
These proficiency rates describe the percentage of 8th graders enrolled in public school whose performance on the National Assessment of Educational Progress (NAEP) exam in math was “proficient” or “advanced.”
Academic proficiency levels are determined by the National Assessment Governing Board, based on judgments about what students should know and be able to do.
Thirty-nine states participated in the 2000 assessment. Measuring Up 2002 reports 7 more states in addition to the 39 participating states, by using the latest data available method (that is, using the 1996 assessment data, as reported in Measuring Up 2000, for the states that did not participate in the 2000 assessment). These additional 7 states are Alaska, Colorado, Delaware, Florida, Iowa, Washington, and Wisconsin.
States for which data are unavailable are New Hampshire, New Jersey, Pennsylvania, and South Dakota.
National Assessment of Educational Progress. 1998 National and State Reading Summary Data Tables for Grade 8 Student Data. Washington, D.C.: U.S. Department of Education, 1998.
These proficiency rates describe the percentage of 8th graders enrolled in public school whose performance on the National Assessment of Educational Progress (NAEP) exam in reading was “proficient” or “advanced.”
Academic proficiency levels are determined by the National Assessment Governing Board, based on judgments about what students should know and be able to do. The reading assessment is usually conducted every four to five years, and the 1998 assessment that was reported in Measuring Up 2000 is still the most recent. Therefore, the reading data could not be updated for Measuring Up 2002.
Thirty-six states participated in the 8th grade reading assessment. States for which data are missing are: Alaska, Idaho, Illinois, Indiana, Iowa, Michigan, Nebraska, New Hampshire, New Jersey, North Dakota, Ohio, Pennsylvania, South Dakota, and Vermont.
National Assessment of Educational Progress. The Nation’s Report Card, Science 2000. Washington, D.C.: U.S. Department of Education. http://nces.ed.gov/nationsreportcard/science/ (accessed 11/21/01).
This new indicator measures the percentage of 8th graders enrolled in public school whose performance on the National Assessment of Educational Progress (NAEP) exam in science was “proficient” or “advanced.”
Academic proficiency levels are determined by the National Assessment Governing Board, based on judgments about what students should know and be able to do.
Thirty-eight states participated in the 2000 assessment and are reported in Measuring Up 2002.
States for which data are missing are: Alaska, Colorado, Delaware, Florida, Iowa, Kansas, New Hampshire, New Jersey, Pennsylvania, South Dakota, Washington, and Wisconsin.
National Assessment of Educational Progress. 1998 National and State Writing Summary Data Tables for Grade 8 Student Data. Washington, D.C.: U.S. Department of Education, 1998.
This measure describes the percentage of 8th graders enrolled in public school whose performance on the National Assessment of Educational Progress (NAEP) exam in writing was “proficient” or “advanced.”
Academic proficiency levels are determined by the National Assessment Governing Board, based on judgments about what students should know and be able to do. The writing assessment is conducted every four to five years, and the 1998 assessment that was reported in Measuring Up 2000 is still the most recent. Therefore, the writing proficiency data could not be updated for Measuring Up 2002.
Thirty-five states participated in the 8th grade writing assessment. States for which data are missing are: Alaska, Idaho, Illinois, Indiana, Iowa, Kansas, Michigan, Nebraska, New Hampshire, New Jersey, North Dakota, Ohio, Pennsylvania, South Dakota, and Vermont.
National Assessment of Educational Progress. The Nation’s Report Card, Mathematics 2000. Washington, D.C.: U.S. Department of Education. http://nces.ed.gov/nationsreportcard/mathematics/ (accessed 2/15/02).
This indicator describes the percentage of public school 8th graders who are eligible for free or reduced-price lunch and whose performance on the National Assessment of Educational Progress (NAEP) exam in math was “proficient” or “advanced.”
Academic proficiency levels are determined by the National Assessment Governing Board, based on judgments about what students should know and be able to do.
Thirty-nine states participated in the 2000 math assessment. Measuring Up 2002 reported four additional states, using the latest data available method (that is, for those that did not participate in the 2000 assessment, Measuring Up 2002 uses the 1996 assessment data as reported in Measuring Up 2000). These states are: Colorado, Delaware, Florida, and Washington.
States for which data are unavailable are: Alaska, Iowa, New Hampshire, New Jersey, Pennsylvania, South Dakota, and Wisconsin.
The College Board. “2001 SAT V+M Score Bands Report,” unpublished report, New York, 2001.
ACT. “Percent of 2001 High School Graduates with ACT Composite Scores of 26 or Higher,” unpublished analysis, Iowa City, Iowa, 2001.
Western Interstate Commission for Higher Education. Knocking at the College Door: Projections of High School Graduates by State and Race/Ethnicity, 1996–2012. Boulder, CO: 1998.
Gary L. Marco, et. al. Methods Used To Establish Score Comparability on the Enhanced ACT Assessment and the SAT. New York: College Board, 1992.
This indicator reflects the prevalence of four-year college test-taking throughout the state as well as the achievement that student test takers demonstrate. College entrance exam achievement is calculated per 1,000 high school graduates in each state, using the following formula:
§ Numerator: (Number of public and private high school seniors scoring at or above 1200 on the SAT test) + (number of public and private high school seniors scoring at or above 26 on ACT test).
§ Denominator: Number of public and private high school graduates for the 2000–01 school year.
Nationally, 18% of test-takers scored at or above 1200 on the SAT in 1999. Students attaining this score or higher approximate the top quintile (20%) of SAT test takers. Though the ACT exams are administered independently and use a different scoring methodology than that used by the College Board for SAT scores, a common conversion method can be applied. A score of 26 on the ACT exam is equivalent to a score of 1200 on the SAT exam.
The National Educational Longitudinal Study of 1988 (NELS) indicates that 15% of high school seniors take both the SAT and the ACT, although data are not collected in such a way as to provide an unduplicated count of test takers. This ratio does not provide information on the number of students in each state who take college preparatory exams. Instead, the numerator measures the total number of scores at or above 1200 on the SAT and 26 on the ACT. Constructed this way, the measure accounts for individual students who perform proficiently on more than one college entrance exam.
Data are available for all 50 states.
The College Board. 2001 State and National Annual Summary Reports. New York: The College Board. http://www.collegeboard.com/ap/library/state_nat_rpts_01.html (accessed 6/14/02).
This indicator measures the number of Advanced Placement subject tests taken by 11th and 12th grade students with scores of 3 or higher per 1,000 11th and 12th grade students enrolled in public and private schools. The measure uses the following calculation:
§ Numerator: Number of 11th and 12th graders’ Advanced Placement subject test scores of 3, 4, or 5.
§ Denominator: Total 11th and 12th graders enrolled in public and private schools.*
* The number of 11th and 12th graders enrolled in public and private schools was computed by multiplying the public enrollment by a private-enrollment adjustment factor developed by a data contractor working with the College Board. The total 11th and 12th grade enrollment in the state is used because the majority of AP test-takers (88% in 2001) were enrolled in these grades.
This ratio does not provide information on the number of students in each state who take an Advanced Placement test. Instead, the numerator measures the total number of scores at or above 3. Constructed this way, the measure accounts for individual students who perform proficiently on more than one AP subject test. Scores at or above 3 are generally recognized for college credit.
Opportunities other than AP exist for high school students to take college-level courses, including the International Baccalaureate (IB) program and college concurrent enrollment programs. The Advanced Placement program offered by the College Board is the most prevalent in U.S. high schools and the most widely recognized for credit by policymakers and colleges and universities.
Data are available for all 50 states.
The participation category assesses the opportunities in each state for residents of varying ages and income levels to enroll in postsecondary education.
To broadly assess state performance in this category, various enrollment patterns and institution types are considered. These include full- and part-time enrollment at both two- and four-year institutions, and public and private colleges. Due to the lack of nationally comparable data, however, participation in non-accredited institutions, corporate or employer-sponsored education or training programs is not included.
The three indicators in participation are divided into two clusters.
|
Participation: Indicators and Weights |
|
|
Indicator |
Weight |
|
Cluster 1: Young Adults |
60% |
|
High school freshmen enrolling in college within 4 years in any state |
40% |
|
18- to 24-year-olds enrolling in college |
20% |
|
Cluster 2: Working-Age Adults |
40% |
|
25- to 49-year-olds enrolled part-time in some type of postsecondary education |
40% |
Thomas Mortenson. “Chance for College by Age 19 by State in 1998.” Postsecondary Education Opportunity. No. 98, August 2000. http://www.postsecondary.org (accessed 11/5/01).
This indicator measures the probability that students will go on to college during the year immediately following an on-time high school graduation (that is, when most students are approximately age 19). To calculate this measure, the high school completion rate is multiplied by the college continuation rate. The following formulas describe the components of this calculation:
§ Numerator: Number of public high school graduates in 1998.
§ Denominator: Number of public school ninth graders in 1994.
§ Numerator: Number of college freshmen in 1998.
§ Denominator: Number of public high school graduates in 1998.
* Data are from National Center for Education Statistics, Washington, D.C.: U.S. Department of Education.
This
indicator adjusts for migration by using the NCES residence and migration
survey, which followed high school graduates to the institutions they chose to
attend. Since many students pursue their college education out-of-state, the
calculation related college freshmen by state of residency to the state data on
high school graduates.
This is a synthetic cohort statistic that cannot adjust for students’ out-of-state migration during the high school years. No nationally comparable longitudinal data exist that precisely measure the college-going rate of 9th grade students in each state.
Data are available for all 50 states.
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
This indicator reports the percentage of all 18- to 24-year-old high school graduates enrolled either full-time or part-time in postsecondary education. The measure uses the following calculation:
§ Numerator: Total population of adults ages 18 to 24 enrolled in grades 13 to 17 who have not yet attained baccalaureate degrees.
§ Denominator: Total population ages 18 to 24.
Students already holding a baccalaureate degree and returning for additional or different credentials are not included in this figure.
This indicator pools three years of the most current data, 1998 to 2000, to obtain a large enough sample size to make reliable state estimates and to account for aberrations in any single year of data. Data are available for all 50 states.
National Center for Education Statistics. “Fall Enrollment Survey, 1999–2000,” unpublished tabulation provided by Tom Snyder at NCES, U.S. Department of Education, Washington, D.C.
U.S. Bureau of the Census. Current Population Survey, October 1999 Supplement. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
This indicator measures the percentage of 25- to 49-year-old high school graduates enrolled part-time in an institution of higher education. The following calculation is used:
§ Numerator: Population of adults ages 25 to 49 with at least a high school credential who are enrolled part-time in an institution of higher education.
§ Denominator: Population of adults ages 25 to 49 with at least a high school credential.
This indicator focuses on part-time enrollment to assess the opportunities in each state to enroll part-time in postsecondary education. It includes both undergraduate- and graduate-level enrollments.
The indicator measures the 25- to 49-year-old adult population, which is slightly different from the 25- to 44-year-old population reported in Measuring Up 2000. This is due to differences in data collection methods that in 2000 did not permit an exact age-match between the numerator and the denominator. This problem has since been resolved, enabling us to measure exactly the same-age population in 2002. The slight age difference in the denominator in the 2002 report compared with the 2000 report does not generate significant changes to the final ratios.
Data are available for all 50 states.
Affordability of higher education is based on three concepts:
§ students’ capacity to pay for college, given the type of institution they attend, the financial aid they receive, and their income constraints;
§ the amount of need-based grant assistance they receive to offset expenses; and
§ the loan burden associated with their higher education expenses.
The six indicators included in this category combine data from a variety of sources. Together, they calculate a reasonable estimate of the actual amount that people in a state pay for higher education as well as the extent to which states employ policies to make college education more affordable.
No comprehensive, student-level,
comparable state data capturing price of attendance for higher education
currently exist. This category uses best estimates to assess the extent to which
college is affordable for residents of varying income levels in each state.
|
Affordability: Indicators and Weights |
|
|
Indicator |
Weight |
|
Cluster 1: Family ability to pay |
50% |
|
Percent of income
(average of all income groups) needed to pay |
Weighted by student enrollment in sector |
|
Percent of income
(average of all income groups) needed to pay |
Weighted by student enrollment in sector |
|
Percent of income
(average of all income groups) needed to pay |
Weighted by student enrollment in sector |
|
Cluster 2: Strategies for affordability |
40% |
|
State grant aid targeted to low-income families as a percent of federal Pell Grant aid to low-income families |
20% |
|
Share of income that poorest families need to pay for tuition at lowest-priced colleges |
20% |
|
Cluster 3: Reliance on loans |
10% |
|
Average loan amount that undergraduate students borrow each year |
10% |
National Center for Education Statistics. Digest of Education Statistics 2001. Washington, D.C.: U.S. Department of Education, 2002, p. 361.
Office of Postsecondary Education. Title IV/Pell Grant End of the Year Report, 2000–2001. Washington, D.C.: U.S. Department of Education, 2002, Table 21.
Sam Barbett and Roslyn Korb. Current Fund Revenues and Expenditures of Degree Granting Institutions, Fiscal Year 1996. Washington, D.C.: National Center for Education Statistics, U.S. Department of Education, 1999, Tables 10, 11, and 12.
U.S. Bureau of the Census. Current Population Survey, March 1999, 2000, and 2001 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
National Center for Education Statistics. National Postsecondary Student Aid Survey,
1999. Washington, D.C.: U.S. Department of Education. Data Analysis System
Variables: PELLAMT, INSTAMT, STATNEED,
STATNOND, TOTGRT By CINCOME. Filtered by institution level and control for
public two-year, public four-year, and private four-year institutions.
Kristen DeSalvatore. National Association of State Student Grant and Aid Programs 32nd Annual Survey, 2000–2001 Academic Survey Report. Albany, NY: National Association of State Student Grant and Aid Programs, 2002, Table 1.
National Center for Education Statistics. Digest of Education Statistics 2001. Washington, D.C.: U.S. Department of Education, 2002, p. 236.
Bureau of Labor Statistics. Consumer Price Index 1996–2001. Washington, D.C.: U.S. Department of Labor. http://www.bls.gov (accessed 5/13/02).
College affordability is based on institutional price, the adequacy of state effort to meet students’ financial need, and students’ personal or family income. The family ability-to-pay cluster of indicators examines the interaction of these important factors given (1) the variation in personal income that families of different means must pay to meet college costs, and (2) the variations in price across the public/private and two- and four-year sectors.
To assess state performance reliably and comparably, this indicator is based on a set of assumptions and a series of calculations that use a combination of national and state-level data. The first set of calculations determines the approximate net price of college attendance, taking into account federal, state, and institutional financial aid. The second set of calculations relates this net price to families’ annual income and takes into account the share of total enrollment at each of the major sectors in higher education: public two-year colleges, public four-year colleges and universities, and private four-year colleges and universities.
The average tuition and fees (for in-state residents) are calculated by state for each of the major sectors in higher education: public two-year colleges, public four-year colleges and universities, and private four-year colleges and universities. This calculation assumes average tuition and fees for each sector charged to the full-time student.
The federal government adds the price of housing, food and other necessary living expenses to tuition and fees when determining a student’s price of attendance at a particular institution. This indicator calculates average room and board fees by state and by type of institution. This calculation assumes that average cost of living expenses at public four-year colleges in the state are the same as incurred by students attending the state’s public two-year community colleges. This assumption is made in part to account for living expenses that must be paid by all students, regardless of whether they live on campus or not.
Average federal financial aid by state is calculated as the average Pell Grant per full-time equivalent enrollment (FTE), by state. Pell Grants are by far the largest component of federal grant aid.
States offer need-based and non-need-based financial aid for college attendance. Separate need-based and non-need-based awards are calculated per FTE.
Institutions offer scholarships, fellowships, and tuition discounts to support undergraduate college attendance. Average institutional financial aid by state is calculated per FTE.
Average financial aid awards mask the deliberateness of policies to target aid at different student populations. Without student unit records available at the state level to provide precise estimates of financial aid received, estimates must be calculated. These estimates are based on the distribution of financial aid received by students, nationally, in each income quintile. For each type of major financial aid (federal, state, institutional), the U.S. Department of Education’s National Postsecondary Student Aid Survey estimates the percent of total aid received by full-time equivalent students in each state. These national percentages are multiplied by the average federal, state and institutional awards described above. These calculations assume that students receive the same percentage of available aid in every state, but the actual amount of financial aid for students in each income quintile will vary by state because the size of the average award varies by sector and by state. These calculations do not take these state variations into account.
Net price in each sector
Average net price of attendance in the major sectors of higher education is calculated by subtracting total average financial aid (federal + state + institutional) from average expenses (tuition + fees + room + board). While expenses do not vary for students with different family incomes, average financial aid is different for each family income quintile in the state.
Paying for college is based both on the net price and the resources available to pay the price. By state, net price at each of the major sectors is calculated as a percentage of median family income in each quintile. The results of these calculations are estimates of the amount of family income required by low-income, middle-income and high-income families to attend college in each of the state’s major sectors.
To estimate affordability for all families in each sector, ability to pay is estimated for families in each income quintile. The average of these five estimates is used to determine the following:
§ average ability to pay for a technical or community college, all families in the state;
§ average ability to pay for a public four-year college or university, all families in the state; and
§ average ability to pay for a private four-year college or university, all families in the state.
These three measures are cumulatively worth 50% of the state’s affordability grade, but the weight assigned to each differs by the share of total full-time equivalent enrollment each sector in the state comprises. This final step ensures that college affordability is determined not only by the state’s efforts to make one sector affordable for all of its residents, but also by the state’s policies to make its most-utilized institutions affordable.
The most precise way to measure students’ ability to pay would be to analyze student-unit record data. While such records are available for national indicators of affordability, it is not possible to develop reliable and comparable indicators from these sources that attest to the level of affordability in each of the 50 states.
Comparable income data on the students enrolled in each sector are not available by state. As a result, this calculation measures the ability of all state residents to pay for college, regardless of whether or not they enroll in a postsecondary institution.
Beginning with Measuring Up 2002, the state financial aid data used to calculate the net price include only the aid awarded to undergraduates.
An unduplicated count of students who receive state or institutional financial aid awards is not available. As a result, the average award likely underestimates the mean value of awards any single student might actually receive.
It is the institution’s discretion to report tuition waivers and/or discounting in its estimation of total scholarships and fellowships, so the average institutional grant may underestimate actual aid received.
Data are available for all 50 states.
Office of Postsecondary Education. Title IV/Pell Grant End of the Year Report, 2000–2001. Washington, D.C.: U.S. Department of Education, 2002, Table 21.
Kristen DeSalvatore. National Association of State Student Grant and Aid Programs 32nd Annual Survey, 2000–2001 Academic Survey Report. Albany, NY: National Association of State Student Grant and Aid Programs, 2002, Table 1.
This indicator measures each state’s commitment to provide aid for low-income students, through the following formula:
§ Numerator: Total amount of state need-based aid awarded to students.
§ Denominator: Distribution of Pell Grant aid by state of residence of students.
Without having data to measure precisely the expected family contribution and amount of unmet need for students in each state, this indicator is a proxy measure for (1) how well the state targets aid to families with the greatest need, and (2) how much need-based aid is made available to all students.
It is assumed that the state’s methodology for awarding state need-based aid is similar enough to the federal methodology that the students awarded need-based aid in the state are the same students covered by the federal Pell Grant program. This may or may not be true in all cases. Due to data limitations, whether or not the two types of financial aid are actually going to the same students cannot be determined.
Beginning with Measuring Up 2002, the state grant data used include only the awards given to undergraduate students.
Data are available for all 50 states.
National Center for Education Statistics. Digest of Education Statistics 2001. Washington, D.C.: U.S. Department of Education, 2002, p. 361.
U.S. Bureau of the Census. Current Population Survey, March 1999, 2000, and 2001 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Tuition levels have been shown to affect whether low-income students choose to go to college. Overall tuition levels are an important part of the concept of affordability. This indicator measures this aspect of affordability with the following formula:
§ Numerator: The listed tuition and fees for full-time residents at the lowest-priced public institutions in the state.
§ Denominator: State median family income in the lowest-income quintile.
The lowest-priced tuition level reflects prices at community colleges. This indicator averages three years of family income data from the most current data available (1999 to 2001) to obtain a large enough sample size to make reliable state estimates and to account for aberrations in any single year of data.
Data are available for all 50 states.
National Center for Education Statistics. FFELP Report, AY 2000–01: Total Loan Guarantees for Undergraduates Only. Washington, D.C.: U.S. Department of Education, 2001.
National Center for Education Statistics. Direct Loans to Undergraduates, AY 2000–01: Number of Loans and Gross Commitments. Washington, D.C.: U.S. Department of Education, 2001.
Federal loans comprise more than 90% of the funds students borrow to attend college. As such, this indicator serves as a proxy for annual student loan burden. The following formula is used to calculate the average loan amount students receive from the federal government:
§ Numerator: (Total dollars in FFELP Stafford subsidized, unsubsidized, and PLUS loans made to parents in AY 2001) + (total dollars in William D. Ford Stafford subsidized, unsubsidized, and PLUS loans made to students in AY 2001).
§ Denominator: Total number of loans from both programs.
The methodology for this measure has been refined. Beginning with Measuring Up 2002, this indicator reports the average loan amount borrowed by undergraduate students only. However, data are still unavailable for the number of recipients by state. For this reason, the denominator used may report individual students who receive more than one loan, understating the total average loan amount.
Data are available for all 50 states.
The five indicators in the completion category are based on two overall concepts: persistence from the first to the second year of college, and completion of certificates and degrees in a timely manner.
|
Completion: Indicators and Weights |
|
|
Indicator |
Weight |
|
Cluster 1: Persistence |
20% |
|
1st year community college students returning their 2nd year |
10% |
|
Freshmen at 4-year colleges/universities returning their sophomore year |
10% |
|
Cluster 2: Completion |
80% |
|
First-time, full-time students completing a bachelor’s degree within 5 years of high school completion |
15% |
|
First-time, full-time students completing a bachelor’s degree within 6 years of college entrance |
15% |
|
Certificates, degrees, and diplomas awarded at all colleges and universities per 100 undergraduate students |
50% |
ACT. “Institutional Data Questionnaire, Fall 2000,” unpublished analysis conducted by ACT, Iowa City, Iowa, 2001.
Using data from the ACT national survey of 3,415 postsecondary institutions, this indicator calculates a weighted mean rate of first-to-second-year persistence for first-time, full-time students enrolled in a technical or community college. The weighted mean rate for states is based on the total first-time, full-time student enrollment of responding institutions.
Since part-time students are not included in the calculations, persistence rates for states with high part-time student enrollment may be overestimated. Furthermore, the data are reported at the institutional level and do not track student transfer. For this reason, the persistence rate may underestimate systemwide persistence if students transfer from one institution to another in the same state.
The most recent 2000 survey provides data for 41 states. Regarding the 9 states for which there are no data, they either: did not respond (Alaska), have a small sample size (Idaho, Nevada, Rhode Island, and Vermont), or have a large standard error (Montana, North Dakota, South Dakota, and Utah). Using the latest data available method, however, Nevada’s and Utah’s figures from the earlier survey are reported in Measuring Up 2002. Altogether, data are reported for 43 states.
ACT. “Institutional Data Questionnaire, Fall 2000,”
unpublished analysis conducted by ACT, Iowa City, Iowa, 2001.
Using data from the ACT national survey of 3,415 postsecondary institutions, this indicator calculates a weighted mean rate of first-to-second-year persistence for first-time, full-time students enrolled in a public or private four-year institution. The weighted mean rate for states is based on the total first-time, full-time student enrollment of responding institutions.
As with the measure of students returning at community colleges, part-time students are not included in the calculations. Therefore, persistence rates for states with high part-time student enrollment may be overestimated, and this measure may underestimate systemwide persistence if students transfer from one institution to another in the same state.
Data are available for all states except Alaska; Alaska’s estimate is not reliable due to the large standard error.
ACT. “Institutional Data Questionnaire, Fall 2000,” unpublished analysis conducted by ACT, Iowa City, Iowa, 2001.
Using data from the ACT national survey of 3,415 postsecondary institutions, this indicator measures the proportion of students in a public or private four-year institution who ultimately complete the baccalaureate degree within five years after high school at the same institution that they entered. The data represent the graduates who completed their degree during the 1999–2000 academic year. The reported rate is the weighted mean rate based on the total first-time, full-time student enrollment of responding institutions.
Part-time, transfer, and returning students are not captured in this measure.
The fall 2000 survey provided data for all 50 states. However, due to the large standard error, Delaware’s figure comes from the earlier survey used in Measuring Up 2000.
National Center for Educational Statistics. Graduation Rate Survey, 1998–99 (preliminary data, Peer Analysis System). Washington, D.C.: U.S. Department of Education. http://www.ed.gov (accessed 6/15/02).
Older adults and students working full-time make up a larger proportion of the college student body today, and more students now take longer to complete the baccalaureate degree. Considering such trends, this new measure is introduced to look at a prolonged time period within which students progress toward the bachelor’s degree. Using preliminary data from the NCES Graduation Rate Survey (GRS), this indicator measures the proportion of first-time, full-time students enrolled in a public or private four-year institution who obtain the bachelor’s degree at the institution they entered within six years of enrolling. The students captured in the data are those who entered in fall 1993 and graduated during the 1998–99 academic year.
Variables used from GRS and Institutional Characteristics Files:
§ Title IV eligibility: opeflag = 1
§ Four-year institutions: iclevel = 1
§ Participation in GRS: comp150 > 0
§ State summaries: adjcoh comp150 by stabbr
The calculation is based on the following procedure: The 1999 institutional characteristics file and the 1999 graduation rate file were downloaded from the on-line IPEDS Peer Analysis System and merged as an SPSS program file. From the resulting file, Title IV–eligible, degree-granting, four-year institutions that reported the number of students who completed within 150% of the allotted time are selected. The total number in the adjusted cohort and the total number of students who completed within 150% of the allotted time are then reported by state. The numbers reflect the sum of all students across institutions. The final graduation rate is calculated by dividing the number of students who completed within 150% of the allotted time by the total number of the adjusted cohort.
As with the five-year graduation rates, part-time, returning, and transfer students are not captured in this measure.
NCES states that the data from the Peer Analysis System should not be used for aggregate estimates. However, analysis by the National Center staff and review by the advisory group suggest that available data are sufficiently robust to make state-level estimates.
Data are available for all 50 states.
National Center for Education Statistics. Completion Survey, 1999–2000. Washington, D.C.: U.S. Department of Education, 2002. State-level data provided by Pinkerton Computer Consultants, 2002.
National Center for Education Statistics. Fall Enrollment Survey, 1999–2000. Washington, D.C.: U.S. Department of Education, 2002. State-level data provided by Pinkerton Computer Consultants, 2002.
This indicator uses the following calculation:
§ Numerator: Total number of associate’s degrees, baccalaureate degrees, certificates, and diplomas awarded throughout the 1999–2000 academic year.
§ Denominator: Part- and full-time undergraduate enrollment in fall 1999.
This measure is not a cohort statistic. However, since both associate’s and bachelor’s degrees are totaled, this indicator does capture the students transferring from one institution to another.
Data are available for all 50 states.
In exchange for its investment in higher education, each state expects to have a more productive workforce, a more informed electorate, and a better functioning society. In addition to these public benefits, each state expects that having more highly educated residents will lead to increased private benefits, such as higher lifetime earnings.
Specifically, this category considers the states’ investments in human capital in relation to four critical concepts:
§ Educational Achievement
§ Economic Benefits
§ Civic Benefits
§ Adult Skill Levels
Geographic and demographic factors that may impact the returns on a state’s education investment are not considered in this category, because of data limitations.
|
Benefits: Indicators and Weights |
|
|
Indicator |
Weight |
|
Cluster 1: Educational Achievement |
30% |
|
Population ages 25 to 65 with bachelor’s degree or higher |
30% |
|
Cluster 2: Economic Benefits |
25% |
|
Increase in total personal income as a result of the percentage of the population holding a bachelor’s degree |
15% |
|
Increase in total personal income as a result of the percentage of the population with some college (including an associate’s degree), but not a bachelor’s degree |
10% |
|
Cluster 3: Civic Benefits |
25% |
|
Residents voting in 1998 and 2000 national elections |
12.5% |
|
Of those who itemize on federal income taxes, the percentage declaring charitable gifts |
12.5% |
|
Cluster 4: Adult Skill Levels |
20% |
|
Adults demonstrating high-level quantitative literacy skills |
6.7% |
|
Adults demonstrating high-level prose literacy skills |
6.7% |
|
Adults demonstrating high-level document literacy skills |
6.6% |
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
This measure assesses the educational attainment of the working-age population, using the following calculation:
§ Numerator: Number of adults ages 25 to 65 with at least a baccalaureate degree.
§ Denominator: Number of adults ages 25 to 65 in the state’s population.
This indicator averages three years of the most current data, 1998 to 2000, to account for aberrations in any single year of data. This indicator does not control for interstate migration. State scores may be higher due to the number of bachelor’s degree holders who have migrated from other states.
Data are available for all 50 states.
U.S. Bureau of the Census. Current Population Survey, March 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Bureau of Economic Analysis. State Personal Income, Annual and Quarterly, for All States and Regions 1999. Washington, D.C.: U.S. Department of Commerce. http://www.bea.doc.gov/bea/regional/spi (accessed 4/1/02).
This indicator reflects the average net contribution of baccalaureate degree holders relative to total personal income. This indicator is measured through a three-step formula. First, this measure calculates the difference in the median earnings between adults whose highest level of education is a high school credential and adults with at least a baccalaureate degree. This earnings differential is then multiplied by the number of adults in the state with a baccalaureate degree. The third step divides this result by total personal income in the state. The following formula is used:
§ Numerator: [(Median earnings of population ages 25 to 65 with at least a baccalaureate degree) – (median earnings of population ages 25 to 65 whose highest education is a high school credential)] X (the number of adults ages 25 to 65 with at least a baccalaureate degree).
§ Denominator: Total personal income in the state.
Personal income is the sum of net earnings adjusted by place of residence, rental income of persons, personal dividend income, personal interest income, and transfer payments. It is measured before the deduction of personal income taxes and other personal taxes and is reported in current dollars (no adjustment is made for price changes). Total personal income is the personal income received by all residents of a state from participation in production, government and business transfer payments, and accumulated government interest.
Earnings of adults who are unemployed or not in the labor force but who have minimal annual earnings are included in the calculation of this measure.
For earnings and population, this indicator averages three years of the most current data, 1998 to 2000, in order to obtain a large enough sample size to make reliable state estimates and to account for aberrations in any single year of data. Data for each state’s total personal income represent a single year.
Data are available for all 50 states.
U.S. Bureau of the Census. Current Population Survey, March 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Bureau of Economic Analysis. State Personal Income, Annual and Quarterly, for All States and Regions 1999. Washington, D.C.: U.S. Department of Commerce. http://www.bea.doc.gov/bea/regional/spi (accessed 4/1/02).
Similar to the economic benefits attributed to the population with a bachelor’s degree, this new indicator approximates statewide income benefits associated with those whose highest education attained is higher than a high school credential, but lower than a bachelor’s degree. It includes those holding an associate’s degree or those who attended some type of postsecondary institution but did not obtain the baccalaureate degree. The same three-step calculation was applied.
First, the difference in the median earnings between adults whose highest level of education is a high school credential and adults with some college (including an associate’s but not a bachelor’s degree) is calculated. This earnings differential is then multiplied by the number of adults in the state with some college (including an associate’s but not a bachelor’s degree). The third step divides this result by total personal income in the state. The following formula is used:
§ Numerator: [(Median earnings of population ages 25 to 65 with some college or an associate’s degree) – (median earnings of population ages 25 to 65 whose highest education is a high school credential)] X (the number of adults ages 25 to 65 with some college or an associate’s degree).
§ Denominator: Total personal income in the state.
Personal income is the sum of net earnings adjusted by place of residence, rental income of persons, personal dividend income, personal interest income, and transfer payments. It is measured before the deduction of personal income taxes and other personal taxes and is reported in current dollars (no adjustment is made for price changes). Total personal income is the personal income received by all residents of a state from participation in production, government and business transfer payments, and accumulated government interest.
Earnings of adults who are unemployed or not in the labor force but who have minimal annual earnings are included in the calculation of this measure.
For earnings and population, this indicator averages three years of the most current data, 1998 to 2000, in order to obtain a large enough sample size to make reliable state estimates and to account for aberrations in any single year of data. Data for each state’s total personal income represent a single year.
Data are available for all 50 states.
U.S. Bureau of the Census. Current Population Survey, November Voting and Registration, 1998 & 2000. Washington, D.C. http://www.census.gov (accessed 6/14/02).
The following calculation is used:
§ Numerator: (Number of voters in November 1998 election) + (number of voters in 2000 election).
§ Denominator: (Voting population* in 1998) + (voting population in 2000).
* Voting population indicates state residents age 18 or above.
Votes cast in local, state, and federal races are included. Due to data limitations, this indicator does not disaggregate the voting rates of residents by level of educational attainment. National studies have shown that voting rates increase with higher levels of educational attainment. This measure is included as a proxy for the civic returns a state enjoys as a result of having a more highly educated population.
Data are available for all 50 states.
Internal Revenue Service. Statistics of Income for Tax Year 2000. Annual State Tax Reports. Washington, D.C.: U.S. Department of the Treasury. http://www.irs.ustreas.gov/pub/irs-soi/00in54cm.exe (accessed 5/6/02).
The charitable giving rate is the number of tax filers who made charitable contributions compared to all those who itemized their tax returns for the 2000 tax year.
§ Numerator: Number of tax filers itemizing charitable contributions on their 2000 federal tax return.
§ Denominator: Number of state residents filing an itemized federal tax return for 2000.
By monitoring the number of donors rather than the dollar amount donated, this indicator captures the prevalence of philanthropy among income earners and tax filers in the state. The number of donors in the state serves as a proxy for residents’ local and regional dollar commitments to public welfare.
Due to data limitations, this indicator does not disaggregate the charitable giving rates of residents by level of educational attainment. Annual analyses by the Washington, D.C.-based Independent Sector correlate income to volunteering and describe a direct relationship between educational attainment and charitable giving.
The indicator may favor states with wealthier populations because only those donations large enough to meet tax-deductible criteria are reported.
Data are available for all 50 states.
National Center for Education Statistics. National Adult Literacy Survey, 1992. Washington, D.C.: U.S. Department of Education, 1994. Collecting agency: Educational Testing Service. Special analysis for non-oversample states by Stephen Reder, Portland State University.
This indicator measures the percentage of the states’ populations whose literacy skills are most similar to the skills of college graduates (level 4 or 5 on a scale of 1 to 5 on the National Adult Literacy Survey, NALS).
Quantitative literacy measures the knowledge and skills required to apply arithmetic operations, either alone or sequentially, using numbers embedded in printed materials. Adults with the highest measured level of quantitative literacy, level 5, can perform multiple arithmetic operations sequentially, and can make inferences about the appropriate operation to perform without prompting from the text.
Due to data limitations, this indicator does not disaggregate the literacy rates of residents by level of educational attainment. Nevertheless, national studies have shown that literacy is attained through, and associated with, higher levels of educational attainment.
Eleven states participated in an oversample of the NALS: California, Illinois, Indiana, Iowa, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington. For some states not participating in the oversample, estimates could be calculated using multivariate regression techniques. Multivariate modeling relied on 1990 census data to predict literacy levels, given demographic and economic data. These results were compared with jackknife estimations of the NALS sample using the 60 replicate weights provided by NALS, and were found to be fairly predictive. As a result of these efforts, data are available for 28 states.
States for which data are unavailable are: Alabama, Alaska, Connecticut, Hawaii, Kentucky, Maine, Maryland, Minnesota, Mississippi, Montana, Nebraska, New Hampshire, New Mexico, North Dakota, Oregon, Rhode Island, South Carolina, South Dakota, Utah, Vermont, Virginia, and Wyoming.
National Center for Education Statistics. National Adult Literacy Survey, 1992. Washington, D.C.: U.S. Department of Education, 1994. Collecting agency: Educational Testing Service. Special analysis for non-oversample states by Stephen Reder, Portland State University.
This indicator measures the percentage of the states’ populations whose literacy skills are most similar to the skills of college graduates (level 4 or 5 on a scale of 1 to 5 on the National Adult Literacy Survey, NALS).
Prose literacy measures the knowledge and skills needed to understand and use information from texts that include editorials, news stories, poems, and fiction. Adults with the highest measured level of prose literacy, level 5, can find information in dense text with considerable distracting information that might seem plausible but is incorrect.
Due to data limitations, this indicator does not disaggregate the literacy rates of residents by level of educational attainment. Nevertheless, national studies have shown that literacy is attained through, and associated with, higher levels of educational attainment.
Eleven states participated in an oversample of NALS: California, Illinois, Indiana, Iowa, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington. For some states not participating in the oversample, estimates could be calculated using multivariate regression techniques. Multivariate modeling relied on 1990 census data to predict literacy levels, given demographic and economic data. These results were compared with jackknife estimations of the NALS sample using the 60 replicate weights provided by NALS, and were found to be fairly predictive. As a result of these efforts, data are available for 28 states.
States for which data are unavailable are: Alabama, Alaska, Connecticut, Hawaii, Kentucky, Maine, Maryland, Minnesota, Mississippi, Montana, Nebraska, New Hampshire, New Mexico, North Dakota, Oregon, Rhode Island, South Carolina, South Dakota, Utah, Vermont, Virginia, and Wyoming.
National Center for Education Statistics. National Adult Literacy Survey, 1992. Washington, D.C.: U.S. Department of Education, 1994. Collecting agency: Educational Testing Service. Special analysis for non-oversample states by Stephen Reder, Portland State University.
This indicator measures the percentage of the states’ populations whose literacy skills are most similar to the skills of college graduates (level 4 or 5 on a scale of 1 to 5 on the National Adult Literacy Survey, NALS).
Document literacy measures the knowledge and skills required to locate and use information contained in materials that include job applications, payroll forms, transportation schedules, maps, tables, and graphs. Adults with the highest measured level of document literacy, level 5, can use complex documents containing distracting information and make high-level inferences.
Due to data limitations, this indicator does not disaggregate the literacy rates of residents by level of educational attainment. Nevertheless, national studies have shown that literacy is attained through, and associated with, higher levels of educational attainment.
Eleven states participated in an oversample of NALS: California, Illinois, Indiana, Iowa, Louisiana, New Jersey, New York, Ohio, Pennsylvania, Texas, and Washington. For some states not participating in the oversample, estimates could be calculated using multivariate regression techniques. Multivariate modeling relied on 1990 census data to predict literacy levels, given demographic and economic data. These results were compared with jackknife estimations of the NALS sample using the 60 replicate weights provided by NALS, and were found to be fairly predictive. As a result of these efforts, data are available for 28 states.
States for which data are unavailable are: Alabama, Alaska, Connecticut, Hawaii, Kentucky, Maine, Maryland, Minnesota, Mississippi, Montana, Nebraska, New Hampshire, New Mexico, North Dakota, Oregon, Rhode Island, South Carolina, South Dakota, Utah, Vermont, Virginia, and Wyoming.
Assessing student learning in college is perhaps the most important criterion for measuring success in higher education. Despite assessment activities in many states, however, there are no common benchmarks for student learning that would allow meaningful state-to-state comparisons. Therefore, Measuring Up 2002, like Measuring Up 2000, gives all states an Incomplete in this performance category.
However, recent efforts to develop better measures of college-level learning are promising. To learn more about these efforts, see the following articles in Measuring Up 2002:
§ “Measuring Up and Student Learning,” by Margaret A. Miller (pp. 69–72).
§
“Grading Student Learning: You Have to Start
Somewhere,” by Peter T. Ewell
(pp. 73–76).
§ “Constructing Indicators: A Proposal for Discussion” (pp. 77–78).
§ “Kentucky: Sample Index Scores for Measures of Education Capital” (p. 79).
Performance gaps measure various gaps in performance—by gender, ethnicity, family income, and/or level of parental education—for indicators presented in Measuring Up 2002. The following list details the indicators and the sources for which data about performance gaps are available.
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Table 17: “Race/Ethnic Differences in Students Taking Chemistry and Algebra 2/Integrated Math 3.” Washington, D.C.: Council of Chief State School Officers, 2002, p. 49.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Table 17: “Race/Ethnic Differences in Students Taking Chemistry and Algebra 2/Integrated Math 3.” Washington, D.C.: Council of Chief State School Officers, 2002. p. 49.
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
National Center for Education Statistics. Completion Survey, 1999–2000. Washington, D.C.: U.S. Department of Education, 2002. State-level data provided by Pinkerton Computer Consultants, 2002.
National Center for Education Statistics. Fall Enrollment Survey, 1999–2000. Washington,
D.C.: U.S. Department of Education, 2002. State-level data provided by
Pinkerton Computer Consultants, 2002.
U.S. Bureau of the Census. Current Population Survey, October 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Net loss revenue measures the loss in annual earnings and annual tax revenues due to the earnings gaps between ethnic minorities and non-Hispanic whites who have equal levels of educational attainment. This loss of earnings is measured relative to total personal income in a state.
U.S. Bureau of the Census, Current Population Survey, March: Annual Demographic File, 1999–2001 [Computer file]. Washington, D.C.: 2001. Calculations by David W. Wright, Associate Professor of Sociology, Wichita State University, Wichita, Kansas. Due to small sample size, data are not available for Maine, New Hampshire, North Dakota, South Dakota, Vermont, and Wyoming.
Bureau of Economic Analysis. Annual State Personal Income, 2000. Washington, D.C.: U.S.
Department of Commerce.
http://www.bea.gov/
This information about change over time complements the graded categories by providing states with a means for measuring progress over 10 years. The long-term change data are available for the following five indicators:
U.S. Bureau of the Census. Current Population Survey, October 1988, 1989, 1990, 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Washington, D.C.: Council of Chief State School Officers, 2002, p. 43.
Rolf K. Blank and Doreen Langesen. State Indicators of Science and Mathematics Education 2001: State-by-State Trends and New Indicators from the 1999–2000 School Year. Washington, D.C.: Council of Chief State School Officers, 2002, p. 44.
U.S. Bureau of the Census. Current Population Survey, October 1988, 1989, 1990, 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
U.S. Bureau of the Census. Current Population Survey, October 1988, 1989, 1990, 1998, 1999, and 2000 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
U.S. Bureau of the Census. State Population Estimates, 2001. Time Series of State Population Estimates, Table ST-2001EST-01. Washington, D.C. http://www.census.gov/ (accessed 5/29/02).
Bureau of Economic Analysis. Gross State Product for States. Regional Accounts Data. Washington, D.C.: U.S. Department of Commerce, 2001. http://www.bea.gov/ (accessed 5/22/02).
U.S. Bureau of the Census. Population Projections for States, 1995–2025. Washington, D.C.: 1999.
Western Interstate Commission for Higher Education. Knocking at the College Door: Projections of High School Graduates by State and Race/Ethnicity, 1996–2012. Boulder, CO: 1998.
Harold Hovey. State Spending for Higher Education in the Next Decade: The Battle to Sustain Current Support. San Jose, CA: National Center for Public Policy and Higher Education, 1999.
U.S. Bureau of the Census. Current Population Survey, March 1999, 2000, and 2001 Supplements. Washington, D.C. State-level data provided by Pinkerton Computer Consultants, 2002.
Annie Casey Foundation. Kids
Count. Baltimore, MD: 2002. http://www.aecf.org/kidscount/c2ss/pdfs/entire_book.pdf (accessed
5/14/02).
U.S. Bureau of the Census. Current Population Survey, March 2000 Supplement. Washington, D.C. http://www.census.gov/population/socdemo/education/p20-536/tab13.pdf (accessed 7/19/02).
Progressive Policy Institute. The State New Economy Index. Washington, D.C.: 2002. http://www.neweconomyindex.org/states/2002/overall_rank.html (accessed
6/14/02).
National Center for Education Statistics. Digest of Education Statistics, 2001. Washington, D.C.: U.S. Department of Education, 2002. http://nces.ed.gov/pubs2002/digest2001/ (accessed 5/29/02).
General Educational Development Testing Service of the American Council on Education. Who Took the GED? GED 2001 Statistical Report. Washington, D.C.: American Council on Education, 2002.
Western Interstate Commission for Higher Education. Knocking at the College Door: Projections of High School Graduates by State and Race/Ethnicity, 1996–2012. Boulder, CO: 1998.
Center for Higher Education & Finance. Grapevine: A National Database of Tax Support for Higher Education. Normal, IL: Illinois State University. Per $1,000 personal income and per capita (FY 2001) data are from http://www.coe.ilstu.edu/grapevine/table10.html (accessed 5/28/02). Data for percentage change in state appropriations (FY 1992 to FY 2002) are from http://www.coe.ilstu.edu/grapevine/table3.html (accessed 5/28/02).
National Association of State Budget Officers, State Expenditure Report, 1990 and 2000 editions. Washington, D.C.: 1991, 2001.
U.S. Bureau of the Census. Population Estimates for States by Race and Hispanic Origin, July 1, 1999. Washington, D.C. Table ST-99-32. http://www.census.gov/ (accessed 5/29/02).
National Center for Education Statistics. Digest of Education Statistics, 2001. Washington, D.C.: U.S. Department of Education, 2002. Table 211. http://nces.ed.gov/pubs2002/digest2001/ (accessed 5/29/02).
The National Center for
Public Policy and Higher Education
The National Center for Public Policy and Higher
Education promotes public policies that enhance Americans’ opportunities to
pursue and achieve high-quality education and training beyond high school. As
an independent, nonprofit, nonpartisan organization, the National Center
prepares action-oriented analyses of pressing policy issues facing the states
and the nation regarding opportunity and achievement in higher
education—including two- and four-year, public and private, for-profit and
nonprofit institutions. The National Center communicates performance results
and key findings to the public, to civic, business, and higher education
leaders, and to state and federal leaders who are poised to improve higher
education policy.
Established
in 1998, the National Center is not affiliated with any institution of higher
education, with any political party, or with any government agency; it receives
continuing, core financial support from a consortium of national foundations
that includes The Pew Charitable Trusts, The Atlantic Philanthropies, and The
Ford Foundation.
152 North Third Street,
Suite 705, San Jose, California 95112
Telephone: 408-271-2699 •
FAX: 408-271-2697
National Center Publications
The National Center publishes:
« Reports and analyses commissioned by the National Center,
« Reports and analyses written by National Center staff,
« National Center Policy Reports that are approved by the National Center’s Board of Directors, and
« CrossTalk, a quarterly publication.
The following National Center publications—as well as a
host of other information and links—are available at www.highereducation.org. Single copies of most of these reports
are also available from the National Center. Please FAX requests to
408-271-2697 and ask for the report by publication number. Measuring Up 2000 and Measuring
Up 2002 are available by calling 888-269-3652.
Measuring
Up 2002: The State-by-State Report Card for Higher Education (October 2002, #02-7). This report card, which
updates the inaugural edition released in 2000, grades each state on its
performance in five key areas of higher education. Measuring Up 2002 also evaluates each state’s progress in relation
to its own results two years ago. Visit www.highereducation.org to
download Measuring Up 2002 or to make
your own comparisons of state performance in higher education. Printed copies
are available for $25.00 by calling 888-269-3652 (discounts available for large
orders).
Technical Guide Documenting Methodology, Indicators
and Data Sources for Measuring Up 2002 (October 2002, #02-8).
State
Policy and Community College–Baccalaureate Transfer, by Jane V. Wellman (July 2002, #02-6). Recommends
state policies to energize and improve higher education performance regarding
transfers from community colleges to four-year institutions.
Fund
for the Improvement of Postsecondary Education: The Early Years (June 2002, #02-5). The Fund for the Improvement of
Postsecondary Education (FIPSE) attained remarkable success in funding
innovative and enduring projects during its early years. This report, prepared
by FIPSE’s early program officers, elaborates on how those results were
achieved.
Losing
Ground: A National Status Report on the Affordability of American Higher
Education (May 2002, #02-3). This
national status report documents the declining affordability of higher
education for American families, and highlights public policies that support
affordable higher education. Provides state-by-state summaries as well as
national findings.
The Affordability of Higher Education: A Review of
Recent Survey Research, by John
Immerwahr (May 2002, #02-4). This review of recent surveys by Public Agenda
confirms that Americans feel that rising college prices threaten to make higher
education inaccessible for many people.
Coping
with Recession: Public Policy, Economic Downturns and Higher Education, by Patrick M. Callan (February 2002, #02-2).
Outlines the major policy considerations that states and institutions of higher
education face during economic downturns.
Competition
and Collaboration in California Higher Education, by Kathy Reeves Bracco and Patrick M. Callan
(January 2002, #02-1). Argues that the structure of California’s state higher
education system limits the system’s capacity for collaboration.
Measuring
Up 2000: The State-by-State Report Card for Higher Education (November 2000, #00-3). This first-of-its-kind report
card grades each state on its performance in higher education. The report card
also provides comprehensive profiles of each state and brief states-at-a-glance
comparisons. Visit www.highereducation.org to download
Measuring Up 2000 or to make your own comparisons of state performance in
higher education. Printed copies are available for $25.00 by calling
888-269-3652 (discounts available for large orders).
Beneath the Surface: A Statistical Analysis of the
Major Variables Associated with State Grades in Measuring Up
2000, by Alisa F. Cunningham and Jane
V. Wellman (November 2001, #01-4). Using statistical analysis, this report
explores the “drivers” that predict overall performance in Measuring Up 2000.
Supplementary Analysis for Measuring Up
2000: An Exploratory Report, by Mario Martinez (November 2001, #01-3). Explores
the relationships within and between the performance categories in Measuring Up 2000.
Some Next Steps for States: A Follow-up to Measuring Up
2000, by Dennis Jones and Karen
Paulson (June 2001, #01-2). Suggests a range of actions that states can take to
bridge the gap between state performance identified in Measuring Up 2000 and the formulation of effective policy to
improve performance in higher education.
A Review of Tests Performed on the Data in Measuring Up
2000, by Peter Ewell (June 2001,
#01-1). Describes the statistical testing performed on the data in Measuring Up 2000 by the National Center
for Higher Education Management Systems.
Recent State Policy Initiatives in Education: A
Supplement to Measuring Up 2000, by Aims McGuinness, Jr. (December 2000, #00-6). Highlights education
initiatives that states have adopted since 1997–98.
Assessing Student Learning Outcomes: A Supplement to Measuring Up
2000, by Peter Ewell and Paula Ries
(December 2000, #00-5). National survey of state efforts to assess student
learning outcomes in higher education.
Technical Guide Documenting Methodology, Indicators
and Data Sources for Measuring Up 2000 (November 2000, #00-4).
A State-by-State Report Card on Higher Education:
Prospectus (March 2000, #00-1).
Summarizes the goals of the National Center’s report card project.
Great
Expectations: How the Public and Parents—White, African American and
Hispanic—View Higher Education,
by John Immerwahr with Tony Foleno (May 2000, #00-2). This report by Public
Agenda finds that Americans overwhelmingly see higher education as essential
for success. Survey results are also available for the following states:
Great
Expectations: How Pennsylvanians View Higher Education (May 2000, #00-2b)
Great
Expectations: How Floridians View Higher Education (August 2000, #00-2c)
Great
Expectations: How Coloradans View Higher Education (August 2000, #00-2d)
Great Expectations:
How Californians View Higher Education
(August 2000, #00-2e)
Great
Expectations: How New Yorkers View Higher Education (October 2000, #00-2f)
Great
Expectations: How Illinois Residents View Higher Education (October 2000, #00-2h)
State
Spending for Higher Education in the Next Decade: The Battle to Sustain Current
Support, by Harold A. Hovey (July
1999, #99-3). This fiscal forecast of state and local spending patterns finds
that the vast majority of states will face significant fiscal deficits over the
next eight years, which will in turn lead to increased scrutiny of higher
education in almost all states, and to curtailed spending for public higher
education in many states.
South
Dakota: Developing Policy-Driven Change in Higher Education, by Mario Martinez (June 1999, #99-2). Describes the
processes for change in higher education that government, business, and higher
education leaders are creating and implementing in South Dakota.
Taking
Responsibility: Leaders’ Expectations of Higher Education, by John Immerwahr (January 1999, #99-1). Reports the
views of those most involved with decision-making about higher education, based
on a survey and focus groups conducted by Public Agenda.
The
Challenges and Opportunities Facing Higher Education: An Agenda for Policy
Research, by Dennis Jones, Peter
Ewell, and Aims McGuinness (December 1998, #98-8). Argues that due to
substantial changes in the landscape of postsecondary education, new
state-level policy frameworks must be developed and implemented.
Higher
Education Governance: Balancing Institutional and Market Influences, by Richard C. Richardson, Jr., Kathy Reeves Bracco,
Patrick M. Callan, and Joni E. Finney (November 1998, #98-7). Describes the
structural relationships that affect institutional effectiveness in higher
education, and argues that state policy should strive for a balance between
institutional and market forces.
Federal
Tuition Tax Credits and State Higher Education Policy: A Guide for State Policy
Makers, by Kristin D. Conklin
(December 1998, #98-6). Examines the implications of the federal income tax
provisions for students and their families, and makes recommendations for state
higher education policy.
The
Challenges Facing California Higher Education: A Memorandum to the Next Governor
of California, by David W.
Breneman (September 1998, #98-5). Argues that California should develop a new
Master Plan for Higher Education.
Tidal
Wave II Revisited: A Review of Earlier Enrollment Projections for California
Higher Education, by Gerald C.
Hayward, David W. Breneman, and Leobardo F. Estrada (September 1998, #98-4).
Finds that earlier forecasts of a surge in higher education enrollments were
accurate.
Organizing
for Learning: The View from the Governor’s Office, by James B. Hunt Jr., chair of the National Center
for Public Policy and Higher Education, and former governor of North Carolina
(June 1998, #98-3). An address to the American Association for Higher Education
concerning opportunity in higher education.
The
Price of Admission: The Growing Importance of Higher Education, by John Immerwahr (Spring 1998, #98-2). A national
survey of Americans’ views on higher education, conducted and reported by
Public Agenda.
Concept
Paper: A National Center to Address Higher Education Policy, by Patrick M. Callan (March 1998, #98-1). Describes
the purposes of the National Center for Public Policy and Higher Education.