NACUBO

My NacuboWhy Join: Benefits of Membership

E-mail:   Password:   

 Remember Me? | Forgot password? | Need an online account?

Business Officer Magazine
Loading

Meaningful Measures

Performance measurement can be an effective internal tool and may help meet increasing external expectations for accountability.

By Brenda Norman Albright

The ABCs of External Expectations

Since the mid-1980s, almost all states have enacted legislation requiring formal performance reporting systems for public higher education institutions, and a few states have tied performance reporting to funding. Colleges and universities are under pressure to demonstrate that they are producing outcomes that are consistent with their missions, worth the cost, and responsive to state needs. Accountability is a hot topic with Congress as well. Institutions are increasingly working with policymakers to find innovative ways to respond to expectations through improved performance measurement and reporting.

Governors, legislators, and other lawmakers consistently view colleges and universities as pivotal to the future. When asked about their expectations from higher education, they identify accountability, better performance, and collaboration as key measurements.

To policymakers, accountability means showing results. Charles Miller, chairman of the U.S. Secretary of Education’s Commission on the Future of Education, recently said that he wanted to hold colleges accountable to taxpayers. He said that policymakers and parents need better information about how colleges are performing. Better performance connotes issues such as productivity, efficiency, and tuition and fee limits. Governing leaders want campuses to focus on collaborating with them to address state and local needs surrounding workforce training, economic development, and improved schools.

A New Reality

Performance measurement and reporting has changed significantly in the past two decades. Reporting now emphasizes strategy and educational outcomes. Colleges and universities are expected to add value for the individual and contribute to the economic and cultural well-being of communities and society through research and development, public services, and access and opportunities. The public expects institutions to improve the quality of life not only for students and alumni but for all members of society.

A second area of change is a focus on assessment. A recent State Higher Education Executive Officers/American Association of State Colleges and Universities survey identified several state- and system-level assessments of college-level learning:

  • Colorado has performance contracts that require institutions to develop a universal assessment of the value-added aspects of each institution’s general education curriculum.
  • Florida requires all students to pass an achievement test to receive an associate’s degree or be admitted to upper-level status.
  • Pennsylvania’s state universities administer an alumni survey on student satisfaction and level of preparation.
  • Tennessee uses assessment of general education as a component of its performance funding program, and has done so since the early 1980s.

A third area of change is accessibility of performance reports. Many institutions embrace performance reporting and make their reports available on their Web sites.

Effective Performance Measurement Principles

Although campuses and states have a variety of systems in place, all effective performance measurement systems include the following principles.  

Integrate with decision making. The overarching purpose of measurement is to enable an organization to perform better, so performance measurement needs to be integrated with decision making. It must support long-term objectives and specific actions and span multiyear planning and budgeting horizons. Strategic performance measurement also must influence budgetary allocations, including hard choices about reallocations and funding high-priority items.

Many institutions now integrate performance measurement with their strategic plans. Using key performance indicators in its multiyear strategic plan greatly benefited Lakeland Community College, Kirtland, Ohio, says President Morris Beverage. Each year, Lakeland sets aside funds for strategic initiatives and allows everyone—including students—to present proposals designed to reach the goals identified in the plan.

At the state level, Kentucky has adopted a public agenda for higher education that focuses on five questions:

  • Are more Kentuckians ready for postsecondary education?
  • Is Kentucky postsecondary education affordable to its citizens?
  • Do more Kentuckians have certificates and degrees?
  • Are college graduates prepared for life and work in Kentucky?
  • Are Kentucky’s people, communities, and economy benefiting?

A few performance measures have been identified for each area, and within the past few years, Kentucky has allocated substantial resources to higher education to achieve results.

Focus on a few desired results. The desired results need to be defined before the means of measuring performance can be determined. How is performance going to be defined? Who are the constituencies and what are the results for each of them? Is the institution effective in its core purposes of educating students and conducting research? Which goals in the strategic plan are most important?

Florida has used performance as a basis for distributing funds to community colleges since the legislature established a separate appropriation for this purpose in 1996-97. For FY05-06, more than $18 million was appropriated, which represents about 2 percent of the colleges’ operating budgets. Ed Cisek, vice chancellor for financial policy, Division of Community Colleges and Workforce Education, identifies two critical aspects of a successful performance funding program: The program must be simple to understand, and performance funding should be used as an incentive, not a punitive mechanism. Cisek says that allocating a portion of state funds based on performance is a strong incentive for institutions to improve.

The performance indicators used for Florida’s community colleges include:

  • efficiency (time to degree as measured by the number of graduates who completed their degrees with 72 hours or fewer),
  • success in passing college preparatory programs with a higher weighting for math,
  • graduates weighted by academic area,
  • graduates in a category of emphasis (e.g., economically disadvantaged), and
  • job placement at a certain salary level or transfer to a state university.

Lead to action. Strategic performance measurement should identify a well-coordinated set of action steps aimed at long-term improvement. Campus leaders need to know what performance is, why performance is at a particular level, and what changes are necessary, including resources needed to bring about improvement.

To illustrate, most colleges measure freshman retention rates and compare them with previous years’ rates and rates for similar institutions. If a college’s rate has declined or is lower than that of peer institutions, leadership should be provided with information that explains why the rate has declined or is low. An analysis may show that students have dropped out for financial reasons and that student financial aid expenditures are lower than at other institutions. The college may consider a strategy of allocating resources to student financial aid to improve retention and then determining the extent to which retention has improved. Simply knowing a retention rate does not adequately inform leadership.

In developing Lakeland’s strategic plan, Beverage says the institutional planning committee adopted a ground rule that there had to be a way to measure the results of each strategy. In many cases, the discussion on measurement helped to focus and improve the strategy.

Include well-selected and communicated measures and performance targets. All performance measures and targets must include a rationale and a purpose so individuals know what to do with the results. Measurement must be based on a careful analysis of the areas over which the campus has control. Only then can individuals take the right actions to improve the institution’s performance as a whole.

Selecting the measures themselves is an essential part of any performance measurement system and can be a formidable task. The quantity of output is not by itself an adequate measure of performance. The complexities of measuring the quality and quantity of student learning are enormous. Many benchmarks are outside of the institution, and collecting accurate and comparable performance data from other institutions, organizations, and states can be problematic. Some performance indicators are susceptible to manipulation without real changes in what they were intended to measure or reflect. Institutions must address other questions including: Is the indicator easily understandable? Against what benchmark or standard will the indicator be compared to chart success or progress? To what extent is the indicator reliable and valid?

According to Beverage, including performance measures in the strategic plan was a valuable communication tool for all campus constituencies and particularly for Lakeland’s board. He believes that the measurement and reporting processes keep everyone involved and working together.

Recognize limitations. All performance measurement systems have limitations. Results are long term, and significant changes in areas such as graduation rates do not occur overnight. Some measures are best used for internal purposes; using them externally to compare one institution to another can be inappropriate and damaging. 

Resources
NACUBO has developed several performance measurement resources, including Managerial Analysis and Decision Support web tools and other supporting information (www.nacubo.org/x5686.xml) as well as a Performance Measurement Toolkit.

For this reason, many higher education leaders express concerns about external performance reporting. Designing a performance reporting system that adequately reflects the mission and quality of multiple institutions can pose practical problems. It is difficult to recognize the distinctiveness of a major research university, a community college, and an open-admissions campus in the same performance system. College and university leaders fear that data required by external policymakers will be misused or used against them and will lead to being judged by inappropriate or unfair standards. Leaders are also concerned that they have limited control over educational outcomes and cannot be responsible for all contributing factors, such as the quality of education in the K-12 system. They recognize that performance measurement reporting is labor intensive and requires substantial institutional resources and question whether it is worth the effort.

A Timely Tool

Performance measurement has become an essential internal tool for college and university leaders. Various reporting mechanisms are used, including dashboards, balanced scorecards, key performance indicators, and internal benchmarking.

Does performance measurement make a difference? That is the ultimate test of a system’s effectiveness, regardless of its form. Having an internal tool in place will help in the event that performance expectations are imposed by external entities.

BRENDA NORMAN ALBRIGHT is a consultant with Franklin Education Group, Franklin, Tennessee.