Compared to What? Using Comparative Data to Improve Services

Thoughtful comparisons with other jurisdictions provide a key to effective performance management.

Jun 15, 2015 | ARTICLE

You’ve initiated performance measurement in your local government organization. You’ve identified key service areas that are critical to your mission. You’ve identified and defined the key data that indicate how effectively and efficiently those services are being delivered. And you’ve collected and recorded the data.

So how are you doing—and how do you know?

Over time, of course, one way is to compare your current data against past results to track changes (hopefully improvements) in performance. Another important way is to compare your organization’s performance with that of other jurisdictions.

Comparison with others provides an external perspective on your performance and can be the basis for establishing performance benchmarks. Perhaps the greatest value, however, is the opportunity to identify and adopt management practices that have been employed in cities and counties with high levels of performance.

Comparing Apples to Apples

Elected officials often want to compare to “the town next door,” so managers ask those jurisdictions for comparable data, often informally. The answers they get might be based on whatever numbers those jurisdictions have handy—budgeted, projected, year-end, pre/post audit, or something else. What is likely to be missing is comparability, caused by one or more of the following:

  • Different definitions
  • Different data sources
  • Different collection methods
  • Different demographic and geographical characteristics.

For example, seemingly comparable activities, such as DUI citations, may yield different results. Some jurisdictions cite driving under the influence, some cite operating a vehicle while intoxicated (OWI) or use some other acronym. In some cases, the definitions may be based on different blood-alcohol concentrations, or even distracted driving, driving while tired, texting behind the wheel. And the citations can be affected by the number, timing, and location of checkpoints.

Similarly, emergency response time is affected by station location, geographic size of the service area, topography, population density, and vehicular traffic. And obviously, street and road maintenance measures are affected by amount of snowfall, staffing levels, underlying pavement condition and materials, and other factors.

Finally, while it can be tempting to think that only jurisdictions in your county or state or of your population range might be comparable, the more relevant criteria might be more intrinsic:

  • Is the level of unionization of the comparison jurisdictions a factor?
  • Are there overriding community characteristics like regional malls, military bases, universities, or beaches that might steer you to a more dispersed network of comparison jurisdictions?
  • Will that list of comparables vary by department? For instance, which jurisdictions use a similar mix of full-time staff, contractors, and/or volunteers to perform a given service?

So when comparing with other jurisdictions, the first steps are: (1) define your terms; (2) weigh the impacts of policy or statutory differences; (3) consider other differences that may affect the measures you’re reviewing; and (4) think outside the box on identifying your peers

Setting Benchmarks

Once you’ve established comparability with another jurisdiction, the easiest approach to setting a benchmark might be to say that whatever level of performance the comparison jurisdiction has achieved is the one you should achieve, too.

For some services, that may be realistic. For example, internal services such as information technology, human resources, or procurement may not be affected by many of the same factors as other services—as long as your definitions are consistent.

For other services, however, that kind of simple benchmarking may be unrealistic. For example, if you’re comparing the tax revenues per capita of your bedroom community with those of a jurisdiction that hosts a major regional shopping center, you may find that you can never match its level of performance. In a case like that, it may be better to note trends in the data and set a target that is a percentage of that performance (e.g., if historical trends indicate that your collections are 20 percent of theirs, you might set a goal of achieving 25 or 30 percent of their revenues).

Beyond that, you may want to select a group of comparison jurisdictions, either in your state or elsewhere, and compare to the median performance of that group. The comparison group may be similar in population, demographics, service delivery characteristics, and/or other criteria.

Improving Management

Reviewing the performance data of comparable jurisdictions can suggest management improvements. What have your comparison jurisdictions done to achieve high levels of performance on key measures?

  • Has the adoption of new software helped streamline emergency dispatch and improve response times?
  • Has the launch of a citizen satisfaction survey helped pinpoint areas in greatest need of measurement and improvement?
  • Has the implementation of an employee wellness program reduced average sick leave days per FTE?
  • If their procurement process is more decentralized or electronic, has this helped them achieve greater efficiencies? Or has it led to more problems in accountability and oversight?
  • Are common types of property inspections consolidated in fewer site visits via scheduling improvements or cross-training?

Most local government managers are willing—even eager—to share management practices that have led to improvements in performance. These successes are shared through one-on-one conversations, programs at state-level meetings, sessions at the ICMA Annual Conference, in publications, and in the Knowledge Network online forum housed on the ICMA website.

The ultimate goal of performance measurement is not simply data collection, but performance management, and thoughtful comparisons with other jurisdictions provide the key.

Based on sections in Getting Started: A Performance Measurement Handbook by Gerald Young, published by ICMA and available in epub or mobi format from the ICMA Online Bookstore.

Want to Hear More About Performance Management? Browse the Performance Management topic and add it to your interests.

Advertisement

You may also be interested in