Section 3: Doing Process Evaluation

Key Points & Introduction

This section describes process evaluation. Process evaluation is a method for determining whether a program is being implemented as planned and is staying "on course" over time. Some aspects of process evaluation, such as monitoring program retention, are more relevant to programs in which clients have several contacts with program staff. Other aspects, like recruitment of the target population, are relevant for all types of programs, even those that are targeting whole communities instead of individual clients. Ultimately, process evaluation is a way of collecting and applying feedback to increase the quality and effectiveness of the program and deal with problems as they arise.

Process evaluation can also help explain the results of an outcome evaluation. A very successful program might call for replication, and, therefore, the evaluator would need to know precisely which components are making the difference. If the program is producing the desired effects, process evaluation can help to determine how these effects are achieved. If a demonstration project failed to show any benefit of the program--or worse yet, if maladaptive behaviors (child or substance abuse) increased-the evaluator would want to understand what could be contributing to these results.

Quality Initiatives and Process Evaluation

Recommendations for quality initiatives in child and family services suggest establishing benchmarks as a means to achieve and maintain program excellence. As derived from Japanese and American industrial models, benchmarking has been defined as an ongoing process of measuring products, services, and practices against those recognized as representing the best. Process evaluation is a major component of quality assurance and quality initiatives (see also, Pecora, Seelig, Zirps & Davis, 1996). Pecora and colleagues (1995) explicitly link process evaluation to quality of the program. Specifically, they indicate that program evaluation asks three key questions.

• Is the program reaching the appropriate target population?
• Are services delivered in a consistent way?
• How much does the program cost per recipient?

Section 3 focuses on two aspects of process evaluation: monitoring the target population and program activities. In many programs, service delivery and outcome evaluation are marred if there is bias, either within the target population or in the way that the program is delivered. This section is designed to anticipate and address these problems.