Section 7: Measuring Outcomes

What Should Be Measured?

A major challenge in studying families and evaluating family change is being able to capture the dynamics and complexity of family interaction. Formal assessments of families conducted in clinical or research environments often fall short of capturing the richness of family life. In addition, many questions exist on the best ways to go about assessing families. Should one focus on the couple and their relationship, on individual children, on all children, or on the ways that family adults socialize and nurture their offspring? If program impact on families is being evaluated, then the evaluator should consider whether the focus of program activities has been on proximal goals (e.g., improving family coping styles), or on distal goals (e.g., preventing child and spousal abuse). In the former case, the appropriate outcome measure would reflect dimensions of family coping. When family violence prevention (e.g., distal goal) is the primary target of program activities, the degree that reductions in child/spousal abuse can be measured reliably would then be considered.

It can be difficult to identify a single best measure that reflects what the program is trying to achieve. Unidimensional measures that assess changes in risk status as the sole measure of outcome are likely not sufficient measures of program effectiveness or family change. Rather, multiple outcome domains need to be assessed. For example, assessments that use multiple methods across three levels of family interaction, including marital, parent-child and the total family, are considered theoretically optimal (Hayden, Schiller, Dickstein, Seifer, Miller, Keitner, Sameroff, & Rasmussen, 1998).

Outcomes also need to be measurable and accurately reflect program goals (See, for example, Section 2, Subsection, "On Identifying Program Goals, Objectives And Outcomes"). Most programs assessing the impact of home visitation services, such as the Healthy Families America Network participants, use an array of standardized measures to capture the efficacy of programs. Some measures may be administered at intake into the program, and again at closure. Examples include demographics, risk assessment scores and measures of substance abuse and family violence. In general, measures used in family visitation programs tap dimensions of family functioning and support, and include the areas of parent and child physical health, parent mental health (depression), child development, parent and child interaction, and parenting stress.

Decision Making

When selecting measures for a particular program, the evaluator should select measures that mirror the program goals. Attention needs to focus on whether the measures tap key goals of the program that are expected to be altered.

Bottom Line: A single class on the dynamics of intimate violence, offered in the context of a one-time presentation such as a Parent University, may increase knowledge and may change attitudes. On the other hand, it may not be realistic to expect an enduring behavioral change (i.e., the cessation of violent behavior) after a limited intervention such as one class on the dynamics of domestic violence. It would, however, be appropriate to develop or identify measures that assess specific content (knowledge) areas addressed in a class.

For the most part, specific information is not available on the duration and intensity of intervention needed to detect change with a particular measure.

Bottom Line: The ability to detect change with any instrument will vary with the program model and goals, the outcome of concern, as well as the complexity of client needs.

The following points should also be kept in mind as measures for the evaluation are considered:

• Does the measure assess relevant areas of client functioning?
• Has the intervention been sufficiently intense to achieve the desired change?
• Is the measure regarded as scientifically valid and reliable?
• Is the measure appropriate to the population being assessed?
• Can the measure be completed in 15 minutes or less?
• Are directions for completion of the measure clear?
• Is the measure available in languages appropriate to the target population?
• Will the items included in the measure be viewed by the population as too sensitive?
• Can the measure be easily scored?

In conclusion, review of the literature might provide some guidelines about the sensitivity of the measure to detect change. However, the evaluator will need to consider if the circumstances of the studies and if the interventions parallel those of the program being evaluated and the target population. Ultimately, the evaluator will need to rely on clinical judgement and experience. Ideally, there should be a mechanism in place to pilot test new measures before they are introduced system-wide.