Section 4: Cost Analysis

Conducting the Cost Analysis

• Identify All Ingredients And The Amounts Required

The ingredients method of cost analysis is based upon the notion that every activity can be described in terms of a recipe consisting of a list of the ingredients (resources) used and the amount of each ingredient required. This method is also called the resource cost model approach. This approach must begin with the evaluator completely specifying the program or programs for which cost data are to be determined. The basis for this should be a logic model that specifies all of the activities that must be conducted as part of the program whether or not these are conducted by the staff. The evaluator will then identify all of the ingredients that are required for each activity and the amounts of each ingredient that are used.

The ingredients method can be applied at any level of generality. An ingredients list can be developed for a program as a whole. In most cases, it will be more useful to divide the program into components (and perhaps even subdivide components into more specific activities) as was explained earlier. An ingredients list can then be developed for each component or activity. This allows the evaluator to determine the costs of specific activities within a program. Such information is helpful in making budget allocations, assessing program priorities, and explaining differences in costs between programs or program sites.

Worksheet 4.2 provides an example of an ingredients or resource cost model work sheet for a USAF-FAP family assistance program at the air base level. The ingredients used in the family support programs are divided into two major categories: personnel and nonpersonnel. The personnel ingredients listed include all who might contribute to the program. Whether or not they do contribute and how much they contribute may be determined from the logic model, interviews with program managers, or by actually collecting data on time use (discussed in more detail below). In this case, the work sheet provides spaces for recording the percentage of time (FTEs or full-time equivalents) devoted to each activity and for the associated costs of that time. For the nonpersonnel ingredients, the amounts of each can also be recorded on a form like this, although in this case the form is designed to allow the user to directly enter the total cost of each type of ingredient accounted for by each activity. How this is done is discussed below. Personnel, staff provided by another organization or program, and any volunteers and clients who contribute to the program should be regarded as costs whether or not they are in the budget. What is important at this stage is whether or not they are needed for the program, not who pays for their time. The same point applies to nonpersonnel ingredients that also may be donated or provided by another agency.

For most prevention programs, ingredients could be listed in each of the following categories: personnel, occupancy (facilities, utilities, communications), equipment, materials, and supplies. Examples of sub-categories in personnel might include: (1) paid direct service personnel, (2) paid supervisory and administrative personnel, and (3) volunteers. Within each of these Air Force personnel categories, personnel could be further divided into active duty military personnel, civil service employees, contracted employees, and others. The most detailed approach would be to list each individual within each category, perhaps according to their qualifications. Worksheets 4.2 and 4.3 illustrate the use of different approaches to categorization. Which is most useful probably depends on the organization's accounting system and the data used for unit costs of each type of personnel (see the next step in the process).

Unless the organization already has a cost analysis data collection system the evaluator will have to create one to go beyond simply listing the types of ingredients used and enumerating the amount of each type of ingredient. The first step in designing such a data collection system is to divide the total program into its component activities. Programs can be divided into at least five components: direct services to client families (children and adults), administration and management, staff training, public awareness or public relations, and program evaluation. The program can be assessed and this list of components modified as needed. The actual components for any program should be consistent with the intended use of the cost analysis. When the evaluator has a component break-out consistent with the project's structure and goals, the components should be compatible with the specific needs for cost information. The evaluator may also need to rearrange some of the components to capture costs associated with certain needs. The columns in Worksheet 4.2 provide one example.

• Estimating Personnel Costs

Personnel usually account for 60 to 80 percent of direct service delivery costs. Thus, when considering personnel cost data, the old adage "time is money" should kept in mind. Any system that seeks to allocate personnel costs across activities must capture the amount of time each staff person spends on various project components and activities. There are a number of alternatives available for estimating the amount of staff time required for each component or activity. One is simply to base this on the program logic model. Obviously, if programs depart from their intended design (the usual case, unfortunately) then this is not satisfactory. In some cases administrative data are available. Whether this is considered accurate or not will depend on why and how it is collected. (The accuracy of such data can be checked by comparing results with another method). The alternatives available for determining how time is allocated include: staff judgments, direct observation, surveys, and time logs, sometimes called time diaries. The most useful of these for anything other than rough estimates (though for some purposes rough estimates may be just fine) is the time log. Advantages and disadvantages of each of the others are briefly considered below.

The least costly approach to estimating time allocation to program activities is to ask the staff or administrators for their judgments. If little detail is required, this may work well. For example, if the evaluator is only interested in the total time use and cost of a program, then time logs are unnecessary. The program's budget will provide most of the needed information, and the time of any staff that was shared with other programs or the time of volunteers may be estimated close enough based on staff interviews or the managers' knowledge.

Direct observation has some clear advantages for obtaining highly accurate information about staff time allocation. It can be collected by an impartial observer who is not distracted by other tasks and who can focus on recording in detail each activity conducted by the staff person being observed. It does not rely on memory. It can be conducted through continuous shadowing or periodic sampling. However, the disadvantages are equally obvious. It is intrusive, and the presence of the observer may well change behavior. In order to code the activities in any useful way, the observer is likely to have to ask the person being observed what they are doing-otherwise they must rely on their own inferences about the purpose of an activity. This is especially likely if the observer is not a service provider in the same type of program, but would occur even in that instance (for example, asking "what was that phone call about?"). Finally, direct observation is an extremely expensive approach to collecting data on time allocation.

Surveys have the advantage of being relatively inexpensive. A questionnaire or interview can be used to ask staff how frequently and for what duration they engage in a list of activities. Sometimes data are collected by asking staff to estimate the percentage of a typical day's time that they allocate to each activity on a list. These approaches do not tend to yield accurate data (Juster and Stafford, 1985). The basic problem is that people are capable of providing accurate information on frequency and duration only for events that are narrowly defined and sharply focused. Moreover, staff responses tend to be shaped by how they believe they are supposed to spend their time.

Time logs or diaries obtain data by having staff record their activities throughout the day or with a short recall (at the end of the day or the next morning) or respond to an interviewer who asks them to recall the previous day. This is a variation on an approach developed for research on time use generally that has been shown to produce valid and reliable data when compared to other approaches (Juster and Stafford, 1985). For example, staff can fill out daily activity diaries on which they list each activity, the time it began and ended, where it took place, who was involved, and what its purpose was. Similar information is collected by recall diaries that can obtained by interview and these are unlikely to differ much in accuracy from logs filled out directly by the staff (Juster, 1986). The recall diary or time log for work activities begins with two questions. "On the previous day, what time did you arrive at work?" and "What was the first thing that you did?" After supplementary questions, this question is followed by "What did you do next?" This sequence is repeated until the end of the workday. The responses are recorded verbatim, and each of these main questions is accompanied by supplementary questions that can include: " where were you, who was with you, how did you get there, and, of course, what time did it end?" Predetermined time blocks are not used.
The component and activities break-out chosen for the cost analysis provides the basis for developing the system used to code the results of the time diaries. Alternatively, if staff are recording their own time, they can simply note the process by coding activities throughout a day. In either case, a list of activities developed for each program component is required, and the evaluator must assign a code for each separate activity. A codebook containing descriptions of the codes should accompany the time logs to assure that each staff person or interview coder uses the codes correctly. Note that an activity such as record keeping may occur as part of several components. If it is desirable to know what component an activity was conducted for codes are required for both activities and program components.

Whether time log sheets are to be completed by staff or diaries are to be collected by interviewers, the forms used to collect the data must be carefully designed. The log sheets should provide space to record the staff person's name, the date, the amount of time spent, the code for the activity, and the component under which the activity occurred.

How often time logs should be filled out or diaries collected and for how many days over what period of time depends on the intended use of the data, the resources available to devote to data collection, and how much activities vary from day to day and month to month. Although some organizations collect time logs every day year-round, this can be a burdensome task when considerable detail is required and may not be very useful. In many cases, a sample of days consisting of one randomly selected day per week for a few weeks might provide adequate information. In some cases, even a much smaller sample might provide important insights into staff time allocation. Beware of assuming that this is always the case, however. Especially if the evaluator is interested in activities that occur infrequently or irregularly, a larger sample of days and/or staff (depending on the number of staff that can be sampled) may be required. Note that the evaluator can increase the number of days sampled if preliminary data suggest that the sample will not be large enough to provide the desired statistical precision. (Issues of sample size and statistical precision should be discussed with a qualified statistician.) Often many programs vary their activities seasonally and are especially affected by major holidays. However, holidays and other "unusual" times should not be avoided if the goal is to obtain an accurate picture of year-round time use.

After data on staff time have been collected, they must be summarized in a form that allows for the calculation of costs. The personnel cost data form should include space for recording time in individual staff person days and individual staff time percentages for each activity, and total person days and percentages for each activity and component. How often a cost data system collects and summarizes time data depends on the needs for information on time use and costs. If the evaluator collects weekly time logs, then it probably makes sense to have an ongoing system that is updated and produces summaries weekly or monthly. This has several advantages. The summaries provide feedback that can be used by management and staff to manage and plan their time. Problems in the collection of time data can be identified and corrected before much incorrect information is collected. In addition, the task of summarizing the data is spread over time so that it is not too much of a burden at any one time.

• Nonpersonnel Costs

The most significant program ingredients in terms of their contributions to nonpersonnel costs are those related to facilities and transportation. These may also be the most complicated to calculate if they are shared by multiple programs and organizations. Facilities-related ingredients include real estate (buildings, playgrounds, parking), utilities, maintenance (not already included in personnel), and insurance. Transportation-related ingredients may include vehicles, fuel, maintenance (not already included in personnel), and insurance. Other common ingredients are office and classroom equipment and supplies, food, staff travel expenses, telephone, postage, fees for services from other agencies, and liability insurance.

After all nonpersonnel ingredients have been identified, the amounts required must be estimated, and if the analysis is looking at the costs of components or activities within a program they must be allocated to specific activities. Some nonpersonnel ingredients and their costs can be easily assigned to a particular program component or activity. Indeed, it may be that estimates of the total amount of some nonpersonnel ingredients are built up from the component or activity level.

It may be possible to keep track of nonpersonnel expenditures by program component with some bookkeeping adjustments. Often an appropriate code can be added to purchase requisitions, travel expense vouchers, and other routine paperwork that documents program expenditures. Of course, this activity is much easier to do as the expenditures occur, rather than weeks or months later when the original purposes may be less clear. Moreover, many organizations will not find this level of record keeping useful given its costs.

When the evaluator summarizes nonpersonnel costs, costs that can be clearly assigned to components should be dealt with first. The remaining costs will have to be prorated, as explained next.

• Prorated Costs

Some nonpersonnel costs are difficult to assign to a particular component or activity because these costs relate to all program components. These can include telephone, postage, general office supplies, utilities, and even the building space. These types of nonpersonnel costs can be prorated, that is, divided proportionately among various components and activities. But a definite procedure and rationale must be developed to determine the share of the costs to be assigned to each component. When prorating is done, it is sometimes most efficient to work directly from total cost figures for these ingredients obtained from budgets or expenditure reports rather than working with the amounts of resources per se (e.g., a percentage of the phone bill rather than numbers of phone calls).

Some nonpersonnel costs can be prorated based on the percentages of time that staff devote to particular components. For example, if time logs show that 30 percent of total staff time is devoted to delivering home-based services, 30 percent of telephone, postage, utilities, and so forth is assigned to that component. The rationale is that these types of nonpersonnel costs generally support staff in their work and, therefore, should be allocated in the same percentages as staff time. Following the prorating procedure, the cost analyst can calculate time percentages for each activity and list the percentages on a personnel time summary sheet. These percentages are then used to prorate nonpersonnel costs to each activity. Consider the following example: A program has three full-time staff who recorded total work days of 210, 230, and 215 days, and two half-time staff who recorded 105 and 115 days. The total for all staff for all activities is 875 work days. Three staff members developed and produced a self-assessment training guide for home visitors: one logged 10 days of work on the training guide; one logged 3 1/2 days; and one logged 1 day, a total of 14.5 days. Thus, the percent of staff time spent on this activity was 1.66 . If the total nonpersonnel cost to be prorated across all activities is $14,300, the amount prorated to the training guide activity (based on the staff time) would be $237.38 . As with assigned costs, the analyst may wish to prorate nonpersonnel costs to components rather than to each individual activity. The prorated costs can be listed by activity or component on the appropriate line on the cost summary form.

• Estimate The Costs Per Unit For Each Ingredient And Calculate Total Cost

Once all of the ingredients have been identified, it is necessary to identify or estimate the cost per unit of each type of ingredient (e.g., health screening and referral for one child, completed visit to a home). Multiplying the number of units of each ingredient used times the unit cost of each ingredient provides the total cost for each type of ingredient.

• Personnel Costs

In the case of personnel, the best estimate of full- and part-time employees is their total compensation; that is, all of the costs to the employer for employing them. This includes salary and fringe benefits (including social security; retirement; health, disability, and life insurance; and any allowances for uniforms or other living costs). Time allocation data can be summarized on forms (see Worksheet 4.2), and then the time data can be used to produce personnel cost data measured in dollars.

In some cases it is simply not possible to identify all unit costs, if some of these are included in budgets to which the program has no access. To the extent that an organization is concerned only with items that appear in its own budget, the problem is simplified.

As it is most often the case that the evaluator will be working with annual compensation data for staff, it is useful to convert data for each person's time for each activity into a percentage of a full-time equivalent (one FTE). This percentage figure is calculated by dividing the amount of time recorded for an activity by the total amount of time recorded by that person for all activities in the time period. This procedure varies slightly for part-time employees. For example, if a half-time employee recorded a total of 105 workdays, and 5 of those days were spent developing a program brochure, then 4.76 percent of the employee's time was spent on this activity. The annual personnel costs associated with each employee is calculated next. This is easy to do for staff who are paid by the program. The cost of each person is the sum of salary and fringe benefits. The program's financial administrator or bookkeeper can help to determine this figure. For example, the annual personnel costs for a certain full-time employee might be $20,989 (i.e., annual salary of $17,500 plus fringe benefits paid by the employer-the employer's share of social security, $1,314.25; health insurance, $480; workers' compensation, $84.50; and retirement plan, $1,610). Of course, actual figures will depend on the fringe benefits paid by each organization.

The inclusion of cost estimates for personnel not paid by the program will depend on the purpose of the cost analysis. If the concern is the cost of meeting a budget, then only those on the payroll should be included. When programs make extensive use of volunteers it may be useful to consider the costs and benefits of volunteering to various types of persons. One approach to the cost of volunteers is to estimate what they could earn if they were not volunteering. The minimum wage or the average wage plus benefits in the area may provide a ballpark figure. It is a mistake to assume that the cost of a person's time is zero because they are not employed outside the home or because it is time outside of work hours. The value of their time at home may be as high or higher than wages in the work place.

As noted at the beginning of Section 4, the key to estimating cost correctly is understanding that the cost of a resource is its "opportunity cost," which is the value of the lost opportunity of using it elsewhere. For most analyses, the best estimate of opportunity cost is what was paid for the resource, since most resources are bought in competitive markets. Of course, when the question addressed is how the time of existing staff should be allocated among alternative activities or programs, then the amount paid loses relevance. The real cost to the organization in that case is measured in time, a topic discussed above.

• Client Costs

The costs to clients from participating in a program can be extremely important and should be included in the cost analysis. The primary reason for including an estimate of costs to the client for participation in a prevention program is the potential that these costs have for affecting participation in the program activities.

Client costs include the costs of time to travel to program activities, to meet with program staff, and to conduct activities recommended by program staff. The approaches to measuring how much client time is required are the same as discussed for staff above, namely, informant estimates, direct observation, activity logs, and time diaries. The unique problem for clients is estimating the value of client time. One way is to estimate what the program would have to pay to hire the client. Another is to try to estimate the value of the activities that client gives up in order to participate. Both of these may be reasonably estimated by the client's compensation rate (salary plus fringe and other benefits) if the client is employed. Clients who are not formally employed also have costs. For example, mothers who are primarily home makers must give up a wide range of valuable activities in order to participate. All parents may incur additional costs for child care in order to participate in activities outside the home or to participate in the home when they have young children who require constant attention. A ballpark estimate can be obtained by using a cost per hour based on what the client would be likely to earn if they were employed in the formal labor market. Costs per hour of child care can be estimated from local or national data on rates, as appropriate. Out-of-pocket costs for travel can be estimated using standard organizational reimbursement rates.

• Nonpersonnel Costs

The unit costs of nonpersonnel ingredients are easily calculated when they are purchased annually or more frequently, and when they are rented. The costs of these ingredients are more difficult to calculate when they are purchased infrequently, are not directly paid for by the program, or are donated. These problems are frequently encountered with respect to facilities. For example, the land and buildings may have been paid for long ago, may be operated by a larger agency that does not charge the program for occupancy, may be estimated by some formula within the larger agency's budget (which can then be used to estimate these costs for the program such as a dollar figure per square for space used by providers within the prevention program), or may have been donated by a charitable organization. Similar problems may be also encountered with vehicles.

There are two alternative approaches that can be followed when these problems are encountered. One approach is to estimate the rental cost of substantially similar ingredients. For example, a program that occupies space in an office building, school, or church can determine the cost of renting space elsewhere that is equally accommodating. The other approach is to calculate interest and depreciation for the value of the ingredients. In most cases, this will be the more difficult alternative and it is not explained here. Those wishing more information about the second alternative are referred to Levin (1983) who explains this method and provides a table that reduces the need for calculations.

Whether it is important to include costs for donated nonpersonnel ingredients depends on the purpose of the cost analysis. Certainly, the full cost of the program to society is not represented unless these are included. Also, comparisons of program costs can be misleading if donated ingredients are not included. One strategy is to calculate the costs of donated ingredients and then include them, present them separately, or leave them out, depending on the report. For example, it is sometimes useful to present the value of donations and volunteers as an indication of a program's community support.

Comparisons also can be misleading if local variations in ingredient prices are not taken into account. Not only will donations be easier or more difficult to come by in some places than others, but wages, rents, utilities charges, and other costs vary from place to place. Thus, when comparing programs in different locations it may be necessary to adjust for differences in the level of prices using indicators of regional price variation (for example, the Consumer Price Index, or CPI, can be used). In addition, the relative prices of different ingredients may vary across locations. Because of these relative price variations, it is quite possible for a program to be the least costly model in one community but not in another. Thus, if the goal is to determine a program's cost in a new location, the best results will be obtained by applying the relevant local prices to the program's ingredients list and looking for ways to vary the ingredients to respond to local prices (including opportunities for donations and volunteers).

• Account For The Effects Of Time

Many prevention programs operate for years before they are evaluated and programs frequently provide more than a single year of service to families. Thus, the evaluator may find that the cost data to which there is access is more than a year or two old or is from multiple years. In such cases, two types of adjustments may be required to account for the effects of time- one for changes in prices and the other for changes in the opportunity cost of resources. Change in prices typically means that inflation reduces the value of a dollar over time. The effects of year-to-year price changes can be removed by converting dollars from different years into their equivalent "real" (constant dollar) values in a single year. There are a wide range of price indexes that can be used to do this. For instance, GDP and other deflators can be obtained from the Survey of Current Business.

Dollars in one year are converted to dollars in another year by dividing dollars in each year by their deflator and multiplying by the deflator for the target year. For example, 1997 dollars can be converted to 1998 dollars by dividing by the 1997 deflator (112.0) and multiplying by the 1998 deflator (113.3). As this example illustrates, in times of low inflation as has been experienced in the last few years, year-to-year differences in purchasing power are of minor importance. Over many years and in times of high inflation, the differences can be much more important.

In addition to adjusting for price change, it is sometimes necessary to account for the opportunity cost of resources over time. A dollar today is worth more than a dollar received next year, even after adjusting for inflation, because the dollar today can be used now. The interest rate that translates the real value of resources in one year to their real value the next is called the real discount rate. This discount rate can be used to convert real dollars from different years into a comparable present value. The use of a discount rate allows the analyst to recognize that costs that can be postponed into the future are less costly than equivalent dollar amounts that most be paid today. Unfortunately, economists do not agree on the discount rate that should be used, though many would agree that rates between 2% and 7% above inflation are usually satisfactory. For presentations of cost data to sophisticated and critical external audiences, a professional economist with experience in policy analyses should be consulted about the best approach to take.

Fortunately, the use of a discount rate is only an issue when estimating the cost of multi-year programs or when a cost analysis is expanded to look at later benefits. When the frame of reference for cost analysis is a single year or less, such efforts are unnecessary. Even when data from multiple years are used, but the program time frame is a single year (for example, the evaluator might have the cost of a year of services in 1995, 1996, 1997, and 1998), it is only necessary to adjust for inflation in order to compare and combine cost data across the years. If it is believed that the cost analysis the evaluator wishes to undertake requires discounting over time, then the evaluator should consult a more detailed guide to that kind of analysis such as Levin (1983).

• Analyze The Cost Data

Analyzing cost data means organizing it in a way that answers specific questions and addresses the specific needs of the audiences. How cost data is organized, then, depends on those questions and needs. This subsection provides some general suggestions for organizing cost data, whether these come from an ongoing data collection system or a one-time data collection effort. The types of analyses described here can be accomplished with a simple calculator. If a microcomputer is available, software packages for producing budget matrices ("spread sheets') are fairly simple to use and widely available.

• Analyzing Total Program Costs

A line-item budget is the most common way to organize cost data. Total program costs are divided into categories (line items), such as personnel costs, travel, supplies, equipment, contractual obligations, telephone, postage, and overhead. A more detailed understanding of total costs can be obtained from a line-item budget matrix. This can be developed for each specific program. When developing this matrix, the evaluator would break personnel and nonpersonnel costs down into the same degree of detail that has been used for all record keeping. A column on the far right of the matrix should display the traditional line-item costs. The total cost of each program component can be displayed across the bottom of the matrix. The columns in the middle of the matrix can show a breakdown of costs for each program component by line item. The Resource Cost Model Worksheet (Worksheet 4.2) is an example of a budget matrix, and illustrates how setting up worksheets to provide the kind of breakdown desired can facilitate analyses for program decision-making.

The budget matrix has several advantages. First, it displays an entire budget in detail on a single form. Second, because it is both a line-item budget and a program component budget, the user can see total program costs simultaneously from both perspectives. Third, the detailed breakdown of each program component and each line item allows the user to see how the subtotal for a given program component was calculated and the cost of a given line item across each component. Fourth, it helps compare costs by program component.

The budget matrix approach can be applied in several ways. For example, if a program has multiple sources of funding, the cost analyst could develop a separate budget matrix for each source of money. Alternatively, the same information could be summarized on a single page by displaying program components (or major activities under each component) down the left side of the page and sources of funds across the top of the page.

The budget matrix can help organize cost data, but it is not necessarily an effective way to report program costs to all audiences. The evaluator will not need or want to report all costs to every audience. The matrix is a general source of cost information from which the analyst can extract and report relevant data to various audiences.

• Other Approaches to Categorizing Costs

In the analysis of cost data, it is possible to categorize costs in ways that meet specific cost information needs. For example, if the purpose is to project the funds that will be required over time for a new program it may be important to separate ongoing costs from one-time or start-up costs that will only be incurred during the first year. (Note that this is not the kind of situation in which discounting is required because the evaluator is not interested in the present value of all costs in year 1, but in how much must be spent in each year). In a situation where cut-backs in spending are required it may be useful to separate costs essential to program operations from costs that can be eliminated without seriously jeopardizing the quality or quantity (number of clients served, hours of operation) of services. As a final example, it might be useful for planning purposes to identify costs that might be provided in the future as in-kind contributions or to identify costs that are donated now but might cease to be donated in the future.

• Average and Marginal Costs

When conducting cost analysis, it is helpful to understand the difference between average and marginal costs. Cost analysis is commonly used to determine the average costs of well-defined units of service, such as:

• Health screening and referral for one child
• One completed visit to a home
• A counseling or therapy session
• A year of complete prevention services

Average cost is equal to the total cost of all service units divided by the number of service units such as, cost per family, cost per case, or cost per therapy session. To calculate an average cost, the program activities that are required to produce the service under evaluation are first determined. The evaluator may want the costs for a day, a month, a year, or a variable period of time required to deliver one unit of service to each person. Second, an appropriate time frame is determined. Third, the total cost for all relevant activities within that time frame is calculated. Fourth, the total number of units produced within that time frame is determined. Finally, the total cost is divided by the number of units.

Analyzing average costs can help in internal budget planning and setting of fee schedules. Typically, average costs are used to make cost comparisons. For example, it might be desirable to compare the cost per client family or the cost per hour of contact time for center-based and home-based service programs.

Sometimes it is helpful to calculate marginal cost as well as average cost. Marginal cost is the addition to total cost that is required to produce one or more additional service unit. For example, it might be important to know what the marginal cost would be to increase the number of clients served per year or increase the number of home visits or counseling sessions delivered. For a program faced with a budget cuts, it might be important to know the marginal costs saved by reducing the number of clients served, by cutting out one particular type or aspect of service, or by reducing the frequency of service to each family. In some cases, average cost provides a reasonably close approximation to the correct answer, but in other cases it is not sufficiently close to the marginal cost to answer to these types of questions.

Consider, as an example, a home visitation program serving 60 families each month with 3 contact hours each per month. A small number of families might be added with very little increase in total cost, and the marginal cost might consist of the costs of overtime pay for the existing staff. Marginal cost could even consist of reduced time devoted to other activities such as paper work. Marginal cost could even be zero if the staff had slack time that was taken up (assuming that the quality of the service did not fall because of greater staff stress, lower morale, or less preparation and planning). A larger expansion, say an increase to 80 families might require the addition of another staff person. If this could be accomplished without increasing the needs for space, clerical help, and administration, then marginal cost could be higher than in the previous example, but still lower than average cost. However, if the size of the program was double to 120 families served per month then it could be that nearly all of the resources required by the program had to double. In this case, marginal cost would be quite close to average costs.

Of course, marginal cost is not always below average cost. For example, it might be that a program is operating at the limits of its administrative and space capacity. If just a few more clients are added, new administrative staff must be hired and more space purchased. This is especially likely to be the case when there are special resources that are available on a limited basis. For example, a manager with special interest and expertise in early intervention programming might launch a pilot program for high risk families using vacant office space and hiring a few new home visitors. If the program was judged a success and the evaluator wanted to know the cost of expanding the program, the pilot program's average cost would not be a good guide. Additional space would have to be found within the overall budgeted space. In addition, at some point, new administrators and support staff would have to be hired. A few elements of cost might fall (volume discounts on supplies and materials, for instance), but overall marginal cost would be higher than average costs.

The calculation of marginal cost is not as simple as for average cost. It is necessary to extrapolate from existing data and consider what would be different about resource requirements after the change contemplated. This is likely to be easy if only a small change is contemplated and more difficult if a large change that is very different from recent experience is contemplated. With this in mind, the steps are actually quite similar to those used to calculate average cost. First, the program activities and related resources that will have to change to produce the desired increase or decrease in services need to be determined. Second, the appropriate time frame is established. Third, the change in total cost for all the changes in activities required over that time frame is calculated. Fourth, the change in total cost (total marginal cost) is divided by the change in the number of units; this gives the marginal cost per unit of the anticipated change.

• Describe Distributional Consequences

Policies and programs can differ considerably in how their costs are distributed-that is, in who bears the costs. Obviously, organizations and people care a great deal about who bears the costs. For example, worksheet 4.3 illustrates a cost analysis in which costs are aggregated for an entire program (that is, not split out by activity or subprogram) and then disaggregated according to whomever pays. In this example, there are three categories: the organization under evaluation, other organizations, and clients. This kind of analysis can be especially useful when the organization is working in partnership with other organizations, when the organization is dependent on decisions made by others that affect the resources available for the program under evaluation, and when considering the impact of program decisions on client behavior. For example, the time costs imposed on clients for participation, as well as travel costs (does someone come to me or do I have to go somewhere and perhaps arrange child care), can significantly impact on client participation rates.

• Conduct Sensitivity Analysis

Cost analysis inevitably relies on assumptions as well as hard data. These assumptions may or may not hold true, so the degree to which the cost estimates depend on them is an important matter. Sensitivity analysis is used to identify critical assumptions and explore the effects of reasonable variations in the assumptions on the conclusions. Sometimes this means producing a range of cost estimates based on different assumptions about staff productivity, the costs of facilities, and other factors. Sometimes the implications of sensitivity analysis are that a program should only be implemented under certain circumstances. For example, if a family support program depends on referrals to supporting mental health and social services in producing its benefits, then it should not be implemented without the supporting programs. This does not mean that a home visiting program cannot be implemented in areas that do not have the supporting programs. However, these supporting programs must be introduced as part of the intervention and their costs taken into account. As another example, it might be that a program relies on local volunteers who are available because of a local university program that requires students to conduct an internship. Expanding this program to communities where similar volunteers were not available could well require hiring considerably more staff in those other communities.

• Implementing a Cost Data Collection System

If the cost analysis requires that the evaluator collect new data rather than use already available administrative data or data than can be obtained from a few phone calls, then the evaluator will have to develop and implement a "system" to collect the data needed. After selecting an approach that appears to meet the needs for a cost data collection system and after data collection instruments have been designed, it will be important to consider any steps that will need to be to followed to implement that design. These activities can help the evaluator avoid some early difficulties in implementing a cost-analysis system:

• Plan an initial orientation and training session for the staff, especially for those who will be collecting the data.
• Gather and summarize data for a trial period (1or 2 weeks) to see how well procedures and responsibilities are understood and if they work as planned.
• With information from the trial period, review and refine plans and procedures as needed.
• Share the results of the trial period, review and refine plans and procedures as needed.
• If the system operates well and requires only minor adjustments, begin full implementation; if not, conduct another trial period with the revised system.

Several multi-site prevention programs using home visitation in civilian contexts have added time recording to their administrative data collection systems. In these data collection systems, time involved in travel and the delivery of direct service units to families have been gathered with simple recording forms that can then be aggregated and used for reflection about practice patterns with specific types of families and resource allocation. However, these systems do not include the more burdensome record keeping that is involved with activities not directly involving client families such as supervision, team meetings, charting, and preparation for activities. These data collection systems have good concordance between the data collected in this format and information abstracted from clinical records. In addition, the programs using these systems have reported good acceptance by providers and managers from the programs. These systems could be easily adapted and used in conjunction with occasional time diary data collection that could capture the full scope of provider generated activities.

• Trial and Error

Sometimes decisions that seemed correct during the design stage do not work out in practice and must be changed. Often, activities are more time-consuming than anticipated, and rules for recording time and other costs may be confusing when put to use. A trial period for gathering and summarizing data allows program staff to learn how to use the system and can help solve problems before they affect the quality of cost data.

• Staff Support

Maintaining staff support for the entire cost analysis effort is crucial. Sometimes, after the trial period, staff will find their new data collection responsibilities tedious. Keeping time logs and coding nonpersonnel expenses can be burdensome and can appear to be of little value compared to other staff responsibilities. When staff neglect their record keeping responsibilities, the completeness and accuracy of the cost analysis data will suffer. Worse, staff morale may dip if they begin to resent the burdens of record keeping or if they believe it is being used as evidence to evaluate individuals' job performance. These negative possibilities must be avoided. People tend to support decisions and efforts that they help shape, so offer frequent opportunities for staff involvement in decision making, especially during the design stage and the trial implementation period. These opportunities should be in addition to the steering committee activities that tend to involve only a few staff representatives. The staff's commitment will be nurtured by participation in decision making.

Staff tend to support data collection when they understand how the data are being used and can see that the data are useful (and not harmful) to them. Periodic reports to the staff will show them how the data are used and can renew their belief that the effort is worthwhile. Time log data, for example, can show the staff where they direct their efforts, and individual staff members can analyze their own time log data to make personal work plan decisions. Avoid using any data from the cost analysis to assess staff performance. Rather, treat the data collection as a systematic way to reflect on practice patterns, involving staff within and across sites in the interpretation of the variation in practice patterns, and in making suggestions for changes in resource allocation that may result in practice patterns closer to what is specified in the program logic model.

It should be noted that data collection systems across large multi-site programs are especially sensitive to staff support issues. Staff will have to be allocated at each site to implement and maintain the data collection system. In addition, monitoring of sites will need to be accomplished by staff at headquarters who are responsible for quality assurance regarding data collection across the multiple sites. Usually, this can be done with a set schedule (e.g., monthly) for sending data to headquarters and with ready access to headquarter's staff for technical assistance requests received from program sites.

• Administrative Support

The cost analysis effort must receive administrative support throughout implementation, especially if data collection extends over a period of months or years. The data collection system may require regular access to administrative financial records or require modifications to bookkeeping procedures. In many cases, the evaluator may need the approval of administrative superiors to conduct a cost analysis. If the cost analysis involves a number of the organizations' programs, the support of the administrative staff of each program will be needed to collect the appropriate data. Administrative support can be maintained by using many of the same strategies used to hold staff support. Obtain their participation in the initial design and throughout implementation, and convince them that the system in question will provide information that they will find useful.

• Confidentiality

To implement the cost data collection system, the evaluator will need access to information of a somewhat confidential nature, such as salaries of individual staff members. Of course, accounting staff will already have access to this information, but care should be exercised in expanding that number and in determining how the information is used. Policies that establish appropriate and inappropriate uses of the data should be developed, and information that might require prior approval before it can be shared with anyone outside of the program needs to be identified.

• Leadership and Coordination

Someone on the staff can be designated as the data collection coordinator or task leader. The coordinator assures that data are gathered routinely, troubleshoots any problems, reports periodically to staff and administration, and is the first contact point for anyone with questions about the cost analysis effort. The coordinator needs the full support of the program director, other administrative officers, and the service and support staff.