Quite often, clients or prospective clients inquire about the advisability or utility of using internal (or external) program advisors. This can be a serious question that if not properly addressed, can result in a bad experience for all persons involved with the program. The answer to this question resides in the fundamental purpose of program evaluation: to assist in the decision-making process. Often conversations about evaluation become so embroiled in topics regarding formative versus summative processes, validity and reliability, evaluation models, statistical methods, etc., that we forget why we need an evaluation. Someone or some group needs to make a decision and determine a course of action.

Internal Evaluators

Many excellent program evaluators are not individuals who have been contracted; rather they work within the agency or organization. They can play an important role in decision making and in the organization’s own efforts toward accountability. Internal evaluators generally have better access to persons working in the program under review as well as records, history, and important artifacts that may not be available to a contractor. Since they observe the program in action, they also probably have a keener sense of how things really operate and are sensitive to the nuances of the organization. They may also have credibility and trust with program staff that may require extensive time for an outsider to build. Internal evaluators can reduce the costs of the evaluation (unless the process significantly interferes with their duties and regular assignments). If the internal evaluators are already paid from some other budget, they are obviously less expensive than an external evaluator. It should be remembered, however, that if the project is paying for everyone who works for it (like a grant-funded project), it will depend on what the internal evaluators are doing. The internal evaluators can be more expensive because they usually do a good deal more than an external evaluator

Even given all the above assets of an internal evaluator, it may not always be a prudent decision. Do they have the technical expertise? Can they be objective or does their role in the organization create some bias, predisposition, or even prejudice? Might they be influenced by others in the organization who desire a certain evaluation outcome? Are the internal politics such that a valid and fair evaluation is difficult, if not impossible? Will their findings have credibility, both within the organization and to external parties?

Owens¹ offers seven principles that should be accepted if an internal evaluation is to be effective:

1. Involve the staff as much as possible.
2. Form a group responsible for day-to-day management of the evaluation.
3. Strive for consensus on the evaluation plan.
4. Provide necessary resources to the evaluation team.
5. Encourage evaluators to report on their progress.
6. Use findings to reflect on the program aspects under review.
7. Develop a systematic plan by which changes can be made.

¹ Owens, J.M. (2007). Program evaluation. NY: The Guilford Press.

External Evaluators

External program evaluators often have assets including technical expertise and experience that are not available in the organization. They may also have colleagues or contacts that have additional expert skills and knowledge. Because they are not members of the agency, they bring a fresh, new perspective and a degree of independence that only outsiders may bring. They are also more immune to politics and political pressures from stakeholders who have a vested interest in the evaluation’s outcomes. Perhaps most importantly, the external evaluator will likely be free of bias; there is not just the potential objectivity of external program evaluators, but the perceived objectivity. The perceived objectivity may be more critical to sponsors that the reality of objectivity.

As outsiders, it will require time to build trust as well as to establish good communications; external evaluators may create some anxiety at first. It will also require time to understand the political powers that are involved. Finally, there are cost issues to consider. It is not unusual for an external evaluation to require as much as 10% of the program budget. Some funding agencies may require an external evaluation; then, of course, there is no option.

Evaluators, whether internal or external, must assume responsibility for establishing confidence between themselves and program stakeholders. Principles of professional ethics and honesty must never waver. Confidence is also derived from the degree of fairness, legitimacy, and objectivity of the program evaluation. These elements must never be compromised.

Suggested WWW Resources:

http://som.flinders.edu.au/FUSA/SACHRU/PEW/evalzone/whodo.htm

http://erc.msh.org/mainpage.cfm?file=2.3.1m.htm&module=info&language=English

http://www.uml.edu/centers/cfwc/Community_Tips/Program_Evaluation/Perform_Eval.html

http://ktl.jyu.fi/arkisto/verkkojulkaisuja/electronic/001/lyytinen.htm

http://www.businessincubation.com.au/Text/1056008145899-2011/pC/10563515

program evaluation program evaluation program evaluation grant grant grant