Ethics in evaluation

Abstract

Evaluation involves at least four levels of social–political interaction – with government and other agency policy makers who commission evaluation; with participants in the programmes, policies and institutions evaluated; with the evaluation profession; and with the wider audiences to whom evaluators in a democratic society have a responsibility to report. Evaluation has to operate in this multilayered context of different interests, providing information to inform decisions while remaining independent of the policies and programmes themselves. In such a context it is not surprising that ethical dilemmas arise as to which is the best course of action to take.

Most evaluation and research texts where ethics are mentioned start with a reminder that there is no context-free abstract set of principles that can be applied to guide ethical decision-making in evaluation. What we encounter in the practice of evaluation are ethical dilemmas, where we have to make a complex judgment, a choice between alternative courses of action, taking into account a myriad of factors – social, personal, political, cultural – that are pertinent in the particular context (House, 1980; Lincoln, 1990; Mabry, 1999; Pring, 2000; Simons & Usher, 2000).

Take for example the following dilemma. A powerful institution, sponsor of a multicountry evaluation study, threatens to hold back payment to colleagues employed by them on projects which have been completed and independently evaluated, until the director of the evaluation agrees to the changes the institution wants made to the final evaluation report. The participants in the projects are dependent on release of the evaluation report for payment and for their livelihood. The institution wants the evaluation to blame certain people in the report. The evaluator argues for the fairness and balance of the report, as he sees it, to be maintained. How should the evaluator respond? Give in to the institution’s wishes so that the participants are paid, though it will compromise the integrity and impartiality of the evaluation? Or continue to negotiate for a reasonable outcome that will maintain fairness to individuals and how they are represented and ensure credibility to external audiences and international sponsor, though it will delay payment to participants and have consequences for them and their families? These are not the only options available in resolving the dilemma. They are merely illustrative of how the argument could run.

Or take the dilemma Morris (2004a, p. 236) raises as to whether or not it is appropriate to have a client representative on the steering committee of an implementation evaluation of an agency providing community-based residential services to the chronically mentally ill. The evaluator, who has experience working with community-based programs for the mentally ill, thinks it is appropriate to have such a client representative. The director of the agency does not, believing that the psychiatric disabilities of the agency’s clients would make it extremely difficult for them to be productive and that it might end up doing more harm than good.

Resolving dilemmas such as these is rarely a choice between right and wrong. This would be relatively easy to resolve. More often than not, it is a choice between mutually conflicting principles, what Russell (1993) has called a “clash between right and right” or House (1993) “trading one off against the other”. In such situations, principles are needed to guide action but they do not determine what is the “right” action to take. How then can the evaluator be assisted to conduct evaluations that are ethically defensible?

Evaluation textbooks on the whole offer little guidance on this issue. Newman & Brown (1996) surveyed 21 evaluation texts from 1972 to 1994 and found that 80% did not mention ethics at all and those that did averaged about four pages per book. A survey ten years on may reveal more attention to ethics, although the contemporary context for evaluation has led to procedures and mechanisms that frequently have more to do with control and surveillance than principles for ethical behaviour. In the last ten years we have witnessed an intensification of accountability and managerialism in institutions, an increase in governance and quality assurance of programmes, and a preoccupation with audit, monitoring, regulation, and control.

The rise of evaluation standard setting by evaluation associations and the intensification of ethical committees and institutional review boards (IRBs) within institutions may be one response to this external pressure for regulation by government and other agencies. However, it is important to distinguish institutional mechanisms for oversight of evaluation which have a regulatory function from the trend in the evaluation profession for setting ethical guidelines, principles, or standards. The latter are properly part of the self-regulation and self-accountability demanded of a profession with responsibility to its members, those they evaluate and the public. It is a separate question, however, to what extent ethics is a central feature of such guidelines or a subset of more general standards.

Ethics is about how we behave or should behave as individuals and as part of the society in which we live in interaction with others. It is, at one level, a personal morality, to do with the rights and wrongs of our actions, pervading our everyday life. Guiding precepts are honesty, sincerity, kindness, respect for persons, being honourable. “Do unto others as you would have them do unto you” is a golden rule, a categorical imperative that often guides our action and enables us to “know” we have done the right thing.

As members of a liberal democratic society we are also likely to share certain principles about how we should behave and how we should treat people in the conduct of our private and public lives. Such principles would include respect for persons, respect for “truth”, equality, justice, freedom and autonomy.

However, as evaluators and as a profession we also share principles about how we should conduct and disseminate evaluation. This is not just a question of our personal predilection or moral stance. It is a question of community – promoting and underpinning what it means to act ethically and promote “right” practice in the professional field of evaluation. This will incorporate public criteria, such as those noted above, and in practice involve recourse to the sources of our personal morality. But there is a professional ethic or ways of thinking ethically concerned with the work that evaluators do. It is this that is the main focus of this chapter.

More information

Identifiers

Catalogue record

Export record