SCEA at AEA 2011: Presentations on Evaluation Theory
As evaluation practitioners, scholars, and students, we are often greeted by questions about the relationship between evaluation theory and practice. Our experiences suggest that development and advancement of these areas inevitably go hand-in-hand. In today’s post, we draw your attention to paper presentations about evaluation theory specifically. Those interested in this particular area might find the following presentations by SCEA members and their colleagues about theory, theory-building, validity, and values/valuing relevant and interesting.
For presentation time and location, please see this year’s AEA Conference Program.
|Values & Validity
ABSTRACT: A panel at last year’s conference featured contemporary responses to Ernie House’s classic Evaluating with Validity. House’s original discussion and last year’s panel focused on what priority should be emphasized. This paper extends that discussion by analyzing the values inherent in validity and in preferences for truth, beauty, and justice. What values are we accepting, rejecting, or balancing when we choose between truth, justice, and beauty? Similarly, what values are inherent in refusing to make the choice? In our analysis, we draw not only from House’s writings and other evaluation theory classics, but also from more contemporary ideas, such as multicultural validity, cultural competence, and praxis. An interesting feature of this discussion is that unlike other theoretical discussions about evaluation, theory and practice are intertwined here. This discussion centers on the collision of theory with the constraints of reality, hence the evaluator’s concern about choosing when values compete.
|Mixed Model Theory Development: Building a Theory-informed and Practice-informed Model of Evaluation
ABSTRACT: Understanding the application of evaluation approaches helps inform our understanding of evaluation theories. Exploring this interaction of practice and theory is at the heart of research on evaluation. Because evaluations are much like the programs we evaluate, where our activities are expected to lead to outcomes, using a program theory-driven evaluation technique of modeling evaluation practice should increase our understanding of our approaches. This presentation extends previous evaluation theory modeling by describing a process that engages evaluators in modeling their practice with theoretically derived variables. In this method, evaluators model their preferred practice in an online modeling software and the produced models are combined to create one representative model that evaluators review and comment on, improving the model’s reflection of practice. The final product is a theory- and practice-informed picture that might then be analyzed and tested for comprehensiveness and consistency in practice.
|Theory Building Through Praxis Discourse: A Theory- And Practice-Informed Model Of A Transformative Participatory Approach To Evaluation
ABSTRACT: The discipline of evaluation is built upon its descriptions, its language, and its approaches towards practice. The development of one such approach – Participatory Evaluation – has focused almost entirely on a practical model, leaving a transformative model mostly neglected. Transformative Participatory Evaluation (T-PE) practitioners find little theoretical or practical guidance in the evaluation literature. This paper redresses this discrepancy by presenting a theory- and practice-informed model of T-PE that will help further discipline development by providing practitioners a model reflecting their work. LeBaron Wallace, Hansen, and Alkin (2009) used qualitative literature analysis to develop logic models reflecting Practical Participatory Evaluation and Transformative Evaluation but overlooked T-PE. The present research extends that work by developing a model of T-PE using both quantitative and qualitative methods that inform one another. The product is a theory- and practice-informed model of a transformative participatory approach to evaluation practice.
|The Evaluator’s Role in Valuing: Who and with Whom
ABSTRACT: Evaluation scholars and practitioners have dedicated much energy and effort to shaping and defining the program evaluation profession. However, careful examination of the program evaluation literature turns up only a few resources that describe and operationalize value judgments, the ways in which they are reached, and who is involved in this aspect of the evaluation process. We argue in this paper that the valuing act may be perceived in many different ways and consider the multiple theoretic perspectives that govern an evaluator’s behavior. Based on this analysis, we develop a typology of evaluator valuing roles and suggest that value judgments are typically reached by stakeholders alone, stakeholders and evaluators in consort with each other, or by evaluators only. This heuristic helps us to gain a more explicit understanding of the valuing act and process as it occurs in the context of an evaluation.