Perspektiver på evidens Hanne Foss Hansen Institut for Statskundskab Københavns Universitet SVUF Stockholm, November 2013
Struktur Begrebet evidens Potentialer i evidens Udfordringer knyttet til spredning af evidens
Begrebet evidens Ikke entydigt Smal definition: Evidens = syntese of evalueringsresultater fra flere randomiserede kontrollerede forsøg (RCTs) Bred definition: Evidens = evalueringsresultater og dokumentation generelt
Begrebet evidens II Trods variationer i betydning tre gennemgående elementer: -information der er relevant i forhold til en given intervention, -information der er fyldestgørende, d.v.s. bekræftes af anden information, -information der er tilvejebragt via en sanddruelig (tillidsfuld) proces, upåvirket af interesser (upartisk)
Smal definition: Evidenshierarki 1. Randomized controlled trials (with concealed allocation) 2. Quasi-experimental studies (using matching) 3. Before-and-after comparisons 4. Cross-sectional, random sample studies 5. Process evaluation, formative studies and action research 6. Qualitative case study and etnographic research 7. Descriptive guides and examples of good practice 8. Professional and expert opinion 9. User opinion Ved smal definition inkluderes kun resultater fra studier der benytter design 1 og eventuelt 2 i den evidenbase, hvorfra der udarbejdes systematiske forskningsoversigter.
Eksempler på organisationer der arbejder på basis af smal definition Internationale organisationer
Bred definition: Typologi Impact evidence (effects of interventions) Implementation evidence (the process of carrying out an activitity) Attitudinal evidence (users, experts) Economic evidence (benefits versus costs) Ethical evidence Different sources: -Research-based evidence -Practice-based evidence
Eksempler på organisationer der arbejder på basis af bred definition UK: Institute of Education, University of London DK: Danish Clearinghouse for Educational Research, Aarhus Universitet
Metodiske potentialer: Behov for både smal og bred tilgang til såvel forskningsdesign som udarbejdelse af systematiske reviews RCT (What works?) Aim of evaluation Stop/go Longitudinal designs (e.g. before-after comparisons) Stop/go Adjustments Case studies (What works for whome in what circumstances?) Explanation Adjustments Intervention characteristics Simple, technical Well-defined Complex Contextual characteristics Low differentiation Moderate differentiation Highly differentiated Ex-ante knowledge about cause-effect relationsship Hypothesis about specific effects Can be modelled statistically Limited knowledge
Evidenspotentialer i praksis Informed decision-making in relation to interventions Assurance that citizens/users are offered best available treatement Winding up of interventions with limited or without effects --- In addition: Uncovering knowledge gaps -- Steadily keeping the knowledge base opdated
Udfordringer knyttet til spredning af evidens I The evidence movement seems to rest on the assumption that the logic of organisational behavior is cognitive, in other words the idea of organisations as learning identities But organisation theory tells us that organisations also are: -Cultures -Arenas for politics -Organized anarchies meaning that decision-making may be characterized by coincidences
Udfordringer knyttet til spredning af evidens II An opdated knowledgebase does not automatically lead to informed decision-making. We should not expect too much instrumental use. But instead expect -Entlightenment use: Indirect, changes thinking and maybe across time also behaviour -Inter-active use: Different streams of information, among these evidence are mixed in decision-making -Political use: Only knowledge supporting own position or undermining the position of the opposition is used
Barriers and bridges for dissemination of evidence In evidence producing organisations cultures and ressources may be barriers for userorientation. Evidence has to be passed on e.g. through policy briefs, educational activities, meetings and networks. In user organisations competencies, attitudes and ressources may be barriers for adoption of evidence. Knowledge adoption capacity has to be developed. Timing may be a challenge. The decision train may have departed when evidence is available.
Litteratur Hanne Foss Hansen & Olaf Rieper (2009): The Evidence Movement. Evaluation, 15 (2), 141-163. Hanne Foss Hansen & Olaf Rieper (2010): The Politics of Evidence-Based Policy- Making. The Case of Denmark. German Policy Studies, 6 (2), 87-112. Hanne Foss Hansen & Olaf Rieper (2010): Institutionalization of Second-order Evidence-Producing Organizations in O. Rieper; F. L. Leeuw & T. Ling (eds.): The Evidence Book. New Brunswick: Transaction Publishers. Hanne Foss Hansen (2013): Systemic evaluation governance. New logics in the development of organisational fields. Scandinavian Journal of Public Administration, 16 (3), 47-64. ------ Gough, D.; S. Oliver & J. Thomas (2012): An introduction to Systematic Reviews. London: Sage. Petticrew, M. & H. Roberts (2006): Systematic Reviews in the Social Sciences. A Practical Guide. Malden: Blackwell Publishing. Pawson, Ray: Evidence-based Policy (2006). A Realist Perspective. London: Sage.