Pre conference workshops
Six workshops have been scheduled for Monday, 23 October 2017 – three in the morning and three in the afternoon. You may choose to attend one workshop (either a morning or afternoon workshop) OR you may attend two workshops. Each workshop will include a tea break and lunch will be provided for all workshop attendees. Please register for the workshops as part of the conference registration process.
Workshop registration fees:
Single workshop (includes morning or afternoon tea and lunch)
Standard registration: $75
Student registration: $55
Full day, two workshops (includes morning and afternoon tea and lunch)
Standard registration: $100
Student registration: $80
Monday 23 October 2017, 9.00 – 12.30pm
1. Introduction to realist evaluation
Facilitated by Andrew Hawkins, ARTD Consultants, Australia
How do policies and programs work to generate change? What does it mean to take a realist approach? When should I be thinking about a realist evaluation? If you are asking any of these questions this workshop is for you. It is designed to orientate commissioners, conductors and consumers of evaluation to the effective use of realist approaches.
The workshop will begin with an exercise to reflect on what it means to say a program caused (or causes) an outcome. We will explore the philosophical underpinnings of realist approaches (including the idea that real causal mechanisms and structures produce actual events a subset of which is empirically observed) and the ‘generative’ understanding of causality employed by realists. The generativist approach will be contrasted with two other long standing modes of causal explanation: the ‘successionist’ (as employed by RCTs and experimental evaluation) and ‘configurationist’. Participants will discuss when, why and how a realist approach may, or may not, provide explanatory agility, external validity and reusable information.
Moving from the battlefield of ideas, the realist approach to program evaluation will be described using an exercise built around Pawson and Tilley’s question of ‘does CCTV work?’ We will focus on mechanisms, contexts and outcomes to illustrate the realist position that programs work by changing the way people feel and reason, their skills and abilities, or the context (nudging/ forcing/constraining) in which behaviour occurs. This leads directly to the realist mantra of ‘what works, for whom, under what circumstances, how and to what effect?’
Finally, we will address some common misconceptions and distinguish programs, interventions and resources from reasoning and mechanisms and how mechanisms only ‘fire’ in certain contexts. We will explore implications for more complicated and complex interventions and introduce Pawson’s VICTORE model for understanding complexity in programs.
2. Complexity and realist approaches
Facilitated by Gill Westhorp, Charles Darwin University, Australia
It is often suggested that realist approaches (both evaluation and literature review) are appropriate for ‘complex programs’. But what does it mean to say that programs are ‘complex’? And what are the implications of those aspects of complexity for evaluation and literature review?
Understandings of complexity are themselves contested. This workshop will compare two understandings of complexity – Pawson’s VICTORE model for explaining why programs are complex; and the basic principles of complexity theory.
Key ideas from the (limited) literature on the application of complexity theory in realist evaluation will also be explained.
Participants will be invited to work in small groups on the implications of particular elements of Pawson’s model, and particular principles of complexity theory, to develop understandings of their implications for ‘complex programs’ and practical strategies that might be applied to deal with them in realist evaluation and literature review.
By the end of the workshop, participants should be able to distinguish between ‘complicated’ and ‘complex’ structures and processes; identify how complexity affects the operations of programs; and describe the strengths and weaknesses of a range of strategies to address complexity in realist research and evaluation.
3. Applying realism to ethnographic research methods
Facilitated by Rebecca Hardwick, University of Exeter and Lisa Burrows, University of Plymouth, UK
Maximum of 15 participants
An ethnographic approach to research design, data collection and analysis enables the researcher to gain a ‘lived experience’ of the social phenomena being studied. It lends itself very well to a realist approach, which aims to understand how context influences the effectiveness of policies, programmes and interventions.
The first part of the workshop is an introduction to ethnographic research and how we have combined realist and ethnographic approaches in our work. This is followed by participants going out into the real world to do fieldwork on ‘café culture’ in nearby streets or within the conference centre, to practise their observation skills. When participants return, they will be shown how to use their observations in a realist way to develop programme theory that will start to uncover ‘how cafés work’.
No prior knowledge of ethnography needed, but some understanding of realism would be valuable for participants to get the most out of the workshop.
At the end of this session, participants should be able to:
- Explain what ethnography is and understand its place in social science
- Understand how a realist approach can be applied to ethnographic research
- Know how it can be a useful methodology (i.e. to answer what kind of questions)
- Name three ethnographic tools / methods (i.e. participant observation, interviewing, field notes)
- Understand the need of reflexivity, and one’s own attitude/opinion/view in this type of methodology
Monday 23 October 2017, 1.30 – 5.00pm
4. A realist approach to qualitative interviewing and analysis: Translating an interview regarding Indian Residential School experience into CMOCs
Facilitated by Tracey Carr, University of Saskatchewan, Canada
Maximum of 15 participants
While realist methodologies have progressed rapidly in the past decade, specific techniques on how to approach data collection and qualitative data analysis are less developed. This half-day workshop will provide participants with opportunities to: a) learn more about the relationship between realism and qualitative research, b) witness an example of qualitative realist interviewing, and c) take a realist approach to qualitative data analysis. The workshop provider, Dr. Tracey Carr, is part of a group of researchers, patients and policy makers led by Dr. Gary Groot who study shared decision making between Indigenous patients and families and health care providers at the University of Saskatchewan, Canada. The workshop will begin with an overview of the historical context of Indian Residential Schools (IRS) and other colonial government policies and their impacts on Indigenous health. Next, Dr. Groot will interview Mr. Gilbert Kewistep, a former IRS student and Cultural Advisor, regarding his residential school experience. Workshop participants will be asked to identify the mechanisms which have led to the current relationships between Indigenous people and their engagement with the health care system. Following the interview, Dr. Carr will lead a demonstration on how to extract and synthesise context, mechanisms, outcome, configurations (CMOCs) to either refine, refute or confirm program theory regarding how Indigenous people engage with the Canadian health care system. In the final portion of the workshop, attendees will have the opportunity to discuss and configure CMOs with the presenters.
5. Introduction to realist synthesis
Facilitated by Gill Westhorp, Charles Darwin University, Australia
Realist synthesis (also known as realist review) is a form of systematic review of existing literature and evidence. Grounded in realist philosophy, it was developed as an alternative to Cochrane-style (quantitative) and narrative style reviews.
This workshop will:
- explain the key differences between realist synthesis and other forms of systematic review;
- explain the key ideas in realist philosophy and their implications for review methods;
- introduce the RAMESES standards for realist synthesis and other resources to support researchers undertaking syntheses;
- provide practical guidance and examples for specific steps in realist synthesis, including developing appropriate questions for realist synthesis, eliciting program theory, literature searching, data extraction, analysis and writing up;
- provide opportunities for participants to workshop issues and questions.
6. Realist quantitative methods
Facilitated by John Eastwood, Sydney Local Health District, Australia
For many realist researchers, the use of quantitative methods is both a philosophical and methodological stumbling block. Yet quantitative methods can contribute to realist theory building and theory testing through their ability to use observable events to infer the existence of underlying mechanisms and to detect changes in both context and outcomes.
This workshop will examine debates about the use of quantitative methods in realist research and evaluation, either as part of a mixed-method design or on their own. Examples of their use for both theory development and theory testing will be provided. The workshop will provide practical demonstration of the use of quantitative methods for both small and large-scale studies. There will be opportunities for participants to apply the workshop content to their own research and evaluation projects.
Knowledge, techniques and practices to be addressed include:
- The philosophical debates concerning the use of quantitative methods for realist research and evaluation
- Quantitative methods used for explanatory theory building research
- Construction of quantitative tools from qualitative data
- Quantitative methods used for evaluation and theory testing
- Application of quantitative methods within realist mixed method study designs
- Practical application of quantitative analysis to infer mechanistic action
By the conclusion of the workshop, participants should be able to:
- Understand the philosophical rejection of quantitative methods for both natural and social scientific realms
- Determine how to interpret quantitative methods and incorporate them into realist research and evaluation while maintaining a distinction between method and methodology
- Describe quantitative methods for inclusion in realist mixed method research and/or evaluation study designs
- Undertake practical realist analysis of quantitative study data using simple statistical methods.
The workshop will include a combination of whole group discussions, presentation of examples from theory development and realist evaluations, and exercises focused on applying the key ideas to a demonstration dataset and the participants’ own research or evaluation work. Learning resources will include case studies and references for further reading.
Post conference workshop
Friday 27 October 2017, 9.00am – 12.30pm
Please register for this workshop as part of the conference registration process.
Integrating your learning
Maximum of 48 participants
Active reflection immediately after new learning supports the integration and use of the knowledge acquired. This half day session will provide an opportunity for you to reflect, in small groups and the whole group, about your learning from the conference; to identify any questions that have not yet been answered; and to consider the implications for your own research and evaluation work. The session is likely to be particularly useful for PhD students and their supervisors, who may use the time to identify what the lessons learned imply for study design; early career researchers, and those whose busy schedules when they return to work may interrupt active reflection and synthesis of new learning.
Workshop registration fee:
Standard registration: $60
Student registration: $40
Morning tea only will be included.