P
ATHFINDER INTERNATIONAL TOOL SERIES
Monitoring and Evaluation – 1
PREPARING A CASE STUDY:
A Guide for Designing and
Conducting a Case Study for
Evaluation Input
By
Palena Neale, PhD,
Senior Evaluation Associate
Shyam Thapa, PhD,
Senior Monitoring and Evaluation Advisor
Carolyn Boyce, MA,
Evaluation Associate
May 2006
P
ATHFINDER INTERNATIONAL TOOL SERIES
Monitoring and Evaluation – 1
PREPARING A CASE STUDY:
A Guide for Designing and
Conducting a Case Study for
Evaluation Input
By
Palena Neale, PhD,
Senior Evaluation Associate
Shyam Thapa, PhD,
Senior Monitoring and Evaluation Advisor
Carolyn Boyce, MA,
Evaluation Associate
May 2006
Acknowledgements
The authors would like to thank the following Pathfinder employees and partners for their
technical inputs into this document: Emmanuel Boadi (Pathfinder/Ghana), Anne Palmer
(Futures Group International), Ugo Daniels (African Youth Alliance (AYA)), Veronique
Dupont (Pathfinder/Extending Service Delivery (ESD)), Cathy Solter, Lauren Dunnington,
and Shannon Pryor (Pathfinder headquarters). Jenny Wilder and Mary Burket are also
thanked for their inputs and assistance in editing and producing this document.
2 P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY
What is a Case Study?
A case study is a story about something unique, special, or interesting—stories can be
about individuals, organizations, processes, programs, neighborhoods, institutions, and
even events.
1 The case study gives the story behind the result by capturing what happened
to bring it about, and can be a good opportunity to highlight a project’s success, or to
bring attention to a particular challenge or difficulty in a project. Cases
2 might be selected
because they are highly effective, not effective, representative, typical, or of special interest.
A few examples of case study topics are provided below—the case studies would describe
what happened when, to whom, and with what consequences in each case.
P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY 3
Case Study Examples Uniqueness/Point of Interest
Shifting Attitudes of Youth-Serving Your program was able to change service providers’ attitudes towards
Service Providers dealing with Adolescent Sexual and Reproductive Health (ASRH) needs in
an environment where provider’s attitudes have been a barrier to young
people accessing SRH services.
Integrating Youth-Friendly Services (YFS) Your program was effective in introducing YFS to the MOH and in
in the Ministry of Health (MOH) institutionalizing an YFS curriculum in a setting where the MOH did not
provide YFS.
Integrating HIV Prevention in Faith-Based Your program was able to integrate HIV prevention in several
Organization (FBO) Health Services Delivery FBO service delivery points in an environment that normally does not
include or welcome HIV prevention activities.
Developing Youth Leadership in Tanzania Your program built the leadership capacity of youth to advocate, promote,
and participate in decision making around ASRH. This transpired in a
setting that did not include ASRH on the agenda nor encourage youth
participation in general or in decision making in particular.
1
Yin, Robert K. (2003). Case Study Research: Design and Methods. Thousand Oakds: Sage Publications.
2
Case refers to the unit of analysis or topic chosen for study (i.e., the individual, organization, or program).
When is a Case Study Appropriate?
Case studies are appropriate when there is a unique or interesting story to be told. Case
studies are often used to provide context to other data (such as outcome data), offering a
more complete picture of what happened in the program and why.
What are the Advantages and Limitations of a Case Study?
The primary advantage of a case study is that it provides much more detailed information
than what is available through other methods, such as surveys. Case studies also allow one
to present data collected from multiple methods (i.e., surveys, interviews, document
review,
and observation) to provide the complete story. There are a few limitations and
pitfalls however, each of which is described below.
Can be lengthy:
Because they provide detailed information about the case in narrative form,
it may be difficult to hold a reader’s interest if too lengthy. In writing the case study, care
should be taken to provide the rich information in a digestible manner.
Concern that case studies lack rigor:
Case studies have been viewed in the evaluation and
research fields as less rigorous than surveys or other methods. Reasons for this include the
fact that qualitative research in general is still considered unscientific by some and in many
cases, case study researchers have not been systematic in their data collection or have
allowed bias in their findings. In conducting and writing case studies, all involved should
use care in being systematic in their data collection and take steps to ensure validity
3 and
reliability
4 in the study.
Not generalizable:
A common complaint about case studies is that it is difficult to generalize
from one case to another. But case studies have also been prone to overgeneralization,
which comes from selecting a few examples and assuming without evidence that they are
typical or representative of the population. Yin, a prominent researcher, advises case study
analysts to generalize findings to theories, as a scientist generalizes from experimental
results to theories.
5
4 P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY
3
Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure.
4
Reliability is the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials.
5
Yin, Robert K. (2003). Case Study Research: Design and Methods. Thousand Oaks: Sage Publications.
What is the Process for Conducting a Case Study?
The process for conducting case study research follows the same general process as is
followed for other research: plan, collect data, analyze data, and disseminate findings. More
detailed steps are given below.
1. Plan
• Identify stakeholders who will be involved.
• Brainstorm a case study topic, considering types of cases and why they are unique or
of interest.
• Identify what information is needed and from whom (see “What are Potential
Sources of Information?” and “What are the Elements of a Case Study?”).
• Identify any documents needed for review.
• List stakeholders to be interviewed or surveyed (national, facility, and beneficiary
levels) and determine sample if necessary.
• Ensure research will follow international and national ethical research standards,
including review by ethical research committees. For more information, please see
the International Ethical Guidelines for Biomedical Research Involving Human
Subjects, available at http://www.cioms.ch/frame_guidelines_nov_2002.htm.
2. Develop Instruments
• Develop interview/survey protocols—the rules that guide the administration and
implementation of the interview/survey. Put simply, these are the instructions that
are followed to ensure consistency across interviews/surveys, and thus increase the
reliability of the findings. The following instructions for the should be included in
the protocol:
• What to say to interviewees when setting up the interview/survey;
• What to say to interviewees when beginning the interview/survey, including
ensuring informed consent of the respondent (see Appendix 1 for an example);
• What to say to respondent in concluding the interview;
• What to do during the interview (Example: Take notes? Audiotape? Both?); and
• What to do following the interview (Example: Fill in notes? Check audiotape for
clarity? Summarize key information for each? Submit written findings?).
• Develop an interview guide/survey that lists the questions or issues to be explored and
includes an informed consent form. Please note that you will likely need interview
guides/surveys for each group of stakeholders, as questions may differ.
• Where necessary, translate guides into local languages and test translation.
P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY 5
3. Train Data Collectors
• Identify and train data collectors (if necessary). (See “Training Tips for Data
Collectors”
6.) Where necessary, use interviewers that speak the local language.
4. Collect Data
• Gather all relevant documents.
• Set up interviews/surveys with stakeholders (be sure to explain the purpose, why the
stakeholder has been chosen, and the expected duration).
• Seek informed consent of each respondent (written or documented oral). Re-explain
purpose of interview, why the stakeholder has been chosen, expected duration of,
whether and how the information will be kept confidential, and the use of a note
taker/tape recorder.
• If the respondent has consented, conduct the interview/survey.
5. Analyze Data
• Review all relevant documents.
• Review all interview/survey data.
6. Disseminate Findings
• Write report (see “What are the Elements of a Case Study?”).
• Solicit feedback.
• Revise
• Disseminate
Training Tips for Data Collectors
Staff, youth program participants, or professional interviewers may be involved in data
collection. Regardless of what experience data collectors have, training should include:
• An introduction to the evaluation objectives,
• A review of data collection techniques,
• A thorough review of the data collection items and instruments,
• Practice in the use of the instruments,
• Skill-building exercises on interviewing and interpersonal communication, and
• Discussion of ethical issues.
6 P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY
6
Adamchak, S., et. al. (2000). A Guide to Monitoring and Evaluating Adolescent Reproductive Health Programs. Available at
http://www.pathfind.org/site/PageServer?pagename=Publications_FOCUS_Guides_and_Tools.
What are Potential Sources of Information?
Case studies typically rely on multiple sources of information and methods to provide as
complete a picture as possible. Information sources could include:
• Project documents (including meeting minutes)
• Project reports, including quarterly reports, midterm reviews
• Monitoring visits
• Mystery client reports
• Facility assessment reports
• Interviews
• Questionnaire/survey results
• Evaluation reports
• Observation
• Other
What are the Elements of a Case Study?
Case studies do not have set elements that need to be included; the elements of each will
vary depending on the case or story chosen, the data collected, and the purpose (for
example, to illustrate a best case versus a typical case). However, case studies typically
describe a program or intervention put in place to address a particular problem. Therefore,
we provide the following elements and example on which you might draw:
1. The Problem
i. Identify the problem
ii. Explain why the problem is important
iii. How was the problem identified?
iv. Was the process for identifying the problem effective?
2. Steps taken to address the problem
3. Results
4. Challenges and how they were met
5. Beyond Results
6. Lessons Learned
P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY 7
1. The Problem:
It is essential to identify what the problem was. Specifically:
i.
Identify the problem. For example, certain FBO leadership was opposed to including
HIV prevention activities within their programming. Specifically, they felt that
prevention activities would promote sex among youth and thereby increase the
incidence of HIV. In addition, the FBO felt that the promotion of condoms as a
form of prevention was morally wrong.
ii.
Explain why the problem is important. Following the same example, to provide a
comprehensive response to HIV/AIDS, prevention activities are an important part
of HIV programming, in addition to care and support. Addressing moral and
religious concerns and objections of FBO leaders in order to incorporate this
component greatly expands the prevention effort and offers a greater breadth and
depth in HIV programming.
iii.
How was the problem identified? It is likely that program staff will have an idea of
what general problems exist. For example, YFS is not integrated in the MOH,
FBOs do not include prevention activities, youth involvement does not occur, etc.
What was likely missing and required exploration were the details around this
general problem. In the case of working with FBOs, an initial project planning
meeting was held to discuss the specifics behind the problem. It was revealed
during this process that certain FBOs who conducted care and support activities
did not offer prevention activities, specifically condom promotion. The primary
reason given was religious objections; another reason identified but not explicitly
stated was stigma.
iv.
Was the process for identifying the problem effective? After additional one-on-one
discussions with FBO leaders, it was determined that the leaders had concerns
based on moral and religious grounds regarding prevention efforts, particularly
promoting condoms. Furthermore, discussions revealed that a number of leaders
had limited and/or erroneous information regarding transmission and
prevention options.
8 P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY
2. Steps Undertaken to Address the Problem:
What was done (activities/ interventions/
inputs), where, by whom, for whom?
In the case of integrating prevention in FBOs, an
illustrative outline of steps undertaken to address the problem is given below:
P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY 9
What was done? Where? By Whom? For Whom?
Phase 1: Sensitization
Sensitization of leadership:
activities included reviewing District/ National Project Staff FBO leaders
FBO policies and bylaws by program staff in order to
demonstrate to leadership that there were no constitutional
objections to prevention activities. Project staff engaged
liberal FBO leaders to supply quotes from the Bible to
build their evidence-based argument to convince other
leaders that there was in fact no biblical basis for the
preclusion of prevention activities.
Sensitization of facility managers:
meet with managers Facilities Project Staff Facility
to discuss leadership approval and present evidence-based Managers
arguments on why integration activities can be included.
Sensitization of community:
work with groups to explain Community Project Staff Community
that these activities have been approved by the FBO catchment areas members
leadership and where they can go to receive services.
Phase 2: Integration
Facility assessments
Selected facilities Project Staff, FBOs,
FBO representatives Project Staff
Training of service providers
Selected facilities Local Implementing Facility Staff
Partners (IPs)
Conduct staff orientations
Facilities Project Staff, Facility Staff
Facility Managers
Physical infrastructure improved
Facilities Contractor Facility
Phase 3: Outreach
Identify peers that you want to work with
Selected facilities Facility Staff Beneficiaries
Training
Facilities IPs Peers
Provide Tools
Project Peers
Supervision
Peer supervisors Peers
Phase 4: M&E
Reassessments
Sample of Project Staff, Facility,
selected facilities Facility Management, Project Staff
Youth
Mystery client interviews
Facilities Trained mystery clients
Monitoring visits
Facilities, Peers Project Staff, Facility staff,
Peer Supervisors Project Staff
3. Results:
What were the results of your intervention, particularly the significant or unique
results? For example: your activities resulted in the FBO including HIV prevention
activities in their programming, which contributed to an increase in condom distribution.
Or, your activities resulted in youth-serving service providers adopting a youth-friendly
attitude which contributed to improved services and an increase in youth visits. Church
leadership may have made a commitment and modified policies to include ASRH
activities, leadership talking about the issues publicly. Facility managers may make
financial commitments to refurbishments, training, etc.
4. Challenges and how they were met:
This focuses on what challenges or difficulties you
encountered and what you did to overcome them. One of the challenges in working with
FBOs may have been dealing with Catholic FBOs and your response could have been to
negotiate with them to include certain prevention activities like abstinence, and/or
providing a referral point to youth who may want to get information about condoms.
Another possible challenge is that not all FBOs who you had hoped to work with were
willing to come on board and this required additional attention.
5. Beyond Results: Are the results mentioned above sustainable?
Why or why not? For
example, an integrated approach to programming (which included a strong policy and
advocacy component) created an enabling policy environment with strong stakeholder
support that can be nurtured and leveraged beyond the project duration.
6. Lessons Learned:
What lessons were learned: programmatic, technical, financial,
process, etc.? For example, the experience showed that it was imperative for project staff
to have a very solid understanding of government policies to speak to these issues with
key stakeholders. In addition, it was important to have a good understanding of the
various religious beliefs in addition to their respective policies and by-laws for
developing and delivering advocacy arguments.
How are Case Studies Presented?
Case studies are flexible in that they can be presented in a number of ways—there is no
specific format to follow. However, like all evaluation results, justification and methodology of
the study should be provided, as well as any supporting information (i.e., copies of instruments
and guides used in the study). Case studies may stand alone or be included in a larger
evaluation report. If presented as a stand-alone report, the following report outline is suggested:
1. Introduction and Justification
2. Methodology
a. How was the process carried out? (Describe the process of selecting the case and
data collection sources, as well as how data was collected.)
b. What assumptions are there (if any)?
10 P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY
c. Are there any limitations with this method?
d. What instruments were used to collect data? (You may want to include some
or all in the appendix.)
e. What sample(s) is/are being used?
f. Over which period of time was this data collected?
3. The Problem
4. The Steps Taken to Address the Problem
5. The Results
6. The Challenges and How They were Met
7. Beyond Results
8. Lessons Learned
9. Conclusion
10. Appendices
Where Can More Information on Case Studies be Found?
Information on Case Study Research
Adamchak, S., et. al. (2000).
A Guide to Monitoring and Evaluating Adolescent Reproductive Health
Programs.
Available at http://www.pathfind.org/pf/pubs/focus/guidesandtools/PDF/Part%20II.pdf.
Patton, Michael Q. (2002).
Qualitative Research & Evaluation Methods. Thousand Oaks: Sage
Publications.
Stake, Robert E. (1995).
The Art of Case Study Research. Thousand Oaks: Sage Publications.
United States General Accounting Office, Program Evaluation and Methodology Division.
(1987).
Case Study Evaluations. Available at http://161.203.16.4/t2pbat22/132683.pdf.
Yin, Robert K. (2003).
Case Study Research: Design and Methods. London: Sage Publications.
Examples of Case Studies
Cornwall, A. and Welbourn, A. (2002). Realizing
Rights: Transforming Approaches to Sexual and
Reproductive Well-Being.
London: Zed Books.
Population Council.
Quality/Calidad/Qualité series. Available at
http://www.popcouncil.org/publications/qcq/default.htm.
SRI International. (2001).
The Organization of Learning in Community Technology Centers:
Learning with Technology in Six Communities.
Available at
http://www.americaconnects.net/research/SRI_case_study_report.pdf.
P
ATHFINDER INTERNATIONAL: WRITING A CASE STUDY 11
Appendix 1: Sample Informed Consent Form
12 P
ATHFINDER INTERNATIONAL: SAMPLE INFORMED CONSENT FORM
Key Components:
• Thank you
• Your name
• Purpose
• Confidentiality
• Duration
• How interview
will be conducted
• Opportunity for
questions
• Signature of
consent
I want to thank you for taking the time to meet with me today. My
name is ________________________ and I would like to talk to you
about your experiences participating in the African Youth Alliance (AYA)
project. Specifically, as one of the components of our overall program
evaluation we are assessing program effectiveness in order to capture
lessons that can be used in future interventions.
The interview should take less than an hour. I will be taping the session
because I don’t want to miss any of your comments. Although I will be
taking some notes during the session, I can’t possibly write fast enough
to get it all down. Because we’re on tape, please be sure to speak up so
that we don’t miss your comments.
All responses will be kept confidential. This means that your interview
responses will only be shared with research team members and we will
ensure that any information we include in our report does not identify
you as the respondent. Remember, you don’t have to talk about anything
you don’t want to and you may end the interview at any time.
Are there any questions about what I have just explained?
Are you willing to participate in this interview?
__________________ __________________ __________
Interviewee Witness Date
______________________________________
Legal guardian (if interviewee is under 18)
Pathfinder International
9 Galen Street, Suite 217
Watertown, MA 02472
USA
Tel: 617-924-7200
Email: Information@pathfind.org
05/06/500
No comments:
Post a Comment