Assignment – Research Evaluation Assignment
The goal of this assignment is to familiarize you with current models for the evaluation and criticism of evidence-based research. These models help the practitioner or student to critique the level of evidence that is produced by a research study. Without such models there would be no standardized means of determining the validity of the claims in literature published on the effectiveness of specific treatments.
Assignment Guidelines
A. Choose a specific research article that evaluates an intervention related to individuals with a diagnosis of serious mental illness.. CBT THERAPY FOR INDIVIDUALS WITH DEPRESSION
B. Write a paper that addresses the following questions and points:
1) Provide a description of the intervention, including the key components that must be present in order for the intervention to have good fidelity.
2) What are the goals, targets and desired outcomes of this intervention, and how is progress toward these outcomes typically tracked?
3) What is the nature of the all the evidence supporting this intervention?
• You should summarize the key relevant professional academic literature in the area (from social work and related professions)—this should be done in a table format (author, sample, interventions tested, design, measures, and results). Include age, race, and ethnicity of sample in sample description whenever this information is reported.
• Your research must extend beyond the required and recommended readings for the course, although you are welcome to use course readings in your paper.
• The information you review in this section should present the state-of-the empirical research relevant to your selected intervention.
• Categorize the intervention according to one of the Level of Evidence classification models presented in this assignment, and present the rationale for this categorization.
4) What additional research is needed on this intervention to enhance its level of empirical support, or to demonstrate its effectiveness with key populations?
C. Reference List. List all references according to American Psychological Association (APA) style. Papers that have references and citations that do not conform to APA format will lose points. You will be graded on: 1) your ability to use the professional literature and other relevant information to address the task and justify the assessment tool/intervention; 2) your ability to critically assess, synthesize, and organize the information gathered; 3) the overall writing quality of the paper (organization, grammar, lack of typos, APA guidelines for all citations and referencing, etc.).
Guidelines for Evaluating Research – Two Models for Research Criticism:
Level of Evidence Coding System:
This model was adopted from the Agency of Health Care Policy’s classification of Level of Evidence system. This system classifies each study into one of six categories in order to denote the strength of the evidence for the intervention (Foa, Keane, Friedman, 2000).
• Level A: Evidence is based upon randomized, well-controlled clinical trials for individuals with target diagnosis.
• Level B: Evidence is based upon well-designed clinical studies, without randomization or placebo comparison for individuals with target diagnosis.
• Level C: Evidence is based on service and naturalistic clinical studies, combined with clinical observations that are sufficiently compelling to warrant use of the treatment technique or follow the specific recommendation for individuals with target diagnosis.
• Level D: Evidence is based on long-standing and widespread clinical practice that has not been subjected to empirical tests for individuals with target diagnosis.
• Level E: Evidence is based on long-standing practice by circumscribed groups of clinicians that has not been subjected to empirical tests for individuals with target diagnosis.
• Level F: Evidence is based on recently developed treatment that has not been subjected to clinical or empirical tests for individuals with target diagnosis.
APA Model for Evaluation of Research:
The APA has developed a model with which a practitioner or student can effectively judge the level of evidence that is provided by an evidence-based report on the effectiveness of a particular psychotherapeutic intervention (Crits-Cristoph, Frank, Chambless, Brody & Karp, 1995). The APA developed this model so that students and clinicians are able to judge the degree to which a particular research project demonstrates the effectiveness of the intervention. In the APA model there are three levels of evidence:
1. Well-Established Treatments
a. Highly specified, typically through treatment manuals, such that procedures could be clearly understood and replicated by others.
b. Validated in studies in which the characteristics of the client samples were clearly described.
c. Documented to be effective in either:
1. at least two group-design studies of adequate statistical power demonstrating efficacy through superiority (statistical significance) to pill or adequate psychological placebo, or through superiority or equivalence to an already established treatment.
2. a large series of single-case design studies using good experimental design and demonstrating superiority to pill or psychological placebo or to another treatment
d. Demonstrated to be effective in studies by at least two different investigators.
2. Probably Efficacious Treatments
a. Highly specified, typically through treatment manuals, such that procedures could be clearly understood and replicated by others.
b. Validated in studies in which the characteristics of the client samples were clearly described.
c. Documented to be effective in either:
1. Two studies showing that the treatment is more efficacious than a waiting-list control-group
2. One study demonstrating efficacy either through performance superior to that of a pill or psychological placebo, or equivalency to an already established treatment
3. A small series of good single-case design studies
4. At least two good studies demonstrating efficacy, but flawed by the heterogeneity of the client samples
3. Experimental Treatments
Treatments that do not meet criteria for either of the above categories are labeled experimental. A treatment with this label may be, but is not definitively effective.
Curtis, George C. (1996). The Scientific Evaluation of New Claims. Research on Social Work Practice, 6, 117-121.
Foa, E.B., Keane, T., Friedman, M. (Eds) (2000). Effective treatments for PTSD: Practice guidelines from the International Society for Traumatic Stress Studies. New York: Guildford Press.
Crits-Christoph, P., Frank, E., Chambless, D. L., Brody, C., & Karp, J. F. (1995). Training in empirically validated treatments: What are clinical psychology students learning? Professional Psychology: Research and Practice, 26, 514 -522.