1​‌‍‍‍‌‍‍‌‌‍‌‍‍‌‌‌‍‍‍‍​] What are two things you learned about data analysis and interpretation? 2] Having this experience, in what ways can evaluation data be used by a program or practitioner? 3] What might be barriers for programs to implement evaluation? For these barriers, 4] what might be strategies that may overcome these barriers. 5] How might various stakeholders (particularly the target population) be involved in program evaluation activities? Resources Limitations to Inferential Statistics and Clinical Significance Although much of the focus has been on evaluating hypotheses in terms of statistical significance it should be noted that this approach does have its limitations, especially for the evaluation of interventions. For one, as you may have noticed in statistical power, these tests are sensitive to sample size. A small effect with a large sample size may yield statistically significant results. Tests of statistical significance are not intended to provide information as to whether the effects of an intervention are curative or helpful, rather, they provide an indication of whether the observed results would be obtained in the absence of some differences in the population (Bothe & Richardson, 2014). The process of selecting an arbitrary criterion (the significance level) to make a binary decision on the effect of an intervention is at times over emphasized in deciding. Increasingly, the use of effect size statistics has also been employed to examine the robustness of results. However, in many cases these are just an extension of statistical inference tests (Pogrow, 2019). It is important to be mindful that over-reliance on statistical procedures may undermine the personal significance that may be attributed to interventions Ethics of Analyzing and Reporting Data In previous research methods courses you have been exposed to ethical conduct in research primarily as it relates to the welfare to research participants and the methods used for recruitment and study participation. However, there are also ethical considerations serving as guidelines related to the dissemination of the results of an evaluation or research project. These include considerations on the usefulness of the hypotheses being tested in research and on how statistical procedures can be appropriately used in the course of analysis. The ethical issues related to analyzing and reporting data go beyond fraud and misrepresentation of data. This module introduces ethical concerns related to the discretion used in articulating hypotheses for the study; the use of statistical procedures; and issues relating to authorship and ownership of the data a​‌‍‍‍‌‍‍‌‌‍‌‍‍‌‌‌‍‍‍‍​nd publications from a research activity. In addition to the introduction to these issues, dynamics such as publication bias and conflict of interest are highlighted. Publication bias is the “bias that something is more likely to be published among what is available to be published” (Kerr, Prescott, Theologis, 2015, p. 215). It has been known that studies with significant results are favored over those studies that support the null hypothesis. Conflicts of interest arise when an author’s relationship with other stakeholders may have or have the appearance of compromising the objectivity of the research and reporting of findings. This is a particularly salient issue in evaluation research where the evaluation’s results may have implications for future funding decisions. Readings Bothe, A. K., & Richardson, J. D. (2011). Statistical, Practical, Clinical, and Personal Significance: Definitions and Applications in Speech-Language Pathology. American Journal of Speech-Language Pathology, 20(3), 233-242. King, C.R. (2009). Issues and best practices related to ethical writing and publishing. Journal of the Association for Vascular Access, 14 (1), 40 -45. Menezes, R. G., Giri, S., Pant, S., Kharoshah, M. A., Madadin, M., & Nagaraja, S. B. (2014). Publication ethics. Medico-Legal Journal, 82(4), 155-158. https://doi.org/10.1177/0025817214526524 (Links to an external site.) Milton, C. L. (2019). Ethics and the Reporting of Research Findings. Nursing Science Quarterly, 32(1), 23. Pogrow, S. (2019). How effect size (practical significance) misleads clinical practice: The case for switching to practical benefit to assess applied research findings. The American Statistician, 73 suppl 1, 223 -234. References Bothe, A.K., & Richardson, J.D. (2011). Statistical, practical, clinical and personal significance: Definitions and applications in speech-language pathology. American Journal of Speech-Language Pathology, 20, 233 – 242. Kerr, A., Prescott, R. & Theologis, T (2015). Bad science and how to avoid it, a movement analysis perspective: Study design, statistics and publication ethics. Gait & Posture, 42, 224 – 226. King, C.R. (2009). Issues and best practices related to ethical writing and publishing. Journal of the Association for Vascular Access, 14(1), 40 -45. Milton, C.L. (2019). Ethics and the reporting of research findings. Nursing Science Quarterly, 32(1), 23-24. Pogrow, S. (2019). How effect size (practical significance) misleads clinical practice: The case for switching to practical benefit to assess applied research findings. The American Statistician, 73 suppl 1, 223 -234​‌‍‍‍‌‍‍‌‌‍‌‍‍‌‌‌‍‍‍‍​.