Critique a Journal Article

This assignment is a guide for how to read articles, not just as a consumer of information, but as a critic of the source of the information. The ability to extract meaning from journal articles and the ability to critically evaluate research from a statistical perspective will be fundamental skills that will help you identify valid and reliable information for use in your career and personal life.
We can have many roles in our personal lives, for example, a man can simultaneously be a father, a brother, a son, an uncle, a friend, etc. We also have different roles when it comes to how we interact with the information around us. We can be a consumer of information or we can be a reviewer (critic) of the information. An important reason to study statistics is to be able to read journal articles. Most articles you read will contain some form of statistics. An understanding of basic statistics will provide you with the fundamental skills necessary to read, interpret, and evaluate most statistical papers.
Given the growing importance of decisions and opinions based on data, it’s crucial that you can critically assess the quality of analyses that others present to you. Statistics allow you to evaluate claims based on quantitative evidence and help you differentiate between reasonable and dubious conclusions. That aspect is particularly vital these days because data are so plentiful along with interpretations presented by people with unknown motivations.

When analysts use statistical procedures correctly, they tend to produce accurate results. In fact, statistical analyses account for uncertainty and error in the results. Statisticians ensure that all aspects of a study follow the appropriate methods to produce trustworthy results. These methods include collecting a representative sample, producing reliable data, analyzing the data appropriately, and drawing reasonable conclusions. Performing an article review should enable each peer-reviewer to provide the writer with a written response that will help the writer determine which parts of the paper are effective as is, and which are unclear, incomplete, or unconvincing.

Read A Guide to Peer Reviewing Articles for more information regarding how to perform an article review.

Instructions: Submit a word file 2 pages in length, standard margins, 12-point font, double spacing.

Your article review will be broken down into 5 parts. Make sure that each section is clearly denoted in your submission.
1. Introduction: Summarize the article. This will be your view of the article as a consumer of the information. In this section, you will provide 1-2 paragraphs summarizing the article in your own words.
2. Major flaws: Indicate which parts of the paper you find most or least effective and explain why. List
the major points of support or evidence
3. Minor flaws: this includes other, lesser suggestions and final comments. This includes grammatical errors and formatting issues in the paper. Indicate sentences or paragraphs that seem out of order, incompletely explained, or otherwise in need of revision. See details below.
4. Final comments: State whether the paper is publishable or whether there are fatal flaws
5. Reflection: Answer the following questions:
A. In your conclusion, provide your overall opinion of the article.
B. What did you learn from the article that was most shocking, unusual, or not what you thought previously?

C. Discuss why the reviewer role is so important. Consider what happens if an article is not appropriately critiqued? We can have articles published that are not valid and reliable studies. For example, the original article “Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children” by A. J. Wakefield et al. was published in 1998, which claimed to find a link between autism and vaccines, was later retracted. This article was not initially reviewed appropriately and resulted in a publication of unreliable research that has vast negative impacts on society.

The major flaws section of the paper needs to be discussed in detail and includes several criteria that need to be evaluated. Evaluate the purpose of the study, the literature review, study design, data collection and sampling, data presentation and analysis, and conclusions.

Study Purpose

1. Was there a clear statement of the purpose and aims of the research?
A. Did the author provide a clear statement of the purpose of the research? The purpose usually stated briefly in the abstract of the article, and again in more detail in the introduction. It may be phrased as a research question. There might be clear hypotheses statements that will be tested.
B. Do you believe that the topic of this research is important, relevant, and of interest?
C. Discuss how the author can possibly improve the clarity and impact of this research.

Literature Review

2. Was relevant background literature reviewed?

A review of the literature should be included in an article describing research to provide some background to the study. It should provide a synthesis of relevant information such as previous work/research, and discussion of the clinical importance of the topic. It identifies gaps in current knowledge and research about the topic of interest, and thus justifies the need for the study being reported. Discuss the following:
A. How many articles were reference in the article and how recent were the articles (years referenced)?
B. Do the articles referenced thoroughly discuss what has been done in the pat related to the
research proposed in the article and how the research in the article builds on past research?
C. Discuss how the author can possibly improve the literature review section of the paper.

Study Design, Data Collection, and Sampling

3. Discuss the internal validity and external validity of the study design.

Internal validity refers to whether the effects observed in a study are due to the manipulation of the independent variable and not some other factor. In other words, that there is a causal relationship between the independent and dependent variable. Internal validity can be improved by controlling extraneous variables, using standardized instructions, counterbalancing, and eliminating demand characteristics and investigator effects.
A. Discuss how the author addressed the internal validity of the study.
B. Discuss how the author can possibly improve the internal validity of the study design.

External validity refers to the extent to which the results of a study can be generalized to other settings (ecological validity), other people (population validity), and over time (historical validity). External validity can be improved by setting experiments in a more natural setting and using random sampling to select participants.
C. Discuss how the author addressed the external validity of the study.
D. Discuss how the author can possibly improve the external validity of the study design.

4. Was the sampling method appropriate?
A. Were participants relevant to the research question and was their selection well-reasoned?
B. What was the criterion used for inclusion in the study (characteristics of the participants in the study, demographics, locations, etc.)?
C. Was the total sample size and the sample size for each group adequate?
D. Discuss how the author could improve the sampling.

5. Measuring: Identify the variables in the study, whether they are independent or dependent variables and the type of variable.

It is important to identify the variables in the study and their classification because the type of graph and statistical test needed depends on the variables being studied.
A. How many variables were collected in this study? Classify the variables as qualitative or quantitative.
B. Which variables were the independent variables and which variables were the dependent variable?

Data Presentation and Analysis

6. Were graphs presented, and if so, were they accurate and easily interpreted?
A. What tables and figures (graphs) are presented in the paper? Were the correct graphs used based on the type of variables being studied?
B. If the paper includes tables, graphs, or figures, what do they add to the paper? Do they aid understanding or are they superfluous? That is, are they aligned to the discussion in the paper?
C. Discuss how the author could improve the data presentation.

7. Was the correct statistical analysis done? (See table 1 at the end of this document.)
A. Different statistical studies are used for the comparison of different variables. What statistical tests were used and were they the correct statistical tests that should have been used?
B. Were effect sizes reported?
C. Were assumptions for the statistical test reported?
D. Discuss how the author could improve the statistical analysis?

Conclusions

8. Were conclusions appropriate, given the study findings? Did the findings contribute to practice or research?

Conclusions should be consistent and congruent with the findings as reported by the researchers. All of the data and findings should be discussed and synthesized. The conclusions of the study should be meaningful to the reader and provide insight into important professional issues. The authors should relate the findings

back to the existing literature and theoretical knowledge in practice. Implications and recommendations should be explicitly linked to practice situations and research directions.
A. Discuss how the author can possibly improve the conclusions section.

Table 1: Choosing the correct statistical test

Type of Graphs Statistical Effect Assumptions covered
variables covered in test we size we in this course
being this course learned in learned
compared this in this
course course
2 Scatterplot Regression r, r2 1) there is a linear
quantitative analysis relationship between X
variables and Y, 2) independence
of the errors, 3) normality
of the errors, and 4) equal
variance of the errors
1 qualitative Bar chart Chi-square Cramer’s Expected counts > 5
variable and a d test V, Phi
distribution
2 qualitative Stacked or
variables clustered
bar graph
1 qualitative Boxplot ANOVA Eta2 Equal variances and
and one quantitative normality
variable