ROBIS: Tool to assess risk of bias in systematic reviews
Use this ROBIS tool for the assigned review article in your Assignment 3.
Reference of the review article under appraisal:
Rawat, D., et al. (2021). Vitamin C and COVID-19 treatment: A systematic review and meta-analysis of randomized controlled trials. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 15(6), 102324.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8552785/
Phase 1: Assessing relevance
PICO Question: In hospitalized patients with COVID-19 does the administration of vitamin C improve clinical outcomes compared to those who did not receive vitamin C?
Intervention reviews:
Component In your PICO question In the review article
Patients/Population(s): Hospitalized patients with Covid-19 Hospitalized patients with Covid-19
Intervention(s): Administration of vitamin C Administration of vitamin C
Comparator(s): Not receiving vitamin C Not receiving vitamin C
Outcome(s): Improve clinical outcomes Improve clinical outcomes
Does the question addressed by the review match your PICO question? Yes/No/Unclear
The PICO question is similar in both cases.
Phase 2: Identifying concerns with the review process
Describe the review process for each domain and then respond to the questions for each domain: Y=Yes; PY=Probably Yes; PN=Probably No; N=No; NI=No Information
DOMAIN 1: STUDY ELIGIBILITY CRITERIA
Describe the study eligibility criteria, any restrictions on eligibility and whether there was evidence that objectives and eligibility criteria were pre-specified:
The eligibility criteria involved patients with Covid-19 or SARS-CoV2. There is evidence that the objectives and the eligibility criteria were pre-specified. The objectives were used to set the inclusion and exclusion criteria. Also, the sample characteristics was restricted; though, the size was determined by the sources collected. The eligibility criteria was clear and this the concerns regarding specification of the criteria are low.
1.11.1 Did the review adhere to pre-defined objectives and eligibility criteria? Y/PY/PN/N/NI
There were pre-defined objectives and eligibility criteria.
1.21.2 Were the eligibility criteria appropriate for the review question?
Y/PY/PN/N/NI
The eligibility criteria was relevant for identifying the right population.
1.31.3Were eligibility criteria unambiguous? Y/PY/PN/N/NI
The eligibility criteria was clear
1.4 1.4Were any restrictions in eligibility criteria based on study characteristics appropriate (e.g. date, sample size, study quality, outcomes measured)? Y/PY/PN/N/NI
The eligibility was based on the population characteristic.
1.51.5 Were any restrictions in eligibility criteria based on sources of information appropriate (e.g. publication status or format, language, availability of data)? Y/PY/PN/N/NI
Studies published on other languages other than English were excluded.
1.6 Concerns regarding specification of study eligibility criteria Low/HighUnclear
The study framework was accordingly designed.
Rationale for concern:
There are a few gaps in the pre-determined objectives and status of the publications.
DOMAIN 2: IDENTIFICATION AND SELECTION OF STUDIES
Describe methods of study identification and selection (e.g. number of reviewers involved, strategies to identify unpublished articles, etc):
The study reviewed a total of 6 RCTs with a sample population of 572 patients. Articles published on the use of vitamin C on Covid-19 patients in all categories were considered. Many databases were used and various sources retrieved. The sources were then narrowed down to the most suitable ones based on the pre-determined criteria. Also, the errors of selection were minimized by determining whether the study had abstract ad full information and eliminating duplications.
2.1 Did the search include an appropriate range of databases/electronic sources for published and unpublished reports? Y/PY/PN/N/NI
The study did not outline the database selection process.
2.2 Were methods additional to database searching used to identify relevant reports? Y/PY/PN/N/NI
The criteria for identifying the relevant database were identified.
2.3 Were the terms and structure of the search strategy likely to retrieve as many eligible studies as possible? Y/PY/PN/N/NI
The term and structure of search was broad and could help get many eligible sources.
2.4 Were restrictions based on date, publication format, or language appropriate? Y/PY/PN/N/NI
The restrictions were based on language, date of publication and availability.
2.5 Were efforts made to minimize error in selection of studies? Y/PY/PN/N/NI
The errors of duplication were eliminated in the study.
Concerns regarding methods used to identify and/or select studies LOW/HIGH/UNCLEAR
The database such was relevant and would help identify the right sources.
Rationale for concern:
The number of resources used were limited. Also, there is description on determining whether the patients had other conditions with confounding effects to the study.
DOMAIN 3: DATA COLLECTION AND STUDY APPRAISAL
Describe methods of data collection, what data were extracted from studies or collected through other means, how risk of bias was assessed (e.g. number of reviewers involved) and the tool used to assess risk of bias:
The results collected are relevant to the research question. Also, the risks for bias in the methodology were assessed and addressed. The data was collected based on systematic review and meta-analysis. The risk graph was constructed and used to determine the suitability of the search strategies and data collected.
3.1 Were efforts made to minimize error in data collection? Y/PY/PN/N/NI
The data synthesis and extraction was objective.
3.2 Were sufficient study characteristics available for both review authors and readers to be able to interpret the results? Y/PY/PN/N/NI
There were enough reports and characteristics for review by the readers.
3.3 Were all relevant study results collected for use in the synthesis? Y/PY/PN/N/NI
Enough relevant study results were used in determining the role of vitamin C in Covid-19 treatment.
3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria? Y/PY/PN/N/NI
The risk for the methodological bias was assessed using Cochrane Systematic Review Guidelines and GRADE-PRO.
3.5 Were efforts made to minimize error in risk of bias assessment? Y/PY/PN/N/NI
There were significant efforts to reduce errors and risk of bias in the data reported.
Concerns regarding methods used to collect data and appraise studies LOW/HIGH/UNCLEAR
The findings reported are credible and reliable.
Rationale for concern:
DOMAIN 4: SYNTHESIS AND FINDINGS
Describe synthesis methods:
The data extracted from the identified sources included the design, setting, severity of the disease, duration and interventions initiated, days of hospital stay and length of ICU stay. Relevant information was synthesized from the identified sources. Though, the in between study variations was not done. The findings were presented in various forms including plot and sensitivity analysis being done and the primary biases minimized.
4.1 Did the synthesis include all studies that it should? Y/PY/PN/N/NI
All the studies were used in the synthesis.
4.2 Were all pre-defined analyses reported or departures explained? Y/PY/PN/N/NI
Adequate explanation was provided on the reports given in the study.
4.3 Was the synthesis appropriate given the nature and similarity in the research questions, study designs and outcomes across included studies? Y/PY/PN/N/NI
Descriptive statistics were used to report the findings.
4.4 Was between-study variation (heterogeneity) minimal or addressed in the synthesis? Y/PY/PN/N/NI
The between-study variations was not analyzed.
4.5 Were the findings robust, e.g. as demonstrated through funnel plot or sensitivity analyses? Y/PY/PN/N/NI
Neither funnel plot nor sensitivity analysis were used.
4.6 Were biases in primary studies minimal or addressed in the synthesis? Y/PY/PN/N/NI
The biases in the primary studies were minimal and this makes the results reliable.
Concerns regarding the synthesis and findings LOW/HIGH/UNCLEAR
There are no significant concerns on the credibility of the study results.
Rationale for concern:
The bias review and credibility of the study findings are elaborately described.
Phase 3: Judging risk of bias
Summarize the concerns identified during the Phase 2 assessment
: Domain Concern Rationale for concern
1. Concerns regarding specification of study eligibility criteria The study was specific The sample populations and study settings were defined
2. Concerns regarding methods used to identify and/or select studies Sample population The sample setting was limited and so could not provide diversified information on the research question. .
3. Concerns regarding methods used to collect data and appraise studies Limited information were provided on appraisal The study provides a sketch of the data collection and appraisal of the selected sources.
4. Concerns regarding the synthesis and findings The synthesis and findings were appropriate The biases in the findings and synthesis were identified.
RISK OF BIAS IN THE REVIEW
Describe whether conclusions were supported by the evidence:
The conclusion in the study was supported by the statically evidences. The findings showed that vitamin C did not reduce the covid-19 related health outcomes.
A. Did the interpretation of findings address all of the concerns identified in Domains 1 to 4? Y/PY/PN/N/NI
The interpretation identified the four domains.
B. Was the relevance of identified studies to the review’s research question appropriately considered? Y/PY/PN/N/NI
The sources selected for review were relevant.
C. Did the reviewers avoid emphasizing results on the basis of their statistical significance? Y/PY/PN/N/NI
The reviewers made conclusion based on the statistical significance; though, without overemphasis.
Risk of bias in the review Risk: Low/High/Unclear
The risk for bias in the study results is limited.
Rationale for: The data collection method was appropriate and the sample population covered in the six sources were enough to provide reliable information on the research question.
Y=YES, PY=PROBABLY YES, PN=PROBABLY NO, N=NO, NI=NO INFORMATION