Chapter 36: Reporting tools

What is a qualitative reporting tool?

Generally speaking, a tool is something that helps someone complete a task. Qualitative reporting tools are usually checklists or frameworks that help researchers demonstrate to publishing journals that they have followed a comprehensive process in reporting on their research. Qualitative reporting tools provide detailed information on the research team, study methods, context of the study, findings, data analysis and interpretations. Although using these tools could be considered unnecessary, they can help in planning the study as well as with reflexivity.

The main reporting tools used in qualitative research

There are three main reporting tools academic journals use to ensure comprehensive reporting and transparency in reporting qualitative results. These are the Consolidated Criteria for Reporting Qualitative Research (COREQ),¹ the Standards for Reporting Qualitative Research (SRQR)², and the Johanna Briggs Checklist for Qualitative Research (JBI).³ A fourth tool, the Critical Appraisal Skills Programme (CASP), a qualitative research checklist,⁴ is also helpful for researchers, although not required by journals. While using these checklists might seem onerous, they are very helpful in focusing the researcher on improved reporting of qualitative research.

COREQ

The COREQ¹ is a 32-item checklist that is designed to help researchers report on their results. It is divided into three domains: research team and reflexivity (8 items), study design (15 items), and analysis and findings (9 items). Many journals request the author(s) complete this checklist and submit it with their manuscript. While it is a lengthy checklist, it is not difficult to complete. The first step in completing the tool is to ensure the manuscript is submitted with line numbers; this is easily done by clicking ‘Line numbers’ in the layout tab in Microsoft Word. Once the lines are numbered, the researcher is able to refer to the line number(s) containing the information being requested. Once completed, the checklist should be uploaded to the journal site as a separate document.

Strengths and limitations of COREQ

A strength of COREQ is that qualitative research is clearly and transparently presented. Reviews of COREQ have highlighted ambiguity in reporting, which could be due to the subjective nature of the items; ethical concerns not being adequately addressed in the checklist; and while the checklist might be completed, the research may still be of poor quality.⁵

SRQR

The SRQR² consists of 21 items and ‘aims to improve the transparency of all aspects of qualitative research by providing clear standards for reporting qualitative research’^(p1245). It includes the article title and abstract; problem formulation and research question; research design and methods of data collection and analysis; results, interpretation, discussion and integration; and other information. The checklist is completed along the same lines as COREQ, by providing answers based on line numbers. Once completed, the checklist is uploaded to the journal site as a separate document.

Strengths and limitations of the SRQR

Strengths of the SRQR are that it is moderate in length, highly transparent in purpose and claims to consider all qualitative research methodologies equally. The tool was developed through a systematic consideration of existing tools and is appropriate for use by experienced qualitative researchers. Limitations are that it is unclear whether SRQR truly considers all qualitative methodologies equally¹⁰and that it does not explicitly mention a conceptual framework.⁶

JBI

JBI³ provides a means for assessing the quality of a study for the possibility of bias in design, conduct and analysis. It comprises 10 questions that researchers answer about aspects of the research, such as congruity between the stated philosophical perspective and the research methodology, whether conclusions from the research flow from the analysis and interpretation of the data. The tool does not ask for line numbers as evidence; rather, it provides checkboxes to confirm items. This checklist is often used as supporting evidence for systematic reviews (evidence synthesis).

Strengths and limitations of JBI

JBI’s focus on congruity of results is considered both a strength and a limitation.^7-9 This brings attention to the study’s philosophical perspective, methodology, research questions and researcher reflexivity but comes at the expense of comprehensiveness. Brevity, ease of use and clarity mean that JBI can be used by less experienced researchers.⁹

CASP

CASP comprises 10 questions divided into three categories: Are the results of the study valid? What are the results? Will the results help locally?⁴ This checklist was designed to be used as an educational pedagogic tool, and is used for quality appraisal in health-related qualitative evidence syntheses.

Strengths and limitations of CASP

CASP has been described as of the most commonly used checklist or criteria-based tools for quality appraisal in health and social care-related qualitative evidence syntheses.⁸ CASP comprises only 10 items and has been criticised for not including items that provide a rationale for sample size and recall bias, and relatively minimal use of the participant voice.⁸ A further limitation is that the tool may prioritise a quantification of quality over content, which can lead to questionable interpretations of quality. Another limitation is that although it is promoted as a good tool for novice researchers, in fact the researchers need help in determining what constitutes good quality qualitative research.⁸ A new question about the study’s underlying theoretical, ontological and epistemological frameworks could be incorporated to strengthen the tool.⁹

Summary

Four qualitative reporting tools have been introduced with brief discussion of how to use them. Each is tool has its uses, and journals usually request one of these tools be completed when submitting a manuscript for publication. Each tool has strengths and limitations. The overarching limitation is that requiring all researchers to adhere to a single standard is restrictive for such a broad field of enquiry. However, all research should be transparently reported, and each tool presents a solid way of reviewing qualitative research to ensure a study has been rigorously appraised.

References

1. Tong A et al. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-357. https://doi.org/10.1093/intqhc/mzm042
2. O’Brien B et al. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-1251 doi: 10.1097/ACM.0000000000000388
3. Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Health. 2015;13(3):179-187.
4. Critical Appraisal Skills Programme. CASP Qualitative Checklist; 2018. Accessed June 17, 2023. https://casp-uk.net/
5. Walsh S et al. Adherence to COREQ reporting guidelines for qualitative research: a scientometric study in nursing social science. Int J Qual Methods. 2020:19. https://doi.org/10.1177/1609406920982145
6. Johnson J et al. A review of the quality indicators of rigor in qualitative research. Am J Pharm Educ. 2020;84(1):7120. doi: 10.5688/ajpe7120
7. Majid U, Vanstone M. Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools. Qual Health Res. 2018;28(13):2115-2131 doi: 10.1177/1049732318785358
8. Sinclair A et al. Optimal strategies for the diagnosis of acute pulmonary embolism: a health technology assessment. Canadian Agency for Drugs and Technologies in Health; 2018. (CADTH Optimal Use Report, No. 6.3b.) Table A2, Strengths and Limitations of Qualitative Studies using Critical Appraisal Skills Programme (CASP) Qualitative Checklist. Accessed June 17, 2023. https://www.ncbi.nlm.nih.gov/books/NBK538878/table/app31.tab1/
9. Long H et al. Optimising the value of the critical appraisal skills programme (CASP) tool for quality appraisal in qualitative evidence synthesis. Res Methods Med Health Sci. 2020;1(1):31-42. doi:10.1177/2632084320947559