Qualitative research review guidelines – RATS

R

Relevance of study question

Is the research question interesting?

Research question explicitly stated

Is the research question relevant to clinical practice, public health, or policy?

Research question justified and linked to the existing knowledge base (empirical research, theory, policy)

A

Appropriateness of qualitative method

Is qualitative methodology the best approach for the study aims?
Interviews: experience, perceptions, behaviour, practice, process
Focus groups: group dynamics, convenience, non-sensitive topics
Ethnography: culture, organizational behaviour, interaction
Textual analysis: documents, art, representations, conversations

Study design described and justified e.g., why was a particular method (i.e., interviews) chosen?

T

Transparency of procedures

Sampling

Are the participants selected the most appropriate to provide access to type of knowledge sought by the study?
Is the sampling strategy appropriate?

Criteria for selecting the study sample justified and explained
theoretical: based on pre conceived or emergent theory
purposive: diversity of opinion
volunteer: feasibility, hard-to-reach groups

Recruitment

Was recruitment conducted using appropriate methods?
Is the sampling strategy appropriate?

Details of how recruitment was conducted and by whom

Could there be selection bias?

Details of who chose not to participate and why

Data collection

Was collection of data systematic and comprehensive?

Method (s) outlined and examples given (e.g., interview questions)

Are characteristics of the study group and setting clear?

Study group and setting clearly described

Why and when was data collection stopped, and is this reasonable?

End of data collection justified and described

Role of researchers

Is the researcher (s) appropriate? How might they bias (good and bad) the conduct of the study and results?

Do the researchers occupy dual roles (clinician and researcher)?
Are the ethics of this discussed?Do the researcher(s) critically examine their own influence on the formulation of the research question, data collection, and interpretation?

Ethics

Was informed consent sought and granted?

Informed consent process explicitly and clearly detailed

Were participants’ anonymity and confidentiality ensured?

Anonymity and confidentiality discussed

Was approval from an appropriate ethics committee received?

Ethics approval cited

S

Soundness of interpretive approach

Analysis

Is the type of analysis appropriate for the type of study?
thematic: exploratory, descriptive, hypothesis generating
framework: e.g., policy
constant comparison/grounded theory: theory generating, analytical

Analytic approach described in depth and justified

Are the interpretations clearly presented and adequately supported by the evidence?

Are quotes used and are these appropriate and effective?

Indicators of quality: Description of how themes were derived from the data (inductive or deductive)
Evidence of alternative explanations being sought
Analysis and presentation of negative or deviant cases
Description of the basis on which quotes were chosen
Semi-quantification when appropriate
Illumination of context and/or meaning, richly detailed

Was trustworthiness/reliability of the data and interpretations checked?

Method of reliability check described and justified
e.g., was an audit trail, triangulation, or member checking employed? Did an independent analyst review data and contest themes? How were disagreements resolved?

Discussion and presentation

Are findings sufficiently grounded in a theoretical or conceptual framework?

Is adequate account taken of previous knowledge and how the findings add?

Findings presented with reference to existing theoretical and empirical literature, and how they contribute

Are the limitations thoughtfully considered?

Strengths and limitations explicitly described and discussed

Is the manuscript well written and accessible?

Evidence of following guidelines (format, word count)
Detail of methods or additional quotes contained in appendix
Written for a health sciences audience

Are red flags present? these are common features of ill conceived or poorly executed qualitative studies, are a cause for concern, and must be viewed critically. They might be fatal flaws, or they may result from lack of detail or clarity.

Grounded theory: not a simple content analysis but a complex, sociological, theory generating approachJargon: descriptions that are trite, pat, or jargon filled should be viewed sceptically
Over interpretation: interpretation must be grounded in "accounts" and semi-quantified if possible or appropriate
Seems anecdotal, self evident: may be a superficial analysis, not rooted in conceptual framework or linked to previous knowledge, and lacking depth
Consent process thinly discussed: may not have met ethics requirements
Doctor-researcher: consider the ethical implications for patients and the bias in data collection and interpretation

The RATS guidelines modified for BioMed Central are copyright Jocalyn Clark, BMJ. They can be found in Clark JP: How to peer review a qualitative manuscript. In Peer Review in Health Sciences. Second edition. Edited by Godlee F, Jefferson T. London: BMJ Books; 2003:219-235