Problem Statement, Conceptual Framework, and Research Question

1. The introduction builds a logical case and context for the problem statement
2. The problem statement is clear and well-articulated
3. The conceptual framework is explicit and justified
4. The research question (research hypothesis where applicable) is clear, concise, and complete
5. The variables being investigated are clearly identified and presented

Reference to the Literature and Documentation

6. The literature review is up-to-date
7. The number of references is appropriate and their selection is judicious
8. The review of the literature is well integrated
9. The references are mainly primary sources
10. Ideas are acknowledged appropriately (scholarly attribution) and accurately
11. The literature is analyzed and critically appraised

Relevance

12. The study is relevant to the mission of the journal or its audience
13. The study addresses important problems or issues; the study is worth doing
14. The study adds to the literature already available on the subject
15. The study has generalizability because of the selection of subjects, setting, and educational intervention or materials

Research Design

The research design:

16. Is defined and clearly described, and is sufficiently detailed to permit the study to be replicated
17. Is appropriate (optimal) for the research question
18. Has internal validity, potential confounding variables or biases are addressed
19. Has external validity, including subjects, settings, and conditions
20. Allows for unexpected outcomes or events to occur
21. And conduct of the study are plausible

Instrumentation, Data Collection, and Quality Control

22. The development and content of the instrument are sufficiently described or referenced, and are sufficiently detailed to permit the study to be replicated
23. The measurement instrument is appropriate given the study’s variables; the scoring method is clearly defined
24. The psychometric properties and procedures are clearly presented and appropriate
25. The data set is sufficiently described or referenced
26. Observers or raters were sufficiently trained
27. Data quality control is described and adequate

Population and Sample

28. The population is defined clearly, both for subjects (participants) and stimulus (intervention), and is sufficiently detailed to permit the study to be replicated
29. The sampling procedures are sufficiently described
30. Subject samples are appropriate to the research question
31. Stimulus samples are appropriate to the research question
32. Selection bias is addressed

Data Analysis and Statistics

33. Data analysis procedures are sufficiently described, and are sufficiently detailed to permit the study to be replicated
34. Data analysis procedures conform to the research design; hypotheses, models, or theory drives the data analyses
35. The assumptions underlying the use of statistics are fulfilled by the data, such as measurement properties of the data and normality of distributions
36. Statistical tests are appropriate (optimal)
37. If statistical analysis involves multiple tests or comparisons, proper adjustment of significance level for chance outcomes was applied
38. Power issues are considered in statistical studies with small sample sizes
39. In qualitative research that relies on words instead of numbers, basic requirements of data reliability, validity, trustworthiness, and absence of bias were fulfilled

Reporting of Statistical Analyses

40. The assumptions underlying the use of statistics are considered, given the data collected
41. The statistics are reported correctly and appropriately
42. The number of analyses is appropriate
43. Measures of functional significance, such as effect size or proportion of variance accounted for, accompany hypothesis-testing analyses

Presentation of Results

44. Results are organized in a way that is easy to understand
45. Results are presented effectively; the results are contextualized
46. The results are complete
47. The amount of data presented is sufficient and appropriate
48. Tables, graphs, or figures are used judiciously and agree with the text

Discussion and Conclusion: Interpretation

49. The conclusions are clearly stated; key points stand out
50. The conclusions follow from the design, methods, and results; justification of conclusions is well articulated
51. Interpretations of the results are appropriate; the conclusions are accurate (not misleading)
52. The study limitations are discussed
53. Alternative interpretations for the findings are considered
54. Statistical differences are distinguished from meaningful differences
55. Personal perspectives or values related to interpretations are discussed
56. Practical significance or theoretical implications are discussed; guidance for future studies is offered

Title, Authors, and Abstract

57. The title is clear and informative
58. The title is representative of the content and breadth of the study (not misleading)
59. The title captures the importance of the study and the attention of the reader
60. The number of authors appears to be appropriate given the study
61. The abstract is complete (thorough); essential details are presented
62. The results in the abstract are presented in sufficient and specific detail
63. The conclusions in the abstract are justified by the information in the abstract and the text
64. There are no inconsistencies in detail between the abstract and the text
65. All of the information in the abstract is present in the text
66. The abstract overall is congruent with the text; the abstract gives the same impression as the text

Presentation and Documentation

67. The text is well written and easy to follow
68. The vocabulary is appropriate
69. The content is complete and fully congruent
70. The manuscript is well organized
71. The data reported are accurate (e.g., numbers add up) and appropriate; tables and figures are used effectively and agree with the text
72. Reference citations are complete and accurate

Scientific Conduct

73. There are no instances of plagiarism
74. Ideas and materials of others are correctly attributed
75. Prior publication by the author(s) of substantial portions of the data or study is appropriately acknowledged
76. There is no apparent conflict of interest
77. There is an explicit statement of appropriate regulatory and governance approval(s) for studies directly involving human subjects or data about them, or a statement why this is/these are not required[1]

[1] The wording of point 77 has been slightly amended with the permission of the AAMC.