Reviewer Instructions: Research Articles

Analytics operates at the intersection of many fields, including writing studies, educational measurement, massive data analysis, digital learning ecologies, and ethical philosophy. Intended to give voice to an emerging community, the journal is devoted to programs of research providing evidence of fair, reliable, and valid analytics.

We have provided detailed Author Guidelines for Research Articles. Based on those guidelines, the categories below ask reviewers to make a series of judgements on each section of the manuscript. After providing scores, reviewers are then asked to provide notes that justify and explain their assessment of the given section of the manuscript.

Reviewers should enact Anti-Racist Scholarly Reviewing Practices, which The Journal of Writing Analytics and the WAC Clearinghouse endorse. Further, reviewer notes should work to enhance the writing analytics community and strengthen the research developing from this emerging field by:

Evaluating the manuscript rather than the manuscript’s writer. When writing your reviewer notes, consider crafting sentences that use grammatical subjects like the manuscript, the text, or the article, rather than grammatical subjects like the author, the writer, or you.
Balancing critique with suggestion. When offering a critique of the manuscript, please translate the critique into a suggestion for how the manuscript could be improved, taking time to explain the basis for the criticism and the rationale for the suggestion.
Defining key terms and using accessible language. Since the journal appeals to a multidisciplinary audience, please take extra time and extra space in your reviewer notes to establish any key constructs and foundational concepts that need additional attention in the manuscript.
Using local context and individual perspective to ground feedback and qualify reviewer notes. The review process should contribute to an exchange of perspective that strengthens research and knowledge-making in the writing analytics community. Help manuscript authors understand your perspective by providing context behind your notes and working to mitigate any biases.
Recommending supplementary research sources that build upon existing knowledge, engage a diverse body of scholarship, validate alternative ways of meaning making, and recognize alternative forms of expertise. Please support and strengthen the research base of the manuscript by pointing to ways the manuscript might more inclusively draw upon a wider range of extant work across disciplines.

Submission Title

Publishing Recommendation

Please select only one recommendation:

Publish as is

Publish with minor changes

Revise and resubmit

Reject

Please briefly explain your decision:

Comment Category 1: Abstract

The abstract (250 words or less) is written in accessible language, presents a structured overview of the study, and uses the following headings: literature review, research questions, methodology, results, discussion, and conclusions.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Abstract:

Comment Category 2: Background

The background section positions the study by including information on the following: the study's focus; the study's relevance to readers of The Journal of Writing Analytics; and the research question or problem statement that the study addresses. The background section concludes with a brief overview of the significance of the study.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Background:

Comment Category 3: Literature Review

The literature review positions the reported research in the context of work by other researchers, identifies needed advances, and establishes justification for the present study.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Literature Review:

Comment Category 4: Research Questions

The research questions are framed in terms of the needed work identified in the literature review. The research questions align the needed work with the chosen research methodology.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Research Questions:

Comment Category 5: Research Methodology

The research methodology section consists of a comprehensive description of how the study was executed. The section is written so that others should be able to replicate the study. The structure includes information on design, sequence, collection, sampling, and analytic method. When describing data collection, the article provides information about the following aspects, as appropriate for the study.

Research Design: The manuscript provides a description of the general study design and positions the study within known and developing programs of research.
Sampling Plan: The manuscript describes the method for securing the sampling plan and includes a statement indicating that the study was approved by a Research Ethics committee or that it was exempt from approval. The manuscript discloses the number of participants, along with any relevant demographic information or other characteristics. The manuscript uses power analysis used to support inferences from the study sample to the general population at hand and reports attrition.
Instruments: The manuscript identifies the analytic methods used in the study and the aims of these methods. The manuscript describes the kind of information collected and establishes necessary qualifications to aid later discussion.
Procedures: The manuscript provides a step-by-step description of how data were collected.
Data Analysis: The manuscript describes how the data were analyzed and provide justification for the analytic methods used.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Research Methodology:

Comment Category 6: Results

The results section provides evidence that the research questions have been addressed. Results are reported according to standards identified by the American Psychological Association in terms of sample size, disaggregation by sub-group, descriptive and inferential statistics, and effect size. Foundational categories of evidence related to fairness, validity, and reliability are essential. High-quality visualizations are used to communicate results.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Results:

Category 7: Discussion

The discussion section provides interpretation, use, and consequences of the results. Attention is also given to claims, warrants, and qualifications regarding the evidence presented.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Discussion:

Category 8: Conclusions

The conclusions section provides justification that the results have answered the research questions with any attendant qualifications. Special attention is paid to applications, especially to groups of diverse learners, and to fairness as a category of evidence.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Conclusion:

Category 9: Directions for Further Research

The manuscript closes with directions for further research intended to advance the body of knowledge and program of research reported in the study.

Very Strongly Agree	Strongly Agree	Agree	Disagree	Strongly Disagree	Very Strongly Disagree
6	5	4	3	2	1

Notes on Directions for Further Research:

Reviewer Notes

Please provide any additional integrative notes on the manuscript related to your publishing recommendation.