Recommendation 4. Train and Mentor Reviewers

Think about how you find your reviewers. Are they mostly from your editorial board or your professional networks? How diverse are your editorial board and professional network? Are they made up of mostly scholars from your same geographic region, career level, and type of institution?

If so, you aren’t alone. Many editorial boards are not as diverse as they could be (Bhaumkik & Jagnoor 2019; Palser, Lazerwitz & Fotopoulous 2022), and it is human nature to identify with people who are similar to us (Montoya, Horton & Kirchner 2008). But think deeply about the benefits that come with tapping into a diverse reviewer pool.

As noted in Recommendation 3, it is important to approach the recruitment and selection of reviewers equitably and inclusively, which can include creating pathways for researcher participation in the editorial and peer review processes. This section looks at how training, mentoring, and co-reviewing opportunities can engage early career, emerging, historically marginalized, and non-scientist researchers—helping to diversify your reviewer pool and ensuring that more reviewers are trained in the best practices of peer review.

Provide accessible learning tools

Reviewing is a skill that must be learned, and not all potential reviewers will have the opportunity to get formal training or quality mentorship in peer review. Below are several free training options to help potential reviewers develop reviewing skills and support early career researchers and others who lack mentor opportunities.

• ACS Reviewer Lab is a free, on-demand training course that provides real-life guidance on how to navigate tricky ethical situations, identify core criteria for evaluating manuscripts, and write a first-rate review. ACS Reviewer Lab is available in both English and Chinese.

• American Psychological Association’s “Learn How to Review a Manuscript” webpage includes a host of online resources and tutorials that provide guidance on reviewing both quantitative and qualitative research papers. While focused on psychology manuscripts, their instructional value may extend to reviewing research papers more broadly. The resources include both video and written content and are freely available on the webpage.

• ASHA Journals Academy: Peer Review Excellence Program and their PREP Development Modules offer free resources on the peer review process, procedures, and policies both broadly and within the ASHA journal program.

• Canadian Science Publisher: “Peer Review Guidelines for Non-Academic Reviewers” has an excellent overview of the peer review process aimed at individuals who are not in academia and also links out to resources on peer reviewing for researchers.

• Nature Masterclasses’s Focus on Peer Review, a free, on-demand course led by Nature editors and active researchers, provides a comprehensive and insightful overview of peer review.

• Wiley Author Services: “How to Perform a Peer Review ” provides comprehensive and non-discipline-specific overviews and resources on peer reviewing, including tips for how to review video and audio content.

Develop peer review mentorship programs and allow for co-reviewing.

Your journal can also help potential reviewers develop—or improve—their skills by giving them hands-on reviewing experience through mentorship programs and co-reviewing.

Such programs explicitly encourage broader involvement from the research community by pairing less experienced reviewers with senior reviewers to support the learning process. These programs benefit new reviewers and editors alike by offering expert training and, therefore, expanding the pool of qualified reviewers.

Mentorship Programs. Establishing a mentorship program helps to formalize a useful practice. Such programs provide valuable experience without potentially compromising the ethics of the peer review process, by pairing someone new to peer review with an experienced reviewer. Examples of mentorship programs include:

• American Psychological Association Journals Mentorship Program

• BMC Peer Review Mentoring

• Nature Communication Early Career Researchers Peer Review programme

• PREreview Open Reviewers

Co-Reviewing. Anecdotally, we know that many researchers learn how to review informally from their academic mentors rather than through a formal training program. This often takes the form of co-reviewing—where a senior scholar, who is the official reviewer of the paper may ask their student, post-doc, or another scholar to help review all or part of that manuscript as a training exercise. The senior scholar oversees the review and provides the new reviewer with valuable feedback on their review. Creating a co-reviewing policy and program to encourage and support the practice helps ensure (a) that all reviewers receive credit for their work and (b) that there is transparency in the peer review process. For more information on how co-reviewing improves equity in peer review, see Recommendation 5. Examples of co-reviewing programs include:

• Co-reviewing at IOPP

• A Review of Coreviewing Experiments at Elsevier (Mehmani, 2019)

Feedback to Reviewers. If you do not have the means to develop a formal mentorship or co-reviewing program, another option is to provide feedback on reviews. Feedback can benefit new and experienced reviewers alike, and can help ensure that all reviewers continue to improve and sharpen their skills.

• It is crucially important that editors have the capacity to identify good reviews, so it may be useful to provide training to newer editors to help them identify the qualities of a strong review and how best to provide constructive feedback to reviewers when they see reviews that could be improved. Editors can also respectfully encourage reviewers to participate in training and utilize tools such as those mentioned in the previous section.

• Reviewers can also benefit from being invited to review a revision and observe how authors responded to their feedback and that of other reviewers. Likewise, by sharing their decision letter with all reviewers, editors ensure that everyone has a fuller view of the editorial process, providing further education.

Make it easy to include new reviewers

Prospective reviewers, especially those from persistently marginalized groups, might not be aware that they are qualified to review for your journal. If there is not currently a way to apply to be a reviewer, you can create a reviewer application process for people from outside of your existing pool of reviewers. To ensure a broad array of applicants, you can post a Call for Reviewers on the journal website (as exemplified in BMJ Open Quality) and your social media feeds, to indicate that your journal is looking for reviewers and that there are opportunities available for new reviewers.

• You may want to host a journal-specific promotion that outlines the specific qualifications a potential reviewer should have, as in this example from Gender in Management.

• Or you may want to create a more general review application process, as the American Psychological Association and the American Society of Civil Engineers (ASCE) did, with broader qualifications for becoming a reviewer.

Consider programs for universal accessibility

To create a welcoming and inclusive environment for reviewers with disabilities, consider seeking input on the accessibility of all aspects of the reviewing process, including instructional materials, submission programs, manuscript display, and published journal content. The recently released Toolkit for Disability Equity contains a wealth of guidance on ensuring accessibility in publishing environments. Some key considerations are detailed here:

• Be mindful of the language used in your guidelines and instructions for your reviewers. Multilingual researchers may have more difficulty with overly technical language, so plain language should be used whenever possible. For tips on writing in plain language, refer to this guide from the Plain Language Action and Information Network.

• It is important that all visual elements in the manuscript be accessible to all reviewers so they can view the full content of the paper they are reviewing. Harvard University has an easy-to-follow Digital Accessibility Digest for creating accessible charts and graphs, which can be shared with your authors.

• Ensure that your journal’s content is accessible and can easily be read by a screen reader. You can find excellent resources about best practices for accessibility in publishing at the DAISY Consortium.