Recommendation 2. Collect and Report Demographic Data

Establishing concrete metrics to measure progress is a critical element in creating transparency and promoting accountability. You can’t know what you don’t measure. If you want to increase the diversity of your publication’s editorial board and peer reviewer pool, gathering demographic data is an important diagnostic step. From there, you can set specific goals and report on the progress and success of your efforts. Mukharji et al. (2020) provide a roundtable discussion on demographic data collection as integral to diversifying scholarly disciplines.

• Gather baseline demographic data. You may have a general sense of the demographic makeup of your journal’s editorial board and which identities are over- and underrepresented, but without asking members to self-identify, your information will likely be incomplete and/or inaccurate. Some areas of diversity may be invisible and how people choose to identify should not be assumed.

  Ensure that you take a respectful approach to gathering demographic data. It should be both anonymous and optional and should include a clear rationale that you are gathering this information as an effort to be inclusive of all identities, particularly individuals from persistently marginalized groups (see JAMA’s updated guidance on reporting race and ethnicity; Flanagin et al., 2021).

  Asking members of different stakeholder groups—editors, reviewers, and authors—to disclose how they identify (for example, in terms of their racial, ethnic, gender, age, sexual orientation, and/or ability identities) may open up surprising insights and illuminate opportunities for improvement. It would be helpful to request data from all authors, rather than simply the corresponding author. Though there will not be a one-to-one relationship between authors and submissions in your analysis, this will give you a better understanding of the diversity of your author pool, which is itself an important measure of the success of your journal’s inclusivity efforts.

• Decide which terms to use. The Joint commitment for action on inclusion and diversity in publishing of the Royal Society of Chemistry recommends using a set of standardized terms when gathering demographic data from journal stakeholders. However, we advise carefully considering the needs of your particular organization and the populations your publications serve when deciding on terms to use.

  The terminology for ethnic, racial, gender, and other forms of identification vary by region and may evolve rapidly over time. Commit to revisiting your language regularly to ensure that you are being both as inclusive and as targeted as possible.

• Methods for gathering data. Broadly speaking, there are two ways to gather demographic data from your journal’s stakeholders—either within your manuscript submission system, or externally.

  • Manuscript submission systems. Collecting data directly in the manuscript submission system offers many benefits including a centralized location for users’ data, connection with submission and reviewing activities, and typically higher response rates than external surveys. The major manuscript submission platforms offer ways to capture self-identified demographic data associated with the user’s account. However, the ease of setting up such surveys and reporting meaningful data may vary by system and present different challenges including system limitations in capabilities and customization. Editors and publishers should engage their platform providers and communicate technical expectations that will support their publishing and editorial missions (see, for example, a collaborative initiative between Aries and the American Psychological Association; Einhorn et al., 2021).

  • External forms or surveys. Cell Press has implemented an interesting workaround, asking authors of accepted manuscripts to fill out a form with diversity information, as described in an interview with (then) Vice President of Editorial Deborah Sweet (Meadows, 2021). This voluntary form, attached to the submission but not delivered as a survey produced by the submission platform itself, addresses potential concerns that demographic information could be used to influence the peer review process, by waiting until after acceptance to collect data. However, it relies on the corresponding author to fill out the form on behalf of all authors, which may make the information less reliable.

  • Membership databases. A third option, of potential interest for membership-oriented society publishers, is to create a bridge between a society’s association management system and the manuscript submission system. You may then be able to access demographic information that is already being gathered by your organization for other purposes, thus reducing the burden on the user. Talk to your Information Technology (IT) and/or membership departments to determine what is possible. Data gathered in this way should be separated from submitted manuscript data to prevent the appearance of bias in selecting member-submitted over nonmember-submitted manuscripts. As described in Giampoala et al. (2021), the American Geophysical Union (AGU) has been monitoring and reporting reviewer demographics for the past five years to determine a baseline and develop goals for reviewer diversity by merging their editorial data with their membership data, where most members have provided their gender identity and year of birth, and some have provided their race and ethnicity.

  • Name inference tools. If user-provided data are not available, it may be possible to use a name-to-gender inference tool that takes an input of first names and classifies each name by its most probable gender using a binary gender system (Sebo, 2021). Additionally, some of these tools can infer other demographic components, including name origin and ethnicity (Sebo, 2023). Before using such a tool, users should be aware of its limitations (most notably, excluding nonbinary people), shortcomings, and accuracy thresholds which can result in an undeterminable or mis-inferred characteristic. Such tools tend to be more accurate with Western names and least accurate with Asian names, and other cultural contexts may produce varying results (Santamaría, 2018). Despite these limitations, in the absence of other available data, having a system-inferred distribution of stakeholders with high accuracy can prove useful for benchmarking current demographics, identifying areas for improvement, and determining tactics to make progress on goals. (See example tools in the Resources section under Name Inference Tools.)

• Privacy concerns. Collecting and measuring demographic data should be done in a way that respects privacy. The data collected should be used to promote diversity, equity, and inclusion, which means that measures should be taken to guard against discriminatory practices.

  When collecting demographic data from users, you must consider the privacy issues, including complying with legislation such as the General Data Protection Regulation (GDPR), a set of regulations aimed at protecting the privacy of European Union (EU) citizens, even if your editors and authors are not European. Here are some key points to consider:

  • Consent: Users must provide explicit consent to the collection and processing of their personal data. Publishers should make clear what data are being collected and how the data will be used, and allow users to opt out if they do not wish to provide their data.

  • Anonymization: Personal data should be anonymized whenever possible. When collecting demographic data, consider using aggregated data instead of individual data to preserve anonymity.

  • Security: Peer review submission systems must ensure that personal data are stored securely and that there are measures in place to prevent unauthorized access, theft, or loss. Check with your vendors to ensure that they are in compliance with GDPR and other obligations.

  • Right to Access and Erasure: Users have the right to access their personal data and request its deletion. Provide a clear process for users to access their data and request its removal.

  • Limit Access: Only allow staff who are actively tasked with data reporting and analysis to access personal data, preferably anonymized. Stakeholders across the organization and journal may request non-anonymized data on editors; however, proper governance for data security should be followed to determine who has access to what data, and no non-anonymized data should be released to the wider organization. You may want to consider third-party vendors who can manage your data wisely.

  • See more guidance on maintaining GDPR compliance here (Wolford, 2020).

• Take a targeted approach. As you begin to gather demographic data, consider a staged rollout to the highest impact stakeholder groups first. For example, collect data from editors, then editorial board members, before extending the effort to the broader pool of authors or reviewers. This allows you to receive feedback from your more involved stakeholders and make adjustments to your messaging and processes accordingly.

• Gather and report interim data. Once you have the data, it’s time to use it! Set realistic but ambitious goals to increase the presence of underrepresented groups in your reviewer pool and on your editorial boards, particularly in prestigious or powerful roles. Communicate these goals publicly; this will hold your publications accountable, inspire further action, and set a precedent for other journals in your field. You may want to consult with an expert in survey methods to review your survey and results to ensure accurate reporting, especially for larger author and reviewer pools. Reporting your progress regularly also allows you to adjust your tactics as needed and, importantly, to celebrate the victories.

  Providing directed feedback to the decision-makers of your journals will help you achieve your goals. Consider adding a standing agenda item on your DEIA efforts to each editorial board meeting. In your communications to editors, remind them to consider diversity in their peer review networks at the manuscript handling level and when recommending colleagues for recruited positions.