Peer Review 2.0: Navigating the Publishing Landscape in the Age of AI

The intersection of artificial intelligence (AI) and scholarly publishing has received increasing attention in recent years, sparking both excitement and concern within the scholarly research community. For some, AI represents a leap forward, providing tools to streamline the peer review process. For others, there are concerns about ethics, transparency, and the very nature of human-centric scholarship. So how should reviewers and publishers adapt to this new paradigm?

The Current Landscape

Recent discussions among scholarly publishing and editorial leaders highlight both the opportunities and the challenges AI poses for the academic world. “AI can be leveraged at many different stages in the lifecycle of an article throughout the scholarly publishing process, from submission to publication and beyond,” says Sonja Krane, Ph.D., Senior Associate Publisher at ACS Publications. Dr. Krane also cautions that “using these technologies in a way that advances science and minimizes bias will be a persistent challenge."

There is no question that AI-driven tools such as ChatGPT are transforming the learning experience and enhancing research productivity. Osvaldo N. Oliveira Jr., Ph.D., Executive Editor of ACS Applied Materials & Interfaces, and his colleagues recently demonstrated the potential of such tools by combining network science and natural language processing (NLP) methods to analyze research articles and identify key topic areas within the field of applied materials science.

“My prediction is that in a few years, the publishing business may be altered drastically, as the machines (intelligent systems) will be able to produce science themselves,” notes Dr. Oliveira. “This changes everything as the papers will have to be machine-readable, and many other issues will appear.”

Want to learn more about AI’s impact in peer review and publishing? On September 27, Join Dr. Krane and Dr. Oliveira for an interview and audience Q&A session on how scientific and academic communities are navigating AI’s current and future role in scholarly publishing.

Opportunities

• Efficiency: AI tools can expedite tasks such as manuscript matching, helping editors find the right reviewers or flagging submissions that might be problematic.
• Support: For reviewers, especially those who are new, AI can serve as a supplementary tool to guide the review process, ensuring that all necessary aspects of a paper are scrutinized.

Challenges

• Ethical Considerations: The use of AI in the peer review process isn't universally accepted. Some organizations, such as NIH, have stated that using AI tools like generative models might breach confidentiality.
• Reliability: How dependable are AI models in assessing the quality of research? There are worries about biases in AI algorithms, potentially amplifying existing issues in peer review.

Ethical Guidelines and Best Practices

Drawing from a plethora of resources, including guidelines from ACS Publications, COPE, and JAMA, there's a consensus about the importance of transparency.

• Declaration: If AI tools, including language models like ChatGPT, are used in any part of the research or review process, this should be explicitly declared.
• Avoid Over-Reliance: AI should complement, not replace, human judgement. The nuanced, contextual understanding that humans bring to the peer review process remains irreplaceable.
• Awareness and Training: Educational initiatives should be introduced to inform reviewers, authors, and editors about the potential pitfalls and advantages of using AI in publishing.

The Future and Recommendations

The integration of AI into peer review is still evolving. Some champion AI as a relief for overburdened reviewers and editors. Others worry it may introduce new biases.

Here are some recommendations as AI's publishing role continues unfolding:

• Collaboration and Dialogue: Encourage collaboration between AI experts, publishers, and the academic community.
• Stay Updated: With rapid advancements in AI, the capabilities and potential applications of these technologies in peer review will continue to evolve. Regularly consulting resources like ACS Axial and The Scholarly Kitchen will be essential.
• Ethical Vigilance: Always prioritize ethical considerations. Drawing on guidelines from trusted institutions will be pivotal in ensuring that AI's integration into publishing respects the sanctity and integrity of academic work.

The conversation surrounding AI's role in academic publishing is dynamic and multi-faceted. By staying informed, engaged, and ethically vigilant, we can harness the power of AI to enhance the peer review process while preserving the core values of academic scholarship.