United States

&lt;p&gt;Pawin Taechoyotin,&lt;sup&gt;1&lt;/sup&gt; Daniel E. Acuna&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective &lt;/h4&gt;
&lt;p&gt;Nowadays, there is an excessive load on human reviewers to the point where the quality of peer reviews is often compromised. To alleviate this excessive load, we explored the possibility of a multiagent AI peer review panel that we have developed that considers text, figures, and citations to produce peer reviews. This study is based on the advancement of large language models&lt;sup&gt;1,2&lt;/sup&gt; and the study of AI agents for peer review,&lt;sup&gt;3&lt;/sup&gt; which so far only utilize the textual content within the manuscript.&lt;/p&gt;
&lt;h4&gt;Design&lt;/h4&gt;
&lt;p&gt; The AI panel consisted of a leader agent, an experiment agent, an impact agent, and a clarity agent. Each agent utilized zero-shot prompting to the LLM model Claude Sonnet 3.5. Prompt engineering was used to craft the system and task prompt. The system prompts guided the agents to assume their specific role, and the task prompts provided clear instructions on the tasks to perform. Novelty was assessed by comparing the manuscript with similar articles from an external database of published literature. The manuscript, novelty assessment, and figures were sent to each agent for analysis, and the leader agent compiled the information from all agents, requested clarifications if needed, and produced the final review. Our AI multiagent panel was evaluated and compared with human reviews as well as simple prompt-based review&lt;sup&gt;2&lt;/sup&gt; and findings of 2 previous studies.&lt;sup&gt;1,3&lt;/sup&gt; Evaluation was done by graduate students via an arena where pairs of systems were selected based on their Elo score. The Elo score is a measure of the relative capability of each system based on their win/loss in the arena against other systems and was calculated using the Bradley-Terry model. The system names were hidden from the participants to reduce bias. The participants evaluated the reviews based on technical quality, constructiveness, clarity, and overall assessment before making their choice.&lt;/p&gt;
&lt;h4&gt;Results&lt;/h4&gt;
&lt;p&gt; Eleven graduate students in the computer science department participated in this study, which resulted in 140 matches between pairs of systems. The systems with similar Elo scores were most likely to be paired together. Mean Elo scores of the reviews produced from our system were higher than those of human review scores (&lt;b&gt;Table 25-0971&lt;/b&gt;). Further analysis showed that reviews produced by our system included more detailed assessment of the limitations and possible improvements of the manuscript than reviews produced by humans. However, our system tended to overly praise work, which humans rarely do.&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/e9b3e219d84889a96133d3b20f6fe73d.png)

&lt;h4&gt;Conclusions&lt;/h4&gt;
&lt;p&gt; Our AI peer review system demonstrated potential in aiding the peer review process of publications as an addition to human reviews. It might act as a quick response to authors while human reviewers are reviewing the manuscript. Although there might be bias within the review, the author can choose to verify and utilize comments from this quick feedback as needed. Harm or the potential for negative effects were not studied. We acknowledge that the number of participants in this study was small. Thus, a larger study might have different results from our findings.&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. Liang W, Zhang Y, Cao H, et al. Can large language models provide useful feedback on research papers—a large-scale empirical analysis. &lt;i&gt;NEJM AI&lt;/i&gt;. 2024;1(8):AIoa2400196. doi:10.1056/AIoa2400196&lt;/p&gt;
&lt;p&gt;2. Wang H, Fu T, Du Y, et al. Scientific discovery in the age of artificial intelligence. &lt;i&gt;Nature&lt;/i&gt;. 2023;620(7972):47-60. doi:10.1038/s41586-023-06221-2&lt;/p&gt;
&lt;p&gt;3. D’Arcy M, Hope T, Birnbaum L, Downey D. Marg: multiagent review generation for scientific papers. &lt;i&gt;arXiv&lt;/i&gt;. Preprint posted online January 8, 2024. doi:10.48550/arXiv.2401.04259&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;Science of Science and Computational Discovery Lab, Department of Computer Science, University of Colorado Boulder, Boulder, CO, US, daniel.acuna@colorado.edu.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt;
&lt;p&gt; Daniel E. Acuna reported being the founder of ReviewerZero AI LLC (&lt;a href=&quot;https://www.reviewerzero.ai&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://www.reviewerzero.ai&lt;/span&gt;&lt;/a&gt;), a company focused on research integrity solutions, which may be relevant to the topic of this publication.&lt;/p&gt;


The 10th Peer Review Congress

[V] Use of an AI Peer Review Panel to Assess Clarity, Novelty, and Impact

artificial intelligence and large language models

peer review process and models

peer review

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Pawin Taechoyotin,<sup>1</sup> Daniel E. Acuna<sup>1</sup></p>
<h4>Objective </h4>
<p>Nowadays, there is an excessive load on human reviewers to the point where the quality of peer reviews is often compromised. To alleviate this excessive load, we explored the possibility of a multiagent AI peer review panel that we have developed that considers text, figures, and citations to produce peer reviews. This study is based on the advancement of large language models<sup>1,2</sup> and the study of AI agents for peer review,<sup>3</sup> which so far only utilize the textual content within the manuscript.</p>
<h4>Design</h4>
<p> The AI panel consisted of a leader agent, an experiment agent, an impact agent, and a clarity agent. Each agent utilized zero-shot prompting to the LLM model Claude Sonnet 3.5. Prompt engineering was used to craft the system and task prompt. The system prompts guided the agents to assume their specific role, and the task prompts provided clear instructions on the tasks to perform. Novelty was assessed by comparing the manuscript with similar articles from an external database of published literature. The manuscript, novelty assessment, and figures were sent to each agent for analysis, and the leader agent compiled the information from all agents, requested clarifications if needed, and produced the final review. Our AI multiagent panel was evaluated and compared with human reviews as well as simple prompt-based review<sup>2</sup> and findings of 2 previous studies.<sup>1,3</sup> Evaluation was done by graduate students via an arena where pairs of systems were selected based on their Elo score. The Elo score is a measure of the relative capability of each system based on their win/loss in the arena against other systems and was calculated using the Bradley-Terry model. The system names were hidden from the participants to reduce bias. The participants evaluated the reviews based on technical quality, constructiveness, clarity, and overall assessment before making their choice.</p>
<h4>Results</h4>
<p> Eleven graduate students in the computer science department participated in this study, which resulted in 140 matches between pairs of systems. The systems with similar Elo scores were most likely to be paired together. Mean Elo scores of the reviews produced from our system were higher than those of human review scores (<b>Table 25-0971</b>). Further analysis showed that reviews produced by our system included more detailed assessment of the limitations and possible improvements of the manuscript than reviews produced by humans. However, our system tended to overly praise work, which humans rarely do.</p>

![](https://assets.underline.io/markdown_image/1/image/e9b3e219d84889a96133d3b20f6fe73d.png)

<h4>Conclusions</h4>
<p> Our AI peer review system demonstrated potential in aiding the peer review process of publications as an addition to human reviews. It might act as a quick response to authors while human reviewers are reviewing the manuscript. Although there might be bias within the review, the author can choose to verify and utilize comments from this quick feedback as needed. Harm or the potential for negative effects were not studied. We acknowledge that the number of participants in this study was small. Thus, a larger study might have different results from our findings.</p>
<h4>References</h4>
<p>1. Liang W, Zhang Y, Cao H, et al. Can large language models provide useful feedback on research papers—a large-scale empirical analysis. <i>NEJM AI</i>. 2024;1(8):AIoa2400196. doi:10.1056/AIoa2400196</p>
<p>2. Wang H, Fu T, Du Y, et al. Scientific discovery in the age of artificial intelligence. <i>Nature</i>. 2023;620(7972):47-60. doi:10.1038/s41586-023-06221-2</p>
<p>3. D’Arcy M, Hope T, Birnbaum L, Downey D. Marg: multiagent review generation for scientific papers. <i>arXiv</i>. Preprint posted online January 8, 2024. doi:10.48550/arXiv.2401.04259</p>
<sup>1</sup>Science of Science and Computational Discovery Lab, Department of Computer Science, University of Colorado Boulder, Boulder, CO, US, daniel.acuna@colorado.edu.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Daniel E. Acuna reported being the founder of ReviewerZero AI LLC (<a href="https://www.reviewerzero.ai"><span class="Hyperlink CharOverride-6">https://www.reviewerzero.ai</span></a>), a company focused on research integrity solutions, which may be relevant to the topic of this publication.</p>


poster

<p>Yidan Sun,<sup>1</sup> Mayank Kejriwal<sup>2</sup></p>
<h4>Objective </h4>
<p>Platforms like OpenReview<sup>1</sup> have become a popular standard for open peer review in conference-oriented fields such as computing.<sup>2</sup> Using OpenReview data from a machine learning conference, we examined (1) whether higher reviewer confidence is associated with more polarized scoring and (2) whether greater score variability is associated with rejection among papers with neutral mean scores. We also explored the assistive potential of a language model to assign sentiment labels to reviews of neutrally rated papers based on text alone.</p>
<h4>Design </h4>
<p>We used the OpenReview application planning interface to collect data on all International Conference on Learning Representations 2019 Conference Track submissions submitted by the September 27, 2018, deadline. The dataset included 4332 reviews across 1419 papers. Each review contained an integer score (1-10), a confidence rating (1-5), and a final accept or reject decision from a meta-reviewer. Analyses were conducted in January 2025. We used all 4332 reviews to assess whether reviewer confidence was associated with score extremity, defined as the absolute difference between a review score and the rubric midpoint (5), using Pearson correlation. We focused on 775 neutral<i> </i>papers—those with all review scores between 4 and 7 (inclusive)—and measured score variability as the SD of scores within each paper. We compared variability between accepted and rejected papers using 2-sided <i>t</i> tests. We then trained a Bidirectional Encoder Representations From Transformers (BERT)–based sentiment classifier<sup>3</sup> using polarized reviews scored less than 4 (negative) and greater than 7 (positive). The model achieved 90.9% accuracy on held-out review-level validation data. We then applied the model to all 2356 reviews from the 775 neutral papers and took the mean of predicted sentiment scores across each paper to produce a paper-level sentiment score. We labeled the top 246 papers—matching the number of actually accepted neutral papers—as “Accept” based on model sentiment. As a baseline, we ranked papers by mean score and labeled the top 246 as “Accept” by score.</p>

<h4>Results</h4> Reviewer confidence was positively correlated with score extremity (<i>r</i> = 0.15 [95% CI, 0.10-0.20]; <i>P</i> &lt; .001). Among the 775 neutral papers, 246 (31.7%) were accepted and 529 (68.3%) were rejected (<b>Table 25-1167</b>). Rejected neutral papers showed greater score variability than accepted papers (mean SD of 0.80 vs 0.59; difference, 0.21 [95% CI, 0.15-0.28]; <i>P</i> &lt; .001), supporting both hypotheses. Of the 246 papers labeled “Accept” by BERT, 137 were accepted. Of the 246 labeled “Accept” by the baseline, 203 were accepted. Among rejected papers, both methods labeled 20 as “Accept.”</p>

![](https://assets.underline.io/markdown_image/1/image/e390106030daa9fb11f54986bee7520b.png)

<h4>Conclusions</h4> Higher reviewer confidence was associated with more decisive scoring. Rejected neutral<i> </i>papers showed greater disagreement among reviewers. Despite moderate scores, review text often conveyed clear positive or negative sentiment, as inferred by the BERT model.</p>
<h4>References</h4>
<p class="abstract_abstract-text ParaOverride-4">1. OpenReview: an open platform for peer review. Accessed July 15, 2025. <a href="https://openreview.net"><span class="Hyperlink CharOverride-6">https://openreview.net</span></a></p>
<p class="abstract_abstract-text ParaOverride-4">2. Ford E. Defining and characterizing open peer review: a review of the literature<i>. J Scholarly Pub</i>. 2013;44(4):311-326. doi:10.3138/jsp.44-4-00</p>
<p class="abstract_abstract-text ParaOverride-3">3. Wolf T, Debut L, Sanh V, et al. Transformers: state-of-the-art natural language processing. In <i>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations</i>. Association for Computational Linguistics; 2020:38-45.</p>
<sup>1</sup>Department of Industrial and Systems Engineering, University of Southern California, Los Angeles, CA, US, yidans@usc.edu; <sup>2</sup>Research Associate Professor, Principal Scientist, Information Sciences Institute, University of Southern California, Los Angeles, CA, US.
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


[V] Reviewer Rating Variability and Confidence and Language Model Sentiment Prediction of Machine Learning Conference Papers

<p>Maud Bernisson<sup>1,2</sup></p>
<h4>Objective</h4>
<p> Researchers have raised concerns about scientific articles written by medical education and communication companies (MECCs) after litigations revealed that MECCs have been using unethical practices like ghostwriting.<sup>1</sup> Little is known about the literature produced by medical writing. While previous methods to assess this literature include surveying authors about ghostwriting,<sup>2,3</sup> this study offers a new method to collect this literature and makes it available to further explore medical writing.</p>
<h4>Design</h4>
<p> The method of collection of this literature used 3 different sources: (1) Web of Science, from which I collected 26,858 DOIs of articles, abstracts, and documents mentioning “medical writ*” in the acknowledgments; (2) the websites of 15 MECCs, found during the mapping of MECCs for another study, that publicize the literature they produce, and from which I collected 4555 additional DOIs; and (3) the Industry Documents Library of University of California, San Francisco (<a href="https://www.industrydocuments.ucsf.edu"><span class="Hyperlink CharOverride-6">https://www.industrydocuments.ucsf.edu</span></a>), in which I found the publication trackers of 5 companies and collected 229 DOIs of publications. The final database includes the metadata of 31,642 documents published between 1972 and 2025, collected from Web of Science, cleaned in R, and analyzed with the library dplyr (version 1.1.4).</p>
<h4>Results</h4>
<p> Transparency practices of disclosure of medical writing differed according to data sources. Only 252 of 4784 publications (5.3%) acknowledged medical writing in the Industry Documents Library and the MECCs websites. In contrast, a timeline of the literature collected from Web of Science identified an increase in transparency practices, with 138 publications acknowledging medical writing in 2008, 1475 in 2016, and 4265 in 2024. Topics also differed according to data sources. Four of the top 10 publishing journals of the MECC literature focused on health economics and outcomes research (<i>Value in Health</i>, <i>Journal of Medical Economics</i>, <i>Current Medical Research and Opinion</i>, <i>Pharmacoeconomics</i>). These 4 journals published 348 documents written by MECCs, which represents 7.6% of the data collected from MECCs websites. In this study, journals that published the MECC-written literature covered a range of topics. Among the journals assessed, a rheumatology journal published 605 MECC-written publications from 2008 to 2024; of these, 524 (86.6%) were meeting abstracts, which represented 43.3% of the 1211 meeting abstracts in the database. Limitations of this study include the difficulty in accessing sources for collecting this literature and that medical writing is not always acknowledged in published articles.</p>
<h4>Conclusions </h4>
<p>This study highlights the need to further explore use and disclosure of medical writing in the health economics and outcomes research literature, as it targets decision-makers such as payers and policymakers. The database used in this study offers different routes to study medical writing in more detail.</p>
<h4>References</h4>
<p>1. Sismondo S. <i>Ghost-Managed Medicine: Big Pharma’s Invisible Hands</i>. Mattering Press; 2018.</p>
<p>2. Flanagin A, Carey LA, Fontanarosa PB, et al. Prevalence of articles with honorary authors and ghost authors in peer-reviewed medical journals. <i>JAMA</i>. 1998;280(3):222-224. doi:10.1001/jama.280.3.222</p>
<p>3. Moffatt B, Elliott C. Ghost marketing: pharmaceutical companies and ghostwritten journal articles. <i>Perspect Biology Med</i>. 2007;50(1):18-31. doi:10.1353/pbm.2007.0009</p>
<sup>1</sup>LISIS, CNRS, INRAE, Université Gustave Eiffel, France, maud.bernisson@cnrs.fr; <sup>2</sup>Institute for Science in Society, Radboud University, the Netherlands.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> The author was part of the NanoBubbles project, which has received Synergy grant funding from the European Research Council, within the European Union’s H2020 programme (grant agreement 951393).</p>
<h4>Additional Information</h4>
<p> The database for this study, including the Web of Science identifiers, DOIs (when they exist), and sources (MECCs websites, Industry Documents Library, and Web of Science) of 31,642 documents involving medical writing, collected on January 9-13, 2025, is available at <a href="https://zenodo.org/records/15537678"><span class="Hyperlink CharOverride-6">https://zenodo.org/records/15537678</span></a>.</p>


[V] Compiling the Publications Produced by Medical Writing

<p>Hannah Grace Varkey,<sup>1</sup> Florian P. Thomas,<sup>1,2</sup> Elli Gourna Paleoudis<sup>3,4</sup></p>
<h4>Objective</h4>
<p> Current scientific practices and culture are largely focused on producing high numbers of publications and positive or newsworthy data. Recently, some journals have dedicated space to the publication of negative results; nevertheless, negative results often remain unpublished, leading to duplication of efforts, lack of transparency, and publication bias. To investigate the evolution of negative data publications over a decade, we reviewed published studies that did not reach statistical significance or reported inconclusive findings or trends.</p>
<h4>Design</h4>
<p> We searched for negative results in 300 publications per year from 2014 to 2024 using Web of Science’s Medline. To limit selection bias, we selected the first 300 articles of each year based on the Web of Science publication date in chronological order, without regard to the scientific or treatment area. Additional requirements included clinical trial or other experimental design, presence of an abstract, and English language; case reports, commentaries, editorials, and case series were excluded. Records from each year were reviewed independently by 2 researchers to ensure reliability and accuracy. Any disagreements were resolved by a tie-breaking third researcher.</p>
<h4>Results</h4>
<p> Over 10 years, we found 2 to 6 articles annually (<span class="CharOverride-4"><b>Figure 25-0879</b></span>) with solely negative results, with a downward trend (from 6 in 2014 to 3 in 2024). This assessment was applied to the primary outcome measure if and as identified. For the purposes of this project, negative results were defined as results that were inconclusive (ie, they did not prove a hypothesis or yield unexplained outcomes) or statistically nonsignificant based on the predefined measure of statistical significance (usually a <i>P</i> value). Identified articles with negative outcomes were most commonly from the fields of cellular biology, biomechanics, and genetics.</p>

![](https://assets.underline.io/markdown_image/1/image/ec21b6408d4e7695210a278eec82df46.png)
	
<h4>Conclusions</h4></span> Recently, there has been increased attention to and support for publishing negative findings in traditional and new scientific journals and search engines (eg, BioNØT<sup>1</sup>) focused specifically on these results. The notion that all research, regardless of outcome, can contribute to scientific advancement is gaining traction among journal editors, funders, and the scientific community overall.<sup>2</sup> This is particularly true in the artificial intelligence and machine learning era, where models are created based on existing (ie, published) data and research is driven by large datasets.<sup>3</sup> Access to negative results would not only limit duplication of effort but also improve planning and interpretation of data. Thus, it may be surprising that publications with negative data remain few in number, and such results are often rejected by journals, forgotten as unpublished data, or buried in supplemental sections.</p>
<h4>References</h4>
<p>1. Agarwal S, Yu H, Kohane I. BioNØT: a searchable database of biomedical negated sentences. <i>BMC Bioinformatics</i>. 2011;12(1):420. doi:10.1186/1471-2105-12-420</p>
<p class="abstract_reference-text ParaOverride-4">2. Sansom O, Bogani D, Reichenbach L, Wells S. Negative equity—the value of reporting negative results. <i>Dis Model Mech</i>. 2024;17(8):dmm050937. doi:10.1242/dmm.050937</p>
<p class="abstract_reference-text ParaOverride-4">3. Taniike T, Takahashi K. The value of negative results in data-driven catalysis research. <i>Nat Catal</i>. 2023;6:108-111. doi:10.1038/s41929-023-00920-9</p>
<p class="abstract_conflict ParaOverride-6" lang="fr-FR"><sup>1</sup>Department of Neurology, Hackensack Meridian School of Medicine, Nutley, NJ, US; <sup>2</sup>Hackensack University Medical Center, Nutley, NJ, US; <sup>3</sup>Investigator Initiated Research Program and Support Services, Office of Research Administration, Hackensack Meridian Health Research Institute, Nutley, NJ, US; <sup>4</sup>Department of Medical Sciences, Hackensack Meridian School of Medicine, Hackensack Meridian Health, Nutley, NJ, US; elli.gournapaleoudis@hmhn.org.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


[V] Biomedical Studies Published With Negative Results Over the Past Decade

<p>Julia Soto Rizzato,<sup>1</sup> Marcus Tolentino Silva,<sup>2</sup> David Moher,<sup>3</sup> Tais Freire Galvão<sup>1</sup></p>
<h4>Objective </h4>
<p>To assess publication trends in epidemics estimated to end by 2030 according to the Sustainable Development Goals (SDGs) in pharmaceutical journals in the past 25 years and the association of the adoption of these SDGs in 2015 with this trend. Assessments of the transition from Millennium Development Goals to SDGs are available,<sup>1</sup> but investigations are lacking on the impact of the adoption of these SDGs on publications of a specific target.</p>
<h4>Design</h4>
<p> This cross-sectional study assessed all journals listed in the Pharmacology and Pharmacy category of the Journal Citation Reports. The primary outcome was the proportion of articles published addressing AIDS, tuberculosis, and malaria—epidemics targeted to end according to SDG 3.3. We consulted the Clarivate Web of Science Core Collection on December 19, 2024, to identify the total number of articles and the number of publications in each journal from 2000 to 2024 related to these epidemics. Data on country were collected from Web of Science metadata, which are based on authors’ affiliations and then classified into low-, middle-, or high-income countries based on World Bank country classifications by income level. A paper was considered from a low- or middle-income country if at least 1 author was from 1 of the countries in the metadata. An interrupted time series analysis was conducted to calculate the regression coefficient (β) and 95% CI of the proportion of papers (per 1000 publications) on SDG 3.3 before and after its adoption in 2015. Stata, version 14.2 was used for statistical analyses.</p>
<h4>Results </h4>
<p>Three hundred fifty-five journals were included, which published 1,313,671 articles, of which 49,687 (3.8%) addressed SDG 3.3 from 2000 to 2024 and 37,521 (2.9%) were published by authors from high-income countries and 12 166 (0.9%) were published by authors from low- and middle-income countries. Between 2000 and 2015, there was an overall growing trend in published papers related to SDG 3.3 (β, 0.47; 95% CI, 0.18-0.76; <i>P</i> = <i>.</i>003), which was more pronounced in developed countries (β, 0.69; 95% CI, 0.38-1.00; <i>P</i> &lt; .001),  while the trend from developing countries was not significant (β, 0.21; 95% CI, −0.19 to 0.06; P = .30) (<b>Figure 25-0901</b></span>). After the adoption of SDGs in 2015, SDG 3.3 publications started to decrease until 2024 (β, −1.00; 95% CI, −1.54 to −0.47; P = .001) in a similar pattern in highincome countries (β, −1.40; 95% CI, −2.00 to −0.81; P < .001), whereas a nonsignificant increase was observed in lower-income countries (β, 0.05; 95% CI, −0.05 to −0.73; P = .89) in the same period.</p>

![](https://assets.underline.io/markdown_image/1/image/3a82bbfa1f2eb9d809c2927e383599b4.png)
	
<h4>Conclusions </h4>
<p>The adoption of SDGs in 2015 did not seem to affect the publication priorities of pharmaceutical journals until 2024, which indicates that these measures may not have stimulated research efforts and publications. Even when considering that the results indicating research agenda priorities would take a few years to be published, the trend decreased until the end of the series, almost a decade after the adoption of SDGs.</p>
<h4>Reference</h4>
<p>1. Díaz-López C, Martín-Blanco C, De la Torre Bayo JJ, Rubio-Rivera B, Zamorano M. Analyzing the scientific evolution of the sustainable development goals. <i>Appl Sci</i>. 2021;11(18):8286. doi:0.3390/app11188286</p>
<p class="abstract_conflict" lang="fr-FR"><span class="CharOverride-3" lang="pt-PT"><sup>1</sup></span><span lang="pt-PT">Faculdade de Ciências Farmacêuticas, Universidade Estadual de Campinas, Campinas, Brazil, </span>taisgalvao@gmail.com<span lang="pt-PT">;</span> <span class="CharOverride-3" lang="pt-PT"><sup>2</sup></span><span lang="pt-PT">Faculdade de Ciências de Saúde, Universidade de Brasília, Brasília, Brazil; </span><sup>3</sup>Centre for Journalology, Methods Centre of the Ottawa Hospital Research Institute, Ottawa, Canada.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>David Moher and <a id="_idTextAnchor039"></a>Tais <span lang="pt-PT">Freire Galvão</span> report being advisory board members of the International Congress on Peer Review and Scientific Publication but were not involved in the review or decision for this abstract. The other authors declare no conflict of interest.</p>
<h4>Funding/Support<a id="_idTextAnchor040"></a> </h4>
<p>This study was funded in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior–Brasil (Finance Code 001, granted to <span lang="pt-PT">Julia Soto Rizzato)</span>. <span lang="pt-PT">Tais Freire Galvão</span> receives a productivity scholarship from the National Council for Scientific and Technological Development (grant 313431/2023-00).</p>
<h4>Role of the Funder/Sponsor </h4>
<p>The funders had no role in this research.</p>


[V] Publication Trends on Priority Epidemics According to the Sustainable Development Goals in Pharmaceutical Journals, 2000-2024

<p>Qinyue Liu,<sup>1</sup> Yagmur <span lang="ar-SA">Öztürk</span>,<sup>1</sup> Cyril Labbé<sup>1</sup></p>
<h4>Objective </h4>
<p>Abstracts of scientific publications are widely used in scientometrics and text mining because they can be extracted in large quantities from bibliometric databases, and they convey important information about the publications. However, we found misregistered abstracts in Crossref published by <i>Science</i> and the <i>Proceedings of the National Academy of Sciences</i> (<i>PNAS</i>). We aimed at estimating the proportion of misregistered abstracts in our dataset and providing the textual similarity between the correct abstracts and the misregistered texts. A previous study also found misregistered metadata in Crossref.<sup>1</sup></p>
<h4>Design </h4>
<p>Having recently worked on citation analysis, we developed a script to automatically collect a dataset of citations extracted from articles published between 2016 and 2024 in <i>The Lancet, Cell</i>, and <i>Joule</i>. Each citation context is paired with the cited abstract mainly extracted using Crossref API. This dataset includes 19,822 citation context-abstract pairs. We noticed that certain abstracts on Crossref are misregistered. For <i>Science</i>, a section that introduces and cites the current article, and for <i>PNAS</i>, a section called Significance, are misregistered as abstracts on Crossref. We developed a script to automatically find these cases from our dataset by searching for distinctive features (string “et al” for <i>Science</i> and “Significance” for <i>PNAS</i>). For the cases in <i>Science</i>, we also manually collected the correct abstracts, and used the Sentence-BERT (SBERT)<sup>2</sup> model to calculate the textual similarity between the misregistered abstracts and the correct abstracts.</p>
<h4>Results </h4>
<p>Of the 821 abstracts of <i>Science</i> publications, 402 (48%) contained the distinctive features.<i> </i>We manually verified the results, and 400 of 402 abstracts were misregistered. The 2 correctly registered abstracts were for report types of publication where there was no other text to be misregistered. Of the 669 <i>PNAS</i> abstracts, 243 (36%) contained Significance at the beginning. Moreover, most of the misregistered abstracts in our dataset, as estimated, had high textual similarity scores (cosine similarity) with the correct abstracts. The scores range from 0.58 to 0.99, with an average of 0.83. This phenomenon occurs because the misregistered abstracts summarize the article and mention key points, resembling the original abstracts.</p>
<h4>Conclusions </h4>
<p>Metadata misregistration in large repositories like Crossref can compromise data quality. Although our method detected misregistered abstracts in a small portion of the dataset using distinctive features, its scalability requires validation with larger datasets.</p>
<h4>References</h4>
<p>1. Cioffi A, Coppini S, Massari A, et al. Identifying and correcting invalid citations due to DOI errors in Crossref data. <i>Scientometrics</i>. 2022;127:3593-3612. doi:10.1007/s11192-022-04367-w</p>
<p>2. Reimers N, Gurevych I. Sentence-BERT: Sentence embeddings using Siamese BERT-Networks. In: <i>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)</i>. Association for Computational Linguistics; 2019:3982-3992. doi:10.18653/v1/D19-1410</p>
<sup>1</sup>Université Grenoble Alpes, French National Centre for Scientific Research, Grenoble INP, Laboratoire d’Informatique de Grenoble, Grenoble, France, qinyue.liu@univ-grenoble-alpes.fr.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4><a id="_idTextAnchor041"></a>Funding/Support</h4>
<p> <a id="_idTextAnchor042"></a>The NanoBubbles project has received Synergy grant funding from the European Research Council, within the European Union’s Horizon 2020 program, grant 951393.</p>
<h4>Role of Funder/Sponsor </h4>
<p>The European Research Council had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.</p>


[V] Science Journal Abstracts Misregistered in the Crossref Database

<p>Younwoo Ki,<sup>1,2</sup> Chungsoo Kim,<sup>3,4</sup> Yuan Lu,<sup>3</sup> Joshua D. Wallach,<sup>5</sup> Behnood Bikdeli,<sup>4,6,7</sup> Milton Packer,<sup>8,9</sup> Harlan M. Krumholz,<sup>4</sup> Seng Chan You<sup>1,2</sup></p>
<h4>Objective </h4>
<p>Recent trends in cardiovascular journals suggest a growing emphasis on publishing guidelines, reviews, and statements.<sup>1,2</sup> Citations, often used to assess the scholarly impact of journals, also influence researcher evaluation and library purchasing decisions. To understand these changes and their implications, we examined the evolving contribution of original research to citation metrics in leading cardiovascular journals.</p>
<h4>Design </h4>
<p>Our cross-sectional bibliometric study used Scopus to identify all articles published in the <i>Journal of the American College of Cardiology</i> (JACC), <i>Circulation</i> (CIRC), and <i>European Heart Journal</i> (EHJ) from 2008 to 2018, with associated citations. Using section headings on respective journal websites and article metadata (title, abstract, and pages), we manually classified articles into (1) original research articles (ORAs, including nonsystematic meta-analyses); (2) reviews as nonoriginal research articles (NORA-reviews, narrative or systematic); (3) guidelines, statements, and statistics as NORAs (NORA-GSS), and (4) uncitable items (brief reports, case reports, commentaries, and editorials). Uncitable items were excluded from analysis. Outcomes were (1) ORAs’ and NORAs’ mean 2-year citation counts, (2) proportion of 2-year citations by each category among total 2-year citations to all citable items, and (3) publication count and proportions of each category among total citable items.</p>
<h4>Results</h4>
<p> Among 22,740 articles, 9117 (39.9%) were ORAs, 2286 (10.0%) were NORA-reviews, 909 (4.0%) were NORA-GSS, and 10,566 (46.2%) were uncitable items. From 2008 to 2018, Journal Impact Factors (JIFs) in each journal increased, but their 2-year mean (SD) citations of ORA plateaued (JACC, 24.9 [24.3] to 26.4 [28.9]; CIRC, 24.00 [26.3] to 23.4 [23.6]; and EHJ, 15.8 [16.9] to 25.2 [20.8]) (<b>Figure 25-0977</b>, A). The annual publication of ORAs among total citable items declined across all 3 journals, although to varying extents (counts with proportions: JACC, 333 [80.5] to 245 [65.8]; CIRC, 415 [79.2] to 267 [78.1]; and EHJ, 251 [87.9] to 165 [69.6]), reflecting the decreased proportion of ORA 2-year citations among total citations (proportion of 2-year citations: JACC, 71.9% [9249/12,858] to 55.1% [7561/13,728]; CIRC, 68.1% [10,657/15,646] to 55.8% [6708/12,027]; and EHJ, 70.1% [4023/5739] to 36.8% [5073/13,771]) (<b>Figure 25-0977</b>, B). Although the annual publications of NORA-GSS were small in portion (counts with proportions: JACC, 33 [7.5%] to 22 [5.7%]; CIRC, 40 [7.0%] to 47 [13.0%]; and EHJ, 9 [3.1%] to 29 [11.9%]), the proportion of their 2-year citations among the total substantially increased (proportion of 2-year citations: JACC, 10.6% [1362/12,858] to 20.0% [2749/13,728]; CIRC, 19.3% [3019/15,646] to 37.7% [4535/12,027]; and EHJ, 16.6% [955/5739] to 52.4% [7211/13,771]) (<b>Figure 25-0977</b>, B).</p>

![](https://assets.underline.io/markdown_image/1/image/047d3162fe848a6fbfe0c11af9284812.png)

<h4>Conclusions </h4>
<p>Rising JIFs in major cardiology journals were associated primarily with an expanding share of nonoriginal content. Overreliance on JIF in academic evaluation may unconsciously steer journals toward prioritizing highly cited nonoriginal content, systematically undermining the value of primary research.</p>
<h4>References</h4>
<p>1. Genuis SJ. The proliferation of clinical practice guidelines: professional development or medicine-by-numbers? <i>J Am Board Fam Pract</i>. 2005;18(5):419-425. doi:10.3122/jabfm.18.5.419 </p>
<p>2. Shuaib W, Khan MS, Shahid H, Valdes EA, Alweis R. Bibliometric analysis of the top 100 cited cardiovascular articles. <i>Am J Cardiol</i>. 2015;115(7):972-981. doi:10.1016/j.amjcard.2015.01.029</p>
<sup>1</sup>Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Korea; <sup>2</sup>Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea, chandryou@yuhs.ac; <sup>3</sup>Section of Cardiovascular Medicine, Department of Internal Medicine, Yale University School of Medicine, New Haven, CT, US; <sup>4</sup>Center for Outcomes Research and Evaluation, Yale New Haven Hospital, New Haven, CT, US; <sup>5</sup>Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, US; <sup>6</sup>Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; <sup>7</sup>Thrombosis Research Group, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; <sup>8</sup>Baylor Heart and Vascular Institute, Baylor University Medical Center, Dallas, TX, US; <sup>9</sup>Imperial College London, London, UK.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Several authors hold editorial positions at the Journal of the American College of Cardiology (JACC): Yuan Lu and Behnood Bikdeli as executive associate editors, Joshua D. Wallach and Seng Chan You as associate editors, Milton Packer as senior consulting editor, and Harlan M. Krumholz as editor in chief. Joshua D. Wallach reported receiving grants from Johnson &amp; Johnson (through the Yale Open Data Access Project), Arnold Ventures, the National Institutes of Health, and the US Food and Drug Administration, as well as consulting fees from Hagen Berman Sobol Shapiro LLP and Dugan Law Firm APLC outside the submitted work. Seng Chan You reported being a chief technology officer of PHI Digital Healthcare and receiving grants from Daiichi Sankyo outside the submitted work.</p>
<h4>Funding/Support</h4>
<p> This study was supported by a Severance Hospital Research Fund for Clinical Excellence (SHRC) (C-2024-0011).</p>


[V] Trends in Citation Impacts of Original Research in Major Cardiovascular Journals, 2008-2018

<p>S. Scott Graham,<sup>1</sup> Joshua B. Barbour,<sup>2</sup> Zoltan P. Majdik,<sup>3</sup> Madeline Bruegger,<sup>1</sup> Carlee A. Baker,<sup>1</sup> Justin F. Rousseau<sup>4</sup></p>
<h4>Objective</h4>
<p> Research on industry funding and author conflicts of interest (COIs) suggests that centralization of funding may lead to detrimental effects on biomedical research.<sup>1,2</sup> This study was conducted to assess relationships between COI networks and drug adverse event rates reported to the US Food and Drug Administration.</p>

<h4>Design</h4> We used a GPT 4.0 pipeline (a series of computational steps applied to a dataset for transformation and/or analysis) to identify and classify reported author COIs and funders associated with 302 of the most commonly used drugs in the US. Included articles included up to 150 of the most relevant PubMed results when searching for each drug’s proprietary and generic name. For each declared COI, the funder was identified and funder types were classified by a fine-tuned GPT 4.0 model as industry, federal, university, patient advocacy organization, professional society, university, or hospital/clinic. COI relationships for each drug product were assembled into network models mapping relationships among all reported funders and all articles for each product. Each network was evaluated for network robustness by measuring critical failure threshold (CFT) (ie, the minimum number of nodes that would need to be removed to result in a 50% collapse of the network).<sup>3</sup> COI network robustness is a useful measure of the centralization of the funding network as a low number of funder networks would have lower failure thresholds. A network with many independent funders would require more nodes to be removed before network failure. Controlling for COI rates, drug features (over-the-counter availability, generic availability, US Drug Enforcement Administration schedule, usage rates), and publication features (mean article year of publication, total authorships in the network), we evaluated the relationship between network robustness and reported adverse events rates (total, serious, and fatal). A secondary model was fit to determine whether individual funder type frequencies or heterogeneity were associated with failure threshold. The GPT classifier was compared with a human annotated sample and evaluated for precent accuracy.</p>
<h4>Results</h4>
<p> We identified 37,993 articles associated with the 302 drugs. The GPT 4.0 classifier achieved 98.9% accuracy compared with the human annotated sample. In the adverse events models, CFT returned adjusted incident rate ratios (IRRs) of 0.94 (95% CI, 0.91-0.97; <i>P</i> &lt; .01) for total events, 0.94 (95% CI, 0.91-0.97, <i>P</i> &lt; .01) for serious events, and 0.94 (95% CI, 0.91-0.98; <i>P</i> &lt; .01) for fatal events. The number of funders per network was not a significant factor in any adverse events model. The network features model (<b>Table 25-0837</b>) returned significant relationships between the CFT and the number of funders in the top 10% by out-degree, a network science measure of the number of outgoing connections from a given node (IRR, 1.03 [95% CI, 1.02-1.04]; <i>P</i> &lt; .001) and funder-type heterogeneity (IRR, 1.19 [95% CI, 1.12-1.26]; <i>P</i> &lt; .001).</p>

![](https://assets.underline.io/markdown_image/1/image/c1c38c93f45a0d4c889dd7b12468fb91.jpg)

<h4>Conclusions</h4>
<p> In this study, COI network robustness was inversely associated with adverse events, including serious and fatal events, reported in drug studies. Secondary analyses indicate that this was associated with funding network centralization and homogeneity. The products evaluated in this study were associated with lower adverse event rates when the collected research had a larger number of funders, regardless of funder type. This suggests that research funding policy should encourage funding diversity.</p>
<h4>References</h4>
<p class="abstract_reference-text ParaOverride-4">1. Graham SS, Harrison KR, Edward JC, Majdik ZP, Barbour JB, Rousseau JF. Beyond bias: aggregate approaches to conflicts of interest research and policy in biomedical research. <i>World Med Health Policy</i>. 2024;16(3):489-505. doi:10.1002/wmh3.608.</p>
<p class="abstract_reference-text ParaOverride-4">2. Sismondo S. Ghost management: how much of the medical literature is shaped behind the scenes by the pharmaceutical industry? <i>PLoS Med</i>. 2007;4(9):e286. doi:10.1371/journal.pmed.0040286</p>
<p class="abstract_reference-text ParaOverride-4">3. Albert R, Jeong H, Barabási AL. Error and attack tolerance of complex networks. <i>Nature</i>. 2000;406(6794):378-382. doi:10.1038/35019019</p>
<sup>1</sup>Department of Rhetoric &amp; Writing, The University of Texas at Austin, Austin, TX, US, ssg@utexas.edu; <sup>2</sup>Department of Communication, University of Illinois Urbana-Champaign, Urbana, IL, US; <sup>3</sup>Department of Communication, North Dakota State University, Fargo, ND, US; <sup>4</sup>Department of Neurology &amp; Peter O’Donnell Jr. Brain Institute, The University of Texas Southwestern Medical Center, Dallas, TX, US.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>S. Scott Graham received grant funding from the National Institute of General Medical Sciences (NIGMS). Joshua B. Barbour received grant funding from the NIGMS, NSF, IC2, and Blue Cross Blue Shield; royalties form Cengage Publishing; and consulting fees from The Aslan Group and BrainCheck. Justin F. Rousseau received grant funding from the NIGMS, National Library of Medicine, National Institute on Aging, National Institute of Allergy and Infectious Diseases, National Institute of Mental Health, National Center for Advancing Translational Sciences, Texas Child Mental Health Care Consortium, Michal &amp; Susan Dell Foundation, Texas Alzheimer’s Research and Care Consortium, Austin Public Health, and the Health Care Cost Institute. No other disclosures were reported.</p>
<h4>Funding/Support</h4>
<p> Funding for this research was provided by the NIGMS of the National Institutes of Health under award number R01GM141476.</p>
<p class="abstract_conflict ParaOverride-7" lang="fr-FR"><h4>Role of the Funder/Sponsor</h4> The sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.</p>


[V] Conflict of Interest Network Robustness and Funder Homogeneity Associated With Reported Adverse Events and Deaths in Published Drug Studies

Alessandro Bianconi,<sup>1</sup> Matteo Fiore,<sup>1</sup> Maria Elena Flacco,<sup>2</sup> Lamberto Manzoli<sup>1</sup></p>
<h4>Objective </h4>Despite global initiatives to improve transparency regarding conflicts of interest (COIs) in scientific publications, the disclosure of competing interests remains inadequate among authors of clinical practice guidelines (CPGs), as well as for other types of publications.<sup>1,2</sup> Open access databases like Open Payments offer a potential resource for assessing undisclosed COIs in scientific literature.<sup>3</sup> This study evaluates the prevalence of undisclosed industry-related financial COIs among authors of US-issued CPGs on obesity management from 2016 to present.</p>
<h4>Design </h4>On June 3, 2024, we identified CPGs on obesity management issued by US agencies through 3 repositories (UpToDate, Medscape, and Guidelines International Network). COI statements for each guideline were extracted. To identify undisclosed industry-related financial COIs, we cross-referenced authors who reported no competing interests with the Open Payments database,<sup>3</sup> considering only any form of payment and fund received prior to the guideline publication date. Undisclosed COIs were classified based on cumulative amounts as greater than or equal to $10,000 and less than $10,000. The presence of COIs was summarized as absolute and relative frequencies. Given the purely descriptive nature of this study, no hypotheses were tested.</p>
<h4>Results </h4>
<p>We retrieved 5 CPGs (American Academy of Pediatrics 2023, American Diabetes Association 2023, American Gastroenterological Association 2022, Veteran Affairs/Department of Defense 2020, Association of American Clinical Endocrinologists/American College of Endocrinology 2016) (<b>Table 25-1155</b>). Among 145 authors, 80 (55.2%) received payments from industry agents related to obesity prevention or treatment. Of these, 33 authors (41.3% [22.8% overall]) had undisclosed COIs. Among undisclosed COIs, 11 (33.3%) were greater than or equal to $10,000 and 22 (66.7%) were less than $10,000. Every guideline had at least 1 chair or first author with industry-related COIs. Seven of 9 (77.8%) of these prominent authors did not disclose any COI, though all undisclosed COIs were less than $10,000.</p>

![](https://assets.underline.io/markdown_image/1/image/87bbfb86490687a5c34e03c17da6b571.png)

<h4>Conclusions </h4>
<p>We found a relevant amount of undisclosed COIs in obesity management CPGs, and this amount may be underestimated because only financial interests could be evaluated. The presence of undisclosed COIs among CPG authors may reflect the complexities of COI disclosure rather than a straightforward lack of transparency. Variations in COI policies across scientific societies and publishers, which indicate the amount and type of payments that represent a COI that should be disclosed, may result in inconsistent reporting standards. This inconsistency presents an opportunity to harmonize COI management policies across publishing entities, promoting more comprehensive disclosure practices. Implementing regular monitoring of COI statements by peer reviewers and editors can further enhance transparency and trustworthiness. Tools like Open Payments can aid this process, though they have limitations, including potential inaccuracies and exclusion of nonfinancial COIs. Addressing these issues can strengthen the reliability of CPGs and other scientific publications. Further research should investigate the prevalence of undisclosed and overall COIs in other fields, assessing their potential impact on guideline recommendations and, thus, on real-world clinical practice.</p>
<h4>References</h4>
<p>1. Khan R, Scaffidi MA, Rumman A, Grindal AW, Plener IS, Grover SC. Prevalence of financial conflicts of interest among authors of clinical guidelines related to high-revenue medications. <i>JAMA Intern Med</i>. 2018;178(12):1712-1715. doi:10.1001/jamainternmed.2018.5106</p>
<p>2. Lenzer J, Hoffman JR, Furberg CD, Ioannidis JPA. Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. <i>BMJ</i>. 2013;347:f5535. doi:10.1136/bmj.f5535</p>
<p>3. Open Payments. Accessed February 14, 2025. <a href="https://openpaymentsdata.cms.gov"><span class="Hyperlink CharOverride-6">https://openpaymentsdata.cms.gov</span></a></p>
<sup>1</sup>Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy, alessandro.bianconi4@studio.unibo.it; <sup>2</sup>Department of Environmental and Prevention Sciences, University of Ferrara, Ferrara, Italy.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>


[V] Disclosed and Undisclosed Conflicts of Interest in US Guidelines for the Management of Obesity

<p>Oludele M. Solaja<sup>1</sup></p>
<h4>Objective</h4>
<p> Scientific peer review often excludes Indigenous knowledge (IK), favoring Western scientific norms. In Nigeria, valuable local insights on health and environmental practices are marginalized, limiting their impact and perpetuating epistemic injustice.<sup>1</sup> Integrating IK into peer review can enhance research relevance, cultural acceptance, and credibility, providing region-specific solutions crucial for sustainable development in Nigerian communities. This research examines how IK can be incorporated into scientific peer review to remove epistemological racism and improve the representation and practicality of IK in Nigerian environment and health research.<sup>2</sup></p>
<h4>Design</h4>
<p> The study follows a 3-phase approach: (1) literature search, (2) stakeholder consultation, and (3) development and pilot testing of guidelines. In phase 1, a comprehensive literature review of 142 peer-reviewed articles (identified via PubMed, Scopus, and African Journals Online) was conducted to identify gaps in IK recognition within environmental and health research in Nigeria. Phase 2 included 396 surveys, 30 semistructured interviews, and 3 focus group discussions involving Nigerian researchers, peer reviewers, and IK holders. This engagement was conducted to highlight cultural barriers and biases in existing peer review practices. Phase 3 involved developing standardized guidelines based on stakeholder feedback to incorporate IK into peer review, followed by pilot testing with a subset of 12 articles from 3 Nigerian journals (<i>Nigerian Journal of Environmental Sciences and Technology</i>, <i>FUTA Journal of Research in Sciences</i>, <i>Ibadan Journal of the Social Sciences</i>) from April to June 2025 to assess feasibility and effectiveness. In this phase, structured reviewer and editorial feedback was collected and is currently being analyzed to evaluate key metrics, including reviewer satisfaction (measured through structured postreview surveys), inclusivity ratings (assessed via 5-point Likert scale responses and demographic tracking of reviewer diversity), and manuscript quality (evaluated by independent editorial panels comparing clarity, cultural sensitivity, and scientific rigor between IK-inclusive and conventional reviews).</p>
<h4>Results</h4>
<p> Among the 142 articles reviewed, only 18 (12.7%) referenced IK, and none demonstrated structured inclusion of IK in peer review or evaluation criteria. In total, 32 of 38 IK stakeholders (84.2%) and 50 of 64 researchers (78.1%) perceived current peer review practices as exclusionary, citing language bias, absence of culturally grounded criteria, and a lack of reviewers who were knowledgeable in IK.<sup>3</sup> Pilot testing yielded the following outcomes: 19 of 24 reviewers (79.2%) reported increased clarity and confidence in evaluating IK-informed manuscripts; inclusivity ratings increased from a baseline mean of 2.8 to 4.3 on the 5-point Likert scale; and independent editorial panels reported that IK-inclusive reviews scored 25% to 30% higher in cultural sensitivity and contextual relevance without compromising scientific rigor. Stakeholders reported that integrating IK leads to a more holistic and locally resonant understanding of health and environmental challenges in Nigeria.</p>
<h4>Conclusions</h4>
<p> Integrating IK into peer review can make research validation more equitable and locally relevant, aligning with global calls for inclusivity in scientific processes. This framework, grounded in empirical findings, may serve as a model for adapting peer review standards to respect diverse knowledge systems and improve the societal impact of research.</p>
<h4>References</h4>
<p>1. Agrawal A. Dismantling the divide between indigenous and scientific knowledge. <i>Dev Change</i>. 1995;26(3):413-439. doi:10.1111/j.1467-7660.1995.tb00560.x</p>
<p>2. Smith LT. <i>Decolonizing Methodologies: Research and Indigenous Peoples</i>. Zed Books; 1999.</p>
<p>3. Nakashima D, Roué M. Indigenous knowledge, peoples, and sustainable practice. In: Timmerman P, ed. <i>Encyclopedia of Global Environmental Change</i>. John Wiley &amp; Sons; 2002:314-324.</p>
<sup>1</sup>Department of Sociology, Olabisi Onabanjo University, Ago-Iwoye, Nigeria, solaja.oludele@oouagoiwoye.edu.ng.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


[V] Integrating Indigenous Knowledge into Peer Review Processes in Nigerian Environmental and Health Research

<p>Hong Chen,<sup>1</sup> Christopher I. Rider,<sup>2</sup> David Jurgens,<sup>1,3</sup> Misha Teplitskiy<sup>1</sup></p>
<h4>Objective</h4>
<p> This study examines which intended scientific contributions become published contributions. While many manuscripts are rejected, there is limited understanding of which researchers successfully navigate rejection and resubmission. We investigate geographical disparities in postrejection publishing outcomes, focusing on differences between authors from Western and non-Western countries. We investigate several potential mechanisms underlying these disparities, especially procedural knowledge of how to interpret editorial decisions for rejection, revision, and resubmission.</p>
<h4>Design</h4>
<p> Partnering with the Institute of Physics Publishing, we collected metadata on 126,000 rejected submissions made in 2018 through 2022 from 62 peer-reviewed journals. We tracked whether, when, and where these rejected manuscripts were eventually published by developing a classifier that we validated with ground-truth data provided by the authors of rejected manuscripts. We defined Western authors as those affiliated with (self-reported) institutions in North America, Europe, or Oceania, and non-Western authors as those from Asia, Africa, and Latin America. The analysis examined the publishing disparities based on author geography while controlling for key confounders—perceived submission quality, proxied by peer reviewer evaluations at the time of rejection (including desk rejection)—as well as time, team composition, and corresponding author characteristics, such as gender, seniority, and current institutional affiliation ranking. The prior Western coauthors and Western affiliations were also included, serving as proxies for access to procedural knowledge. To further probe procedural knowledge as a mechanism, we examined multiple dimensions of how authors interpret and respond to rejection in a survey of corresponding authors whose manuscripts were rejected.</p>
<h4>Results</h4>
<p> Among the authors included, 43% had Western affiliations and 57% had non-Western affiliations. Authors from Western countries had better postrejection outcomes. Despite similar perceived quality at the time of rejection, Western authors were 6.7% more likely to publish the rejected papers elsewhere. Among those who ultimately published (59.3% of all rejected papers), Western authors published approximately 23 days faster and did 5.9% less abstract revising and 12.0% less authorship team change. We also found evidence suggesting that greater access to procedural knowledge, proxied by prior publishing experience and prior Western affiliation and coauthorship, contributed to better postrejection outcomes. These factors explain a substantial portion of the observed disparities in postrejection publication rates, more so than perceived submission quality or institutional resourcefulness. We also surveyed 10,000 corresponding authors of rejected manuscripts in June 2023 and received 287 responses. Limited by the very low response rate, survey responses indicated no significant differences between Western and non-Western authors in their perceptions of feedback fairness or negativity, planned revisions, or target outlet prestige, suggesting that other dimensions of procedural knowledge might be key.</p>
<h4>Conclusions</h4>
<p> This study identified geographical disparities in the file drawer of science. Western authors experienced better postrejection outcomes than non-Western authors, and the findings suggest that differential access to procedural knowledge may contribute to these disparities.</p>
<sup>1</sup>School of Information, University of Michigan, Ann Arbor, MI, US, tepl@umich.edu; <sup>2</sup>Ross School of Business, University of Michigan, Ann Arbor, MI, US; <sup>3</sup>Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Next from The 10th Peer Review Congress

[V] Reviewer Rating Variability and Confidence and Language Model Sentiment Prediction of Machine Learning Conference Papers

Similar lecture

Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Assessments