Clin Orthop Relat Res. 2023 Jan 18.
BACKGROUND: Because research experience is increasingly important in ranking orthopaedic residency and fellowship applicants, determining the accuracy of candidates reporting their scholarly activity is essential. However, disparate and inconsistent findings have made it difficult to draw meaningful conclusions from individual studies.QUESTIONS/PURPOSES: In this systematic review, we asked: (1) What percentage of research publications are misrepresented among orthopaedic residency and fellowship applicants? (2) What percentage of applications contain one or more example of academic misrepresentation? (3) Is research misrepresentation associated with any individual applicant characteristics? (4) What is the publication status of articles listed by applicants as having been submitted to journals?
METHODS: A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. PubMed, EBSCOhost, Medline, and Google Scholar electronic databases were searched on March 10, 2022, to identify all studies that evaluated research misrepresentation in orthopaedic residency and fellowship applications between January 1, 1995, and March 1, 2022. Articles were included if full-text articles in English were available and the study reported on research misrepresentation among orthopaedic residency or fellowship applicants. Studies investigating nonorthopaedic publications, systematic reviews, case studies, duplicate studies among databases, and gray literature were excluded. Two reviewers independently evaluated the quality of included studies using the Methodological Index for Nonrandomized Studies (MINORS) tool. This is a validated assessment tool that grades noncomparative studies from 0 to 16 and studies with control groups from 0 to 24, based on eight criteria related to study design, outcomes assessed, and follow-up. All included articles were noncomparative studies, so the maximum score here was 16, with higher scores indicating better study quality. The mean MINORS score was 13 ± 1 in the studies we included. The final analysis included 10 studies with 5119 applicants. Eight studies evaluated orthopaedic residency applicants and two evaluated fellowship applicants. The applicant classes ranged from 1996 to 2019. Research misrepresentation was defined among studies as nonauthorship of an existing article, claimed authorship of a nonexistent article, or incorrect listing of authorship order for an existing article. Each study's findings and definition of research misrepresentation were considered to allow for a discussion of overall trends. The percentage of misrepresentation was further broken down by the misrepresentation type. Applicant characteristics and destination of submitted articles were also evaluated. Given the potential overlap between applicants among the studies, no pooled analysis was conducted, and results are presented as a narrative summary.
RESULTS: The percentage of overall publication misrepresentation was estimated to range between 1% (13 of 1100) and 21% (27 of 131), with more-recent studies reporting a lower proportion of overall articles misrepresented. Most studies we found claimed that authorship of a nonexistent article was the most common type of misrepresentation. Nonauthorship of an existing article and incorrect authorship order were less common. The percentage of applications with at least one misrepresentation was approximately 20% between 1998 and 2017. Most studies found no applicant characteristics, such as match outcomes, demographic markers, or academic records, that were consistently associated with a higher odds of the candidate misrepresenting his or her research credentials. Finally, approximately half of the articles listed as submitted to journals went on to publication, with one-third going to a different journal with a lower Impact Factor.
CONCLUSIONS: Our systematic review found that the percentage of overall publication misrepresentations among orthopaedic residency and fellowship applicants has generally been low over the past 20 years. However, approximately one-fifth of applications had at least one research misrepresentation, with 2% having multiple misrepresentations on reported publications. There were no consistent applicant characteristics associated with higher odds of research misrepresentation. Additionally, most of the articles listed as submitted to journals for publication were ultimately published.
CLINICAL RELEVANCE: Although the decrease in overall publication misrepresentation is encouraging, our finding that one-fifth of applicants have research misrepresentation is a cause for concern. In light of a continually evolving application process, orthopaedic residency and fellowship programs must ensure there is integrity related to information that is self-reported by applicants. These findings also serve to encourage faculty members involved in the application screening and decision process to limit biases related to applicant demographics perceived to be associated with a high odds of misrepresentation. Furthermore, governing agencies and program leadership should evaluate methods of verifying unpublished work and provide opportunities for applicants to give publication updates throughout the application cycle.