bims-evares Biomed News
on Evaluation of research
Issue of 2019‒10‒13
sixteen papers selected by
Thomas Krichel
Open Library Society


  1. Urology. 2019 Oct 02. pii: S0090-4295(19)30851-9. [Epub ahead of print]
    Lee A, Ramstein J, Cohen AJ, Agochukwu-Mmonu N, Patino G, Breyer BN.
      OBJECTIVE: To examine the most cited literature in urethral reconstruction, review types of work published, and observe research trends.METHODS: The Web of Sciences Sci-Expanded Index was used to conduct a search for urethral reconstruction. References were assessed for relevance to urethral reconstruction by two independent reviewers and a final list of the top 100 articles ranked by citation count was obtained. For each article, citation count, publication date, corresponding author, origin institution, origin country, topic area, study design, level of evidence, and origin journal were collected.
    RESULTS: The mean citation count per publication was 108 (median= 94.5; range= 69-366, SD=43) with a total of 10,874 citations for all papers since 1970. The top 100 articles were published between 1973 and 2011, came from 19 different countries and 16 different journals. Nearly half were case series and most studies were Level III evidence or lower. The United States was the largest contributor to the top 100 with 56 publications, followed by Italy (14), England (12) and Egypt (7). "Outcomes of surgical treatment for urethral stricture disease (USD)" was the most prevalent topic area comprising 55 articles in the top 100, with most articles including descriptions or outcomes of novel surgical techniques.
    CONCLUSIONS: In this study, we discovered that the most cited literature in the field of urethral reconstruction is singularly focused and lacking in high levels of evidence. The top 100 cited articles originate primarily from the United States, focus on short-term outcomes after surgical treatment for USD, and are predominantly case series.
    Keywords:  bibliometric; citation analysis; influential; urethral reconstruction
    DOI:  https://doi.org/10.1016/j.urology.2019.08.052
  2. Cureus. 2019 Aug 03. 11(8): e5313
    Huang M, Naser-Tavakolian K, Clifton M, Franceschi AM, Kim D, Zhang JZ, Schweitzer M.
      Objective To investigate gender difference patterns in article citations, by first and last authors, in four radiology journals. Materials and methods Articles by authors published in four major radiology journals from 1984, 1994, 2004, and 2014 were categorized into 12 subspecialties. The number of citations, references used, co-authors, and length of the article (number of pages) were documented. The genders of first/last authors were determined. Data were analyzed using chi-square and logistic regression. Results The gender of the first author was determined in 2679 articles and that of the last author in 2717 articles. Over the selected years, 1984 to 2014, female first authorship grew from 13.0% to 31.5% (p<0.001), and female last authorship grew from 9.3% to 22.1% (p<0.001). Primary female authorship papers were cited less often as compared to males (OR 0.9972, 95% CI: 0.9948-0.9996, p=0.021), after adjusting for publication year and subspecialty. Across most subspecialties, female first authorship received fewer citations. In 1984, primary female authorship papers received on average 28.9 citations versus males at 39.1; in 1994, 50.4 versus 60.8; in 2004, 41.5 versus 44.4; and in 2014, 7.0 versus 7.8. The mean difference in the number of citations received by male and female first authors decreased from 10.47±6.09 in 1984 and 9.49±7.12 in 1994 to 1.93±5.63 in 2004 and 0.79±0.39 in 2014. However, there was no statistical difference demonstrated in article citations between male and female last authorship (OR 0.9990, 95% CI: 0.9966-1.0013, p=0.392). Conclusions Primary female authorship garnered fewer citations than men, despite the increasing frequency of authorships. However, this differential in the number of citations is narrowing.
    Keywords:  citations; female authorship; gender differences; radiology journals
    DOI:  https://doi.org/10.7759/cureus.5313
  3. Orthop Rev (Pavia). 2019 Sep 24. 11(3): 7786
    Park J, Gil JA, Kleiner J, Eltorai AEM, Daniels AH.
      There is a growing demand for evidence- based practices and informed clinical decision making supported by reliable, high-quality research. The aim of the study is to analyze trends in the level of evidence of publications and to evaluate the publication characteristics that influence the quality of research in The Spine Journal (TSJ). This is a comprehensive publication assessment that reviews and analyses all studies published in TSJ from the years 2005, 2007, 2009, 2011, 2013, and 2015. Level of evidence, study type, funding source, author country, author department, number of citations were considered as the outcome measures. Multivariable logistic regression, multivariable linear regression analyses, and chi square tests were used to analyze the trends of published studies level of evidence, study type, the specialties of authors, author countries, number of citations, and funding sources. A total of 1456 articles were evaluated. There was a decrease in the percentage of high-level evidence (level 1 and 2) studies from 73.6% in 2005 to 49.8% in 2015 (P=0.0045). There was a significant increase in the percentage studies with reporting funding support (P<0.0001). Funded studies were more likely to have a higher level of evidence (P<0.0001). The percentage of studies from international authors increased from 17.8% in 2005 to 69.1% in 2015 (P<0.0001). The percentage of studies with orthopedic authors decreased from 67% in 2005 to 44.9% in 2015 with a corresponding increase in the percentage of studies with neurosurgeon authors from 14.4% in 2005 to 23.2% by 2015, as well as an increase in the percentage of studies with a collaboration of authors from both specialties from 5.1% in 2005 to 8.7% in 2015 (P=0.0007). Orthopedic and neurosurgery collaboration in authorship did not affect the level of evidence of the studies nor the number of citations of the studies (P=0.7583). Earlier studies had a higher Scopus citation number but were not affected by the level of evidence (P=0.2515) nor the department of the author(s) (P=0.9107). We can conclude that the publication characteristics of articles in TSJ have evolved between 2005 and 2015 with a 3.9-fold increase in international authorship and a 32% decrease in the proportion of Level I and Level II studies. Inter-departmental collaboration, funding source, and country of origin may affect level of evidence and number of citations. Continued efforts to increase level of evidence should be considered.
    Keywords:  Level of evidence; citations; conflict of interest; funding; impact
    DOI:  https://doi.org/10.4081/or.2019.7786
  4. Medicine (Baltimore). 2019 Oct;98(41): e17527
    Chien TW, Wang HY, Hsu CF, Kuo SC.
      BACKGROUND: Disparities in health outcomes across countries/areas are a central concern in public health and epidemiology. However, few authors have discussed legends that can be complemental to choropleth maps (CMs) and merely linked differences in outcomes to other factors like density in areas. Thus, whether health outcome rates on CMs showing the geographical distribution can be applied to publication citations in bibliometric analyses requires further study. The legends for visualizing the most influential areas in article citation disparities should have sophisticated designs. This paper illustrates the use of cumulative frequency (CF) map legends along with Lorenz curves and Gini coefficients (GC) to characterize the disparity of article citations in areas on CMs, based on the quantile classification method for classes.METHODS: By searching the PubMed database (pubmed.com), we used the keyword "Medicine" [journal] and downloaded 7042 articles published from 1945 to 2016. A total number of 41,628 articles were cited in Pubmed Central (PMC). The publication outputs based on the author's x-index were applied to plot CM about research contributions. The approach uses two methods (i.e., quantiles and equal total values for each class) with CF legends, in order to highlight the difference in x-indices across geographical areas on CMs. GC was applied to observe the x-index disparities in areas. Microsoft Excel Visual Basic for Application (VBA) was used for creating the CMs.
    RESULTS: Results showed that the most productive and cited countries in Medicine (Baltimore) were China and the US. The most-cited states and cities were Maryland (the US) and Beijing (China). Taiwan (x-index = 24.38) ranked behind Maryland (25.97), but ahead of Beijing (16.9). China earned lower disparity (0.42) than the US (0.49) and the rest of the world (0.53) when the GCs were applied.
    CONCLUSION: CF legends, particularly using the quantile classification for classes, can be useful to complement CMs. They also contain more information than those in standard CM legends that are commonly used with other classification methods. The steps of creating CM legends are described and introduced. Bibliometric analysts on CM can be replicated in the future.
    DOI:  https://doi.org/10.1097/MD.0000000000017527
  5. J Thorac Cardiovasc Surg. 2019 Sep 09. pii: S0022-5223(19)31759-3. [Epub ahead of print]
    Narahari AK, Cook IO, Mehaffey JH, Chandrabhatla AS, Hawkins RB, Tyerman Z, Charles EJ, Tribble CG, Kron IL, Teman NR, Roeser ME, Ailawadi G.
      OBJECTIVE: To determine trends in National Institutes of Health (NIH) funding for cardiac surgeons, hypothesizing they are at a disadvantage in obtaining funding owing to intensive clinical demands.METHODS: Cardiac surgeons (adult/congenital) currently at the top 141 NIH-funded institutions were identified using institutional websites. The NIH funding history for each cardiac surgeon was queried using the NIH Research Portfolio Online Reporting Tools Expenditures and Results (RePORTER). Total grant funding, publications, and type was collected. Academic rank, secondary degrees, and fellowship information was collected from faculty pages. Grant productivity was calculated using a validated grant impact metric.
    RESULTS: A total of 818 academic cardiac surgeons were identified, of whom 144 obtained 293 NIH grants totaling $458 million and resulting in 6694 publications. We identified strong associations between an institution's overall NIH funding rank and the number of cardiac surgeons, NIH grants to cardiac surgeons, and amount of NIH funding to cardiac surgeons (P < .0001 for all). The majority of NIH funding to cardiac surgeons is concentrated in the top quartile of institutions. Cardiac surgeons had a high conversion rates from K awards (mentored development awards) to R01s (6 of 14; 42.9%). Finally, we demonstrate that the rate of all NIH grants awarded to cardiac surgeons has increased, driven primarily by P and U (collaborative project) grants.
    CONCLUSIONS: NIH-funded cardiac surgical research has had a significant impact over the last 3 decades. Aspiring cardiac surgeon-scientists may be more successful at top quartile institutions owing to better infrastructure and mentorship.
    Keywords:  NIH funding; cardiac surgery research; surgeon scientists
    DOI:  https://doi.org/10.1016/j.jtcvs.2019.08.032
  6. AIDS Rev. 2019 Oct 03. 21(3):
    Tran BX, Nathan KI, Phan HT, Hall BJ, Vu GT, Vu LG, Pham HQ, Latkin CA, Ho CSH, Ho RCM.
      Tremendous progress has been made over the past three decades in the fight against the human immunodeficiency virus (HIV) epidemic; however, children experienced substantial physical and psychosocial effects of HIV infection. To inform services and policy development, we analyzed research growth and current trends in studies on children with HIV/acquired immune deficiency syndrome (AIDS). A total of 17,598 research papers and reviews in English, which were published on Web of Science, were downloaded. VOSviewer was used to generate an author keyword cooccurrence network and a network illustrating the connection among countries by shared coauthorships. Exploratory factor analysis was employed to identify research domains emerging from the abstracts' contents. The number of papers and their impacts had grown significantly in the past decade. The majority of study settings were in African countries. Research topics related to HIV in children were robust in areas of prevention of mother-to-child transmission and HIV and comorbidities treatments. Although psychosocial and behavioral disorders were recognized in previous studies, the number of interventions in these fields is still limited or not sustained. This study presents the global research trends and interests, points out research gaps of available publications and suggests several implications for services and intervention programs for children infected with HIV/AIDS. While preventions of mother-to-child transmission have been extensively studied, more efforts should be made to fulfill the lack of research on young people who are at risk of being infected or who are already infected with HIV. System thinking approach is needed in the design and implementation of future studies.
  7. BJS Open. 2019 Oct;3(5): 696-703
    Hinrichs DL, Debus ES, Grundmann RT.
      Background: Surgical publication activity in the English literature over a 10-year interval may have changed. This study sought to identify which countries make the most contributions and whether significant shifts have occurred in this time.Methods: Screening of 17 international journals in PubMed was performed for the time periods 2006-2007 and 2016-2017, for papers published by a first author belonging to a general surgical department. Data were collected by country regarding the total number of publications, cumulative impact factors (IFs), publications per inhabitant, IFs per inhabitant, and number of RCTs, meta-analyses and systematic reviews per country in both periods.
    Results: A total of 2247 and 3029 papers were found for 2006-2007 and 2016-2017 respectively. In 2006-2007, most papers (605, 26·9 per cent; 2697·3 IFs) came from the USA, followed by Japan (284, 12·6 per cent; 1042·1 IFs) and the UK (197, 8·8 per cent; 923·1 IFs). In 2016-2017, the USA led again with 898 papers (29·6 per cent; 4575·3 IFs), followed by Japan with 414 papers (13·7 per cent; 1556·6 IFs) and the Netherlands with 167 (5·5 per cent; 885·2 IFs). From the top 15 countries, Sweden, the Netherlands and Switzerland contributed the most articles per inhabitant during both time periods. During both periods, the UK published the most RCTs, meta-analyses and systematic reviews.
    Conclusion: Surgeons from the USA were the most productive in total number of publications during both time periods. However, smaller European countries were more active than the USA in relation to their population size.
    DOI:  https://doi.org/10.1002/bjs5.50172
  8. AIDS Rev. 2019 10 03. 21(3):
    Tran BX, Wong FY, Pham KTH, Latkin CA, Ha GH, Vu GT, Ho CSH, Ho RCM.
      INTRODUCTION: In recent years, there have been numerous calls by researchers to adopt multi-disciplinary and international perspectives to address the human immunodeficiency virus (HIV) pandemic. Meaningful and prudent public health policy should be based on sound empirical data and research. Henceforth, our study aims to contribute to the current literature by conducting a comprehensive global mapping and determine the landscapes of HIV/acquired immune deficiency syndromes (AIDS) research covering the years between 1983 and 2017.METHODS: Bibliometric and content analysis was used to describe trends in research productivity, usages, research collaborations, and clusters of research topics. Exploratory factor analysis, Jaccard's similarity index, and Ward dendrogram were applied to abstracts' contents to determine the development of interdisciplinary research landscapes.
    RESULTS: The United States of America continues to lead in research production and be main hub for author- and country-level collaborations. Research employing an epidemiological, social, and/or behavioral perspective for studying HIV/AIDS was found to dwarf in the presence of basic and biomedical HIV research. Interdisciplinary approaches to HIV research have been increasing with the creation of various research landscapes: strong constructs of studies examining health status, clinical responses, and HIV treatment, risk behaviors have been formed, while research topics relating to psycho-behavioral and cultural aspects as well as services have emerged along.
    CONCLUSIONS: To effectively prevent and control the disease, more researches are needed to provide culturally relevant and/or contextualized evidence of effective interventions. It is also necessary to enhance the ability and partnership of local researchers as well as invest in research infrastructure at national and regional levels to implement high-quality studies since they are the "gate-keepers" who could respond to local changes in a timely manner. These types of research could be a helpful guide for international donors, governments, and academicians to set up research priorities in target groups and settings, and to develop future research agendas globally.
  9. Orthop Traumatol Surg Res. 2019 Oct 03. pii: S1877-0568(19)30272-5. [Epub ahead of print]
    Trost M, Langer F, Lechler P, Schröder F, Wetterkamp M, Schulte TL, Eysel P, Boese CK.
      BACKGROUND: The publication rate of presented abstracts is an important parameter to assess the scientific quality of medical congresses. It has been investigated for many congresses in orthopaedics and traumatology, but until now, it has not been studied for the congress of the European Federation of National Associations of Orthopaedics and Traumatology (EFORT). The aims of this study were to determine: (1) the publication rate of the EFORT congress, (2) factors that favour publication of abstracts presented at the EFORT congress, (3) the consistency between the congress abstract and publication in relation to authorship.HYPOTHESIS: There are factors that favour publication of abstracts presented at the EFORT congress and there is a high consistency between the congress abstract and publication in relation to authorship.
    MATERIALS AND METHODS: All 1624 abstracts presented at the EFORT congress in 2011 were included in this study, to allow a 5-year period for publication after the congress. The characteristics of the abstracts presented were studied and the publication rate in peer-reviewed journals was determined using a Medline search.
    RESULTS: The publication rate for studies presented at the 2011 EFORT congress was 42% (677/1624 abstracts), with a mean of 16 months (-56 to 60 months) between congress and publication. The mean impact factor of the publications was 1.8 (0-7.6). A significantly higher publication rate was found for: oral presentations (52%; 322/617) versus posters (35%; 355/1007) (p<0.01), experimental studies (53%; 110/208) versus clinical studies (40%; 507/1254) (p<0.01), and studies with higher levels of evidence of I or II (59%; 144/244) versus studies with lower levels of evidence of III or IV (36%; 362/1005) (p<0.01). A new author was added in 59% (403/677) of the publications.
    DISCUSSION: Factors that favour publication of abstracts presented at the EFORT congress are oral presentation, experimental study, and a study with a higher level of evidence of I or II. It is common that a new author is added in the publication. Nevertheless, a high percentage of congress abstracts (58%; 947/1624) remains unpublished.
    LEVEL OF EVIDENCE: IV, retrospective study.
    Keywords:  Congress; EFORT; Orthopaedics; Publication rate; Traumatology
    DOI:  https://doi.org/10.1016/j.otsr.2019.07.014
  10. BJU Int. 2019 Oct 10.
    Chandrasekar T, Goldberg H, Klaassen Z, Wallis CJ, Leong JY, Liem S, Teplitsky S, Noorani R, Loeb S.
      OBJECTIVE: To provide the first comprehensive analysis of the Twitter-verse amongst academic urologists and programs in North America.METHODS: Using national accreditation and individual program websites, all active urology residency programs (USA & Canada) and academic Urology faculty at these programs were identified. Demographic data for each program (AUA section, resident class size) and physician (title, fellowship training, Scopus H-index and citations) were documented. Twitter metrics (Twitter handle, date joined, # tweets, # followers, # following, likes) for programs and physicians were catalogued (data capture: March-April 2019). Descriptive analyses and temporal trends in Twitter utilization amongst programs and physician were assessed. Multivariable (MV) logistic regression was used to identify predictors of Twitter use.
    RESULTS: 156 academic programs (143 USA, 13 Canada) and 2214 academic faculty (2015 USA, 199 Canada) were identified. Twitter utilization is currently 49.3% and 34.1% amongst programs and physicians, respectively, and continues to increase. On MV analysis, programs with 3-5 residents/year and programs with a higher percentage of faculty Twitter engagement were more likely to have Twitter accounts. From a physician perspective, those with fellowship training, lower academic rank (clinical instructor, assistant professor, associate professor vs. professor) and higher H-indices were more likely to have individual Twitter accounts.
    CONCLUSION: There is a steady increase in Twitter engagement amongst Urology programs and academic physicians. Faculty Twitter utilization is an important driver of program Twitter engagement. Twitter SoMe activity is strongly associated with academic productivity, and may in fact drive academic metrics. Within Urology, SoMe presence appears to be proportional to academic activity.
    Keywords:  Academics; Kardashian Index; Social Media; Survey; Twitter; Urology
    DOI:  https://doi.org/10.1111/bju.14920
  11. J Glob Oncol. 2019 Oct;5 1-12
    Vogel AL, Puricelli Perin DM, Lu YL, Taplin SH.
      PURPOSE: International research networks have the potential to accelerate scientific progress via knowledge sharing and collaboration. In 2018, the US National Cancer Institute evaluated the International Cancer Screening Network (ICSN), in operation since 1988.METHODS: ICSN hosts a biennial scientific meeting and scientific working groups. A survey was fielded to 665 ICSN participants, and a bibliometric analysis was conducted for ICSN publications.
    RESULTS: A total of 243 individuals completed the survey (36.5%). They reported that participating in the ICSN helped advance their knowledge of cancer screening research (75.7%), policy development (56%), and implementation (47.7%). Approximately three-quarters agreed that ICSN facilitated knowledge sharing and networking among researchers and implementers (79.9%) and those working on different continents (74.0%) and cancer sites (73.7%). More than half reported that participating helped them form new collaborations in screening implementation (58.0%) or research (57.6%). Most agreed that ICSN helped to advance screening research and evaluation (75.4%), effective screening practices (71.2%), and screening policies (60.9%). Many reported that participating informed advances in their own research (68.7%) and screening implementation (50.2%) and policies (49.4%) in their settings. Approximately two-thirds agreed that ICSN helped advance career development among current experts (66.6%) and train the next generation (62.2%). Half (51.4%) reported that participating advanced their own careers. The 20 ICSN publications included 75 coauthors. They were cited in 589 publications with more than 2,000 coauthors.
    CONCLUSION: Findings provide evidence of the influence of ICSN on international knowledge dissemination, collaboration, and advances in cancer screening research, implementation, and policies and highlight the potential value of longstanding international research networks.
    DOI:  https://doi.org/10.1200/JGO.19.00197
  12. PLoS Biol. 2019 Oct;17(10): e3000416
    Hutchins BI, Davis MT, Meseroll RA, Santangelo GM.
      Fundamental scientific advances can take decades to translate into improvements in human health. Shortening this interval would increase the rate at which scientific discoveries lead to successful treatment of human disease. One way to accomplish this would be to identify which advances in knowledge are most likely to translate into clinical research. Toward that end, we built a machine learning system that detects whether a paper is likely to be cited by a future clinical trial or guideline. Despite the noisiness of citation dynamics, as little as 2 years of postpublication data yield accurate predictions about a paper's eventual citation by a clinical article (accuracy = 84%, F1 score = 0.56; compared to 19% accuracy by chance). We found that distinct knowledge flow trajectories are linked to papers that either succeed or fail to influence clinical research. Translational progress in biomedicine can therefore be assessed and predicted in real time based on information conveyed by the scientific community's early reaction to a paper.
    DOI:  https://doi.org/10.1371/journal.pbio.3000416
  13. J Contemp Dent Pract. 2019 Jul 01. 20(7): 867-872
    Natto ZS, Aljehani A, Sarhan A, Nawawi E, Abdullatef H, Samarkandi L, Nasser M, Badri R, Quqandi R, Waheeb S, Aljahdali S, Merdad Y.
      AIMS: This article describes the methodologies used in the dental literature and described how these approaches have changed over time.MATERIALS AND METHODS: Thirty-three ISI peer-reviewed journals were included in the analyses. Data were extracted independently by 11 investigators and in duplicate. Any differences in the results were resolved via discussion or by a third reviewer when necessary. Data were collected regarding the methodology used in the article, and dental specialty related to different study designs. In the case in which more than one study design or specialty was reported, reviewers were trained to identify the main methodology/specialty.
    RESULTS: The majority (36.96%) used a case report (CR) as the primary methodology, followed by a clinical trial (CT) (18.21%) or randomized CT (15.11%). The least used methodologies included a cohort (COH) study (6.07%) or a systematic review (SA)/meta-analysis (MA) (6.73%). Periodontology published the highest number of case controls (CCs) (46.8%), randomized CTs (RCTs) (29.9%), cross-sectional (CS) studies (26.0%), SRs/MAs (19.8%), and CTs (17.1%). Oral and maxillofacial surgery published the highest number of CRs/case series (54.5%) and COH studies (30.5%), whereas operative dentistry published the lowest number of CRs/case series (0.7%), CCs (2.9%), and SRs/MAs (2.3%). CRs/case series retain the highest number of publications across all time points in the dental literature overall.
    CONCLUSION: Our results indicate an improvement in the types of research and the pyramid of evidence, which will help in applying evidence-based dentistry (EBD) in clinical decision-making.
    CLINICAL SIGNIFICANCE: Types of studies used in the dental field are not yet investigated. Thus, little is known about the common study design types in dental literature. This can affect the decision made regarding technique, risk factors, prevention, or treatment.
    Keywords:  Case controls; Case reports; Clinical trials; Cohort designs; Cross-sectional studies; Dental; RCTs Type of articles.; Randomized clinical trials
  14. J Insect Sci. 2019 Sep 01. pii: 6. [Epub ahead of print]19(5):
    Weintraub PG.
      
    DOI:  https://doi.org/10.1093/jisesa/iez093
  15. PLoS Biol. 2019 Oct;17(10): e3000385
    Hutchins BI, Baker KL, Davis MT, Diwersy MA, Haque E, Harriman RM, Hoppe TA, Leicht SA, Meyer P, Santangelo GM.
      Citation data have remained hidden behind proprietary, restrictive licensing agreements, which raises barriers to entry for analysts wishing to use the data, increases the expense of performing large-scale analyses, and reduces the robustness and reproducibility of the conclusions. For the past several years, the National Institutes of Health (NIH) Office of Portfolio Analysis (OPA) has been aggregating and enhancing citation data that can be shared publicly. Here, we describe the NIH Open Citation Collection (NIH-OCC), a public access database for biomedical research that is made freely available to the community. This dataset, which has been carefully generated from unrestricted data sources such as MedLine, PubMed Central (PMC), and CrossRef, now underlies the citation statistics delivered in the NIH iCite analytic platform. We have also included data from a machine learning pipeline that identifies, extracts, resolves, and disambiguates references from full-text articles available on the internet. Open citation links are available to the public in a major update of iCite (https://icite.od.nih.gov).
    DOI:  https://doi.org/10.1371/journal.pbio.3000385