bims-evares Biomed News
on Evaluation of research
Issue of 2019‒12‒08
fourteen papers selected by
Thomas Krichel
Open Library Society


  1. Proc Biol Sci. 2019 Dec 04. 286(1916): 20192047
      Research is a highly competitive profession where evaluation plays a central role; journals are ranked and individuals are evaluated based on their publication number, the number of times they are cited and their h-index. Yet such evaluations are often done in inappropriate ways that are damaging to individual careers, particularly for young scholars, and to the profession. Furthermore, as with all indices, people can play games to better their scores. This has resulted in the incentive structure of science increasingly mimicking economic principles, but rather than a monetary gain, the incentive is a higher score. To ensure a diversity of cultural perspectives and individual experiences, we gathered a team of academics in the fields of ecology and evolution from around the world and at different career stages. We first examine how authorship, h-index of individuals and journal impact factors are being used and abused. Second, we speculate on the consequences of the continued use of these metrics with the hope of sparking discussions that will help our fields move in a positive direction. We would like to see changes in the incentive systems, rewarding quality research and guaranteeing transparency. Senior faculty should establish the ethical standards, mentoring practices and institutional evaluation criteria to create the needed changes.
    Keywords:  academic ethics; academic evaluation; academic standards; h-index; impact factors; publishing practices
    DOI:  https://doi.org/10.1098/rspb.2019.2047
  2. Elife. 2019 Dec 06. pii: e52646. [Epub ahead of print]8
      Preprints in biology are becoming more popular, but only a small fraction of the articles published in peer-reviewed journals have previously been released as preprints. To examine whether releasing a preprint on bioRxiv was associated with the attention and citations received by the corresponding peer-reviewed article, we assembled a dataset of 74,239 articles, 5,405 of which had a preprint, published in 39 journals. Using log-linear regression and random-effects meta-analysis, we found that articles with a preprint had, on average, a 49% higher Altmetric Attention Score and 36% more citations than articles without a preprint. These associations were independent of several other article- and author-level variables (such as scientific subfield and number of authors), and were unrelated to journal-level variables such as access model and Impact Factor. This observational study can help researchers and publishers make informed decisions about how to incorporate preprints into their work.
    Keywords:  computational biology; none; systems biology
    DOI:  https://doi.org/10.7554/eLife.52646
  3. PeerJ. 2019 ;7 e8145
      Background: This study aimed to draw a series of scientific maps to quantitatively and qualitatively evaluate hot spots and trends in postmenopausal osteoporosis research using bibliometric analysis.Methods: Scientific papers published on postmenopausal osteoporosis were extracted from the Web of Science Core Collection and PubMed database. Extracted information was analyzed quantitatively with bibliometric analysis by CiteSpace, the Online Analysis Platform of Literature Metrology and Bibliographic Item Co-Occurrence Matrix Builder (BICOMB). To explore the hot spots in this field, co-word biclustering analysis was conducted by gCLUTO based on the major MeSH terms/MeSH subheading terms-source literatures matrix.
    Results: We identified that a total of 5,247 publications related to postmenopausal osteoporosis were published between 2013 and 2017. The overall trend decreased from 1,071 literatures in 2013 to 1,048 literatures in 2017. Osteoporosis International is the leading journal in the field of postmenopausal osteoporosis research, both in terms of impact factor score (3.819) and H-index value (157). The United States has retained a top position and has exerted a pivotal influence in this field. The University of California, San Francisco was identified as a leading institution for research collaboration, and Professors Reginster and Kanis have made great achievements in this area. Eight research hot spots were identified.
    Conclusions: Our study found that in the past few years, the etiology and drug treatment of postmenopausal osteoporosis have been research hot spots. They provide a basis for the study of the pathogenesis of osteoporosis and guidelines for the drug treatment of osteoporosis.
    Keywords:  Bibliometric analysis; Biclustering; Co-word analysis; Hot spots; Mapping theme trends; Postmenopausal osteoporosis
    DOI:  https://doi.org/10.7717/peerj.8145
  4. J Family Med Prim Care. 2019 Nov;8(11): 3475-3479
      Currently, there is an increased dependency on the impact factor in scientific research publications. Sometimes the overzealousness in implementing the impact factor value to a publication is detrimental to the growth of the scientific authors, especially the junior lot, for no fault of theirs. The inept and myopic application of the impact factor defeats the purpose of making the value judgment and hence has been criticized by many learned authors. The scientist Eugene Garfield, who formulated the impact factor, feels wrong that it is being not judiciously used. A corollary is the invention dynamite by Alfred Nobel which instead of helping, aiding, and reducing human effort and endeavor it was more misused for human annihilation and pained the scientist who gave it to the world. The authors reexamine the application of impact factor to scientific manuscripts for rightful application of the value judgment.
    Keywords:  Predatory journals; Research integrity; Science Citation Index; Thomson Reuters
    DOI:  https://doi.org/10.4103/jfmpc.jfmpc_515_19
  5. Evid Based Ment Health. 2019 Dec 04. pii: ebmental-2019-300122. [Epub ahead of print]
      Metrics which quantify the impact of a scientist are increasingly incorporated into decisions about how to rate and fund individuals and institutions. Several commonly used metrics, based on journal impact factors and citation counts, have been criticised as they do not reliably predict real-world impact, are highly variable between fields and are vulnerable to gaming. Bibliometrics have been incorporated into systems of research assessment but these may create flawed incentives, failing to reward research that is validated, reproducible and with wider impacts. A recent proposal for a new standardised citation metric based on a composite indicator of 6 measures has led to an online database of 100 000 of the most highly cited scientists in all fields. In this perspective article, we provide an overview and evaluation of this new citation metric as it applies to mental health research. We provide a summary of its findings for psychiatry and psychology, including clustering in certain countries and institutions, and outline some implications for mental health research. We discuss strengths and limitations of this new metric, and how further refinements could align impact metrics more closely with wider goals of scientific research.
    Keywords:  adult psychiatry; child & adolescent psychiatry
    DOI:  https://doi.org/10.1136/ebmental-2019-300122
  6. BMC Med Educ. 2019 Dec 03. 19(1): 449
      BACKGROUND: Scientometric studies are one of the most important and useful tools to assess the research performance and knowledge impact of researchers. The aim of this study was to map out the scientific performance of the Iranian medical academics with respect to a detailed range of scientometric indicators.METHODS: Using scientometric approach, individual and scientific performance data of medical academic staff were extracted from the Iranian Scientometric Information Database (ISID). Total number of publications, total number of citations, citation per paper, h-index, international collaboration, self-citation, SJR decile, i10-Index, Quartile distribution were the studied scientometric variables. Out of the registered 19,023 academic staff, 746 were included in the study through simple random sampling method using random sample extraction function in STATA. Data were analyzed using STATA 14 statistical software package.
    RESULTS: Most of the included academicians were men (60%). A total of 13,682 articles were published by them until 2018, being cited 114,928 times with a mean of 5.77 citation per paper. H-index median was three and about 90% of the staff had an H-index below 10. Number of published papers, cite per paper and H-index metrics were significantly different with respect to gender, academic position/degree, and general field of study (p < 0.05). About 2.5% of published articles were contributed through international collaboration. The scientometric performance of academic staff was highly diverse with respect to the employing institution and its national classification group (type 1, 2, 3).
    CONCLUSIONS: Nevertheless to the great scientific production of medical academics, individual and institutional characteristics were identified as effective variables in academics research performance and should be considered in their assessment. Academicians affiliated with type 2 and 3 universities (based on national ranking of medical universities) had weaker research performance compared to those affiliated with type 1 universities. However, low rate of international research collaborations was a common challenge in medical universities.
    Keywords:  Evaluation; Medical research; Research performance; Scientometry
    DOI:  https://doi.org/10.1186/s12909-019-1892-4
  7. Laryngoscope. 2019 Dec 04.
      OBJECTIVE: To examine trends in female author representation within original otolaryngology research between 2000 and 2015.METHODS: Original research articles published in 11 otolaryngology journals were analyzed for 2000, 2003, 2006, 2009, 2012, and 2015. The genders of the first and last authors for each article were recorded. Overall female authorship was calculated by summing the numbers of the first, last, and both first and last female-authored articles. Student t test and Cochran-Armitage trend test were utilized to determine significance between years and groups.
    RESULTS: Of the 9,623 research articles published during 5 representative years, 223 were excluded due to one or more gender-indeterminate authors. Female first authorship exhibited a significant upward trend from 2000 to 2015 (P < 0.0001), as did the proportion of literature with female first and last authors (P < 0.0001). Although female senior authorship in literature with an impact factor (IF) greater than 2 did not increase significantly (10.0% in 2000 to 10.1% in 2015; P = 0.738), this metric did increase significantly just within journals with an IF between 1 and 2 (9.7%-12.3%, P = 0.036). The proportion of articles with a female author in the first, last, or both positions increased from 28% to 39% (P < 0.0001).
    CONCLUSION: Increasing female representation in otolaryngology literature may reflect the rising proportion of women within otolaryngology as well as greater mentorship availability. Despite these auspicious strides, female-authored articles nonetheless represent a smaller proportion of the literature, and female senior authors remain a stark minority. Future studies should identify the barriers to female access and advancement within the field.
    LEVEL OF EVIDENCE: III Laryngoscope, 2019.
    Keywords:  Otolaryngology; literature; research; surgery; women
    DOI:  https://doi.org/10.1002/lary.28372
  8. Cureus. 2019 Oct 21. 11(10): e5955
      Family medicine is one of the most vital health specialties in the field of medicine. This can be attributed to the wide range of health services for all people regardless of age, gender, and diagnosis. Saudi Arabia as suggested by various studies put more of its attention at all levels to family medicine in order to produce an adequate number of family physicians and improve both the academic aspects and the services provided by family medicine in the country. Thus, a comprehensive national survey was suggested to analyze the current situation of the specialty in the country and to draw up a strategic plan to achieve the national vision for family medicine by 2020. In light of the aforementioned, this study deemed it necessary to examine family medicine research productivity in Saudi Arabia in order to recommend possible measures to increase the quality of research output related to the field. The Web of Science (WoS) bibliographic database search engine was used to retrieve and analyze data. The terms "Family Medicine" and "Saudi Arabia" were entered in the search address fields using the SAME boolean operator (i.e., Family Medicine SAME Saudi Arabia) to retrieve records that contained these two terms and were in the same address. Research articles that were published from 1 January 2004 to 31 December 2018 were included for data analysis. Publication details such as the year of publication, document type, research area, authors' affiliation, journal name, international collaborators, journal impact factor (JIF), and citation reports were all considered in the analysis. As for the result, Saudi Arabia's research productivity in the field of family medicine is very poor, with a very gradual linear increase over the last 15 years. Further, health institutions have low research productivity compared to universities and medical colleges. Finally, most of the publications were published in low- or no-impact-factor journals. Therefore, this study concludes that Saudi Arabia has had low research productivity in family medicine. It recommends that an educational research program with the supervision of Ministry of Health (MOH) be implemented among family medicine physicians featuring organizational support and well-structured communication between health and educational institutions to increase research productivity in the country.
    Keywords:  family medicine; research productivity; saudi arabia
    DOI:  https://doi.org/10.7759/cureus.5955
  9. J Nurs Manag. 2019 Dec 06.
      AIM: This study presents a general bibliometric overview of the developmental status of Journal of Nursing Management from its first issue (1993) to 2018.BACKGROUND: Bibliometric method is proved to be effective in analyzing the publication characteristics and influence of a given journal and it gained considerable interest from the scientific community in recent years. However, to the best of our knowledge, bibliometric approach was not previously used to analyze the publications of Journal of Nursing Management.
    METHODS: 1913 publications of Journal of Nursing Management were retrieved from Scopus database. Bibliometric analysis was used to explore the dynamic trends of publications and to identify most influential authors, institutions, countries and research teams in this journal. This paper also uses the VOSviewer software to graphically map the hot topics of Journal of Nursing Management.
    RESULTS: After a rapid growth from 2006 to 2008, the number of publications of Journal of Nursing Management tend to stabilize with exception of several years in the latest 11 years. The top prolific institutions are mainly from the UK, Canada, Finland, and Sweden, and Journal of Nursing Management has attracted increasing of attention from researchers around the world. Nursing, nurses, management, leadership and job satisfaction are the most frequently used keywords in Journal of Nursing Management.
    CONCLUSIONS: Nursing management, nursing leadership/leaders, nursing human resource management, nursing quality and safety management, nursing communication, and conflict management have always been popular topics in Journal of Nursing Management.
    IMPLICATIONS FOR NURSING MANAGEMENT: This study informs scholars and managers within nursing management field about structured knowledge of the research status and development of Journal of Nursing Management.
    Keywords:  Journal of Nursing Management; bibliometrics; citation analysis; nursing; scientific outputs
    DOI:  https://doi.org/10.1111/jonm.12925
  10. Urology. 2019 Dec 01. pii: S0090-4295(19)31088-X. [Epub ahead of print]
      OBJECTIVE: To characterize the current landscape and future directions of academic benign prostatic hyperplasia (BPH) by using bibliometric analysis.METHODS: We used the Web of Science Core CollectionTM to conduct a bibliometric analysis of leading BPH articles. Bibliometric analyses are quantitative approaches examining the impact of academic literature. We used the following search terms and Boolean logic "("benign prostat*") AND (hyperplasia OR enlarg*)" and characterized the 100 most-cited BPH articles through 2018 including citations, journal, author, year, and country.
    RESULTS: The top 100 BPH articles were published between 1978 and 2012. Citations ranged from 153 to 2,171 across 27 different journals, including 10 urology-specific journals. The Journal of Urology was the most published journal (n=25), followed by European Urology (n=17), and Urology (n=15). In general, the oldest 10 articles focused on BPH etiology/pathogenesis, while the newest 10 focused on treatment. The 1990's was the most productive decade with nearly half of the top 100 articles (n=44). Twenty-six different countries contributed to the top 100 articles, with the US (n= 74), Italy (n=19), and Canada (n=12) being the most common.
    CONCLUSIONS: This study represents the first bibliometric analysis of the leading BPH articles impacting the academic literature. The focus has evolved from BPH pathogenesis to treatment, perhaps reflecting a shift in research funding and capacity. These findings may guide research priorities for this increasingly common condition.
    Keywords:  Research; citation analysis; funding; impact; prostate; trends
    DOI:  https://doi.org/10.1016/j.urology.2019.11.033
  11. Dent Traumatol. 2019 Dec 04.
      The impact of a scientific article in its respective field is reflected by its citation count. The purpose of this review was to conduct a citation analysis in order to identify and analyze the top 50 most-cited articles published in Dental Traumatology since its inception in order to highlight the contribution of the journal to the field of Dental Traumatology. Elsevier's Scopus was used to search and analyze the 50 most frequently cited scientific papers. After the screening process, two reviewers arranged the articles in a descending order based on their citation counts. Each article was then cross-matched with Google Scholar. The articles were analyzed and information including citation counts, citation density, publication year, authorship, contributing institutions and countries, article topic, study design and keywords was extracted. The literature search identified 2421 articles. The citation counts of the 50 selected articles varied from 117 to 580 (Scopus) and 206 to 1130 (Google Scholar). The year in which most top 50 articles were published was 2002 (n=5). Among 105 authors, the greatest contribution was made by JO Andreasen (n=12). Most of the articles originated from the United States (n=12) with the greatest contributions from the University Hospital (Rigshospitalet), Copenhagen, Denmark (n=6). Original research article was the most frequent study design (n=34). The majority of the top 50 articles were focused on traumatic dental injuries. Among 131 unique key words, root resorption (n=6) was the most frequently used. A non-significant correlation occurred between citation count (correlation coefficient = 0.127, p = 0.378), citation density (correlation coefficient = 0.654, p = 2.493) and publication age. This study identified the top 50 most-cited articles published in this journal in the specialty of dental traumatology. The publication year of an article was not significantly associated with citation count nor citation density.
    Keywords:  50 most cited; Dental Traumatology; bibliometric analysis; citation analysis; classic article
    DOI:  https://doi.org/10.1111/edt.12534
  12. J Orthop Surg Res. 2019 Dec 04. 14(1): 412
      BACKGROUND: Over the past few decades, more and more articles about total hip arthroplasty have been published. We noticed, however, little is known about the characteristics and qualities of these studies.METHODS: The databases of Web of Science Core Collection, BIOSIS Citation Index, MEDLINE, etc. were utilized for the identification of articles published from 1990 to May 2019. Total hip arthroplasty-related articles were identified, and the 100 most cited articles were selected for subsequent analysis of citation count, citation density (citations/article age), authorship, theme, geographic distribution, time-related flux, level of evidence, and network analysis.
    RESULTS: The selected 100 articles were published mainly in the 1990s (46%) and 2000s (47%) with almost equal amount. Citations per article ranged from 994 to 191. Leading countries were the USA followed by Canada, England, and Sweden, all located in North America and Western Europe. The most highlighted study themes were postoperative thrombosis and surgical methods and materials. The most common level of evidence was level III (35%). The network analysis connoted that radiography, acetabulum, reoperation, and bone cement had a high degree of centrality in the 1990s, while cement had a high degree of centrality in the 2000s and 2010s.
    CONCLUSIONS: The time, area, and theme distribution of the top 100 most cited articles in the total hip arthroplasty have been thoroughly analyzed. It is noticeable that postoperative thromboembolism currently plays a major role in the field of total hip arthroplasty researches. However, most of them focus on the effectiveness of different treatments and drugs; little is known about its underlying mechanisms and influencing factors.
    Keywords:  Bibliometric analysis; Total hip arthroplasty; Total hip replacement
    DOI:  https://doi.org/10.1186/s13018-019-1476-3
  13. Am J Cardiol. 2019 Nov 09. pii: S0002-9149(19)31240-8. [Epub ahead of print]
      Citation classics are widely being implemented in the field of medical and scientific research assessment. The frequency of citation of a particular article is used to gauge its contribution and impact on the clinical world of practice and research. A thorough search of the literature showed a lack of bibliometric analysis on atrial fibrillation (AF). Thereby the main purpose of this study is to pinpoint the trend of the top 100 cited articles on AF. In June 2019, 2 databases, Scopus and Web of Science, were used to acquire the articles published on AF, which belonged to various genres including medicine, medical and interventional cardiology, electrophysiology, and thoracic surgery. The data were thoroughly reviewed and analyzed by 2 reviewers with regards to the number of citations for each article, publications per area, document type, first author name, country of origin, institute of origin, and year of publication. Approval of the Institutional Board Review was not required as we used publically available data retrospectively. The number of citations in the top 100 articles ranged from 622 to 6,641 times with an average citation of 1041.3 per article. The most significant number of articles was published in the year 2004 which ranged up to 11 in total. All the articles are published in 21 English language journals. In these 100 articles, the most were from the United States (n = 63) followed by Canada (n = 24), and other countries (n = 17). The top 5 institutions include McMaster University (n = 9), Institut de Cardiologie de Montreal (n = 8), Population Health Research Institute (n = 8), Harvard Medical School (n = 7), and Mayo Clinic (n = 7). In conclusion, authors seeking to publish a highly referenced article on AF will be determined by source journal, the language of publication, geographic origin, methodology, or research outcome.
    DOI:  https://doi.org/10.1016/j.amjcard.2019.10.053
  14. Hipertens Riesgo Vasc. 2019 Nov 27. pii: S1889-1837(19)30074-1. [Epub ahead of print]
      INTRODUCTION: Argentine science has played an important role in the study of blood pressure. However, this scientific production has not been classified. We set out (1) to analyse the contribution of scientific publications indexed in MEDLINE of authors with Argentinean academic affiliation in the field of blood pressure and hypertension in the last 50 years and, (2) determine the characteristics of the scientific journals in which they were published.METHODS: The 831 indexed MEDLINE publications by authors from Argentina were analysed quantitatively and qualitatively (period 1966-2017).
    RESULTS: The number of publications has increased 5.4 times in the last 20 years. Eighty percent of the publications were original manuscripts and 15% reviews. Sixty-five percent of the publications addressed clinical research, 33% basic research. The average authors per paper was 6 (89% as first author), 74% belonged to public institutions. The research was published in journals published in the United States (36%), the United Kingdom (27%), the Netherlands (12%), Spain (6%) and Argentina (4%). Eighteen percent of the publications were in journals with impact factor >3.88 (first quartile). Only 5% accessed journals with a factor ≥10. The average SJR index was 1.66.
    CONCLUSIONS: Argentine scientific production in MEDLINE in the field of blood pressure and hypertension showed constant growth. The vast majority is original research, directed by researchers with affiliation to public institutions. Foreign journals are accessed in the main, with acceptable quality indexes.
    Keywords:  Análisis bibliométrico; Argentina; Bibliometric analysis; Blood pressure; High blood pressure; Hipertensión arterial; Presión arterial; Producción científica; Scientific output
    DOI:  https://doi.org/10.1016/j.hipert.2019.09.004