bims-evares Biomed News
on Evaluation of research
Issue of 2019‒09‒29
fourteen papers selected by
Thomas Krichel
Open Library Society


  1. Urology. 2019 Sep 24. pii: S0090-4295(19)30827-1. [Epub ahead of print]
    Nocera AP, Boyd CJ, Boudreau H, Hakim O, Rais-Bahrami S.
      OBJECTIVES: To determine if article Altmetric scores correlate with journal impact factor and citation count in the urologic literature.METHODS: We identified the top 10 most-cited articles for the 15 urology journals with the highest impact factor in 2013 and 2016. Citation count and Altmetric scores were recorded for each of the articles. The journal impact factor and date of Twitter account development was recorded for each of the journals. The variables were analyzed in Microsoft excel using Pearson's correlation testing.
    RESULTS: A total of 300 articles were analyzed. In 2013, Altmetric scores and citation number showed a significant positive correlation (r=0.164, p=0.045), although Altmetric scores did not correlate with journal impact factor (r=0.005, p=0.957). In 2016, there was significant positive correlation between Altmetric scores and citation number (r=0.268, p=0.0009), as well as between Altmetric scores and journal impact factor (r=0.201, p=0.014). The total citation count decreased from 15,235 in 2013 to 8,622 in 2016 while the total Altmetric score increased from 1,135 in 2013 to 2,563 in 2016. Older Twitter accounts were not associated with increasing correlations between Altmetric score and bibliometrics in either 2013 (r=0.221, p=0.54) or 2016 (r=0.083, p=0.819).
    CONCLUSIONS: At this point in time, Altmetric score is only weakly correlated with citation counts in the urology literature. Altmetrics and traditional bibliometrics should be viewed as complements to one another rather than surrogates when determining research dissemination and impact.
    Keywords:  Altmetric; bibliometric; citations; peer-review; social media
    DOI:  https://doi.org/10.1016/j.urology.2019.09.014
  2. Stud Hist Philos Sci. 2019 Aug;pii: S0039-3681(18)30004-9. [Epub ahead of print]76 49-59
    Harnagel A.
      This paper provides an account of mid-level models which calibrate highly theoretical agent-based models of scientific communities by incorporating empirical information from real-world systems. As a result, these models more closely correspond with real-world communities, and are better suited for informing policy decisions than extant how-possibly models. I provide an exemplar of a mid-level model of science funding allocation that incorporates bibliometric data from scientific publications and data generated from empirical studies of peer review into an epistemic landscape model. The results of my model show that on a dynamic epistemic landscape, allocating funding by modified and pure lottery strategies performs comparably to a perfect selection funding allocation strategy. These results support the idea that introducing randomness into a funding allocation process may be a tractable policy worth exploring further through pilot studies. My exemplar shows that agent-based models need not be restricted to the abstract and the apriori; they can also be informed by empirical data.
    DOI:  https://doi.org/10.1016/j.shpsa.2018.12.010
  3. Account Res. 2019 Sep 24.
    Valderrama Á, Jiménez-Contreras E, Valderrama P, Escabias M, Baca P.
      It is generally accepted that the Journal Impact Factor is a quality criterion. The objective was to determine the evolution along the period 2010-2016 of number of different types of papers, reviews and clinical trials, published by dental journals, as well as if they are related with the quartile occupied in the Journal Impact Factor 2017 ranking. To this end, ten journals per quartile belonging to the field Dentistry, Oral Surgery and Medicine in the 2017 Journal Citation Reports were randomly selected. For each journal and year, the total number of narrative reviews, systematic reviews (with and without meta-analysis), meta-analysis, clinical trials and randomized controlled trials were obtained from Pubmed. To achieve our goal, the slope of these variables over time was estimated using the least squares method, after which one-way analysis of variance of mean values was performed. In Dentistry, the journals of the top quartiles show a trend to publish increasing amounts of systematic reviews and meta-analysis, than the ones of the third and fourth quartile. On the other hand, globally, there was virtually no increase in narrative reviews, clinical trials and randomized controlled trial. Possible causes of this behavior are also discussed in this article.
    Keywords:  Dentistry; Impact factor; analysis of variance; clinical trial; meta-analysis; systematic reviews
    DOI:  https://doi.org/10.1080/08989621.2019.1672541
  4. BMJ Glob Health. 2019 ;4(5): e001692
    Kokabisaghi F, Miller AC, Bashar FR, Salesi M, Zarchi AAK, Keramatfar A, Pourhoseingholi MA, Amini H, Vahedian-Azimi A.
      International research collaborations improve individual, institutional and governmental capacities to respond to health crises and inequalities but may be greatly affected by political environments. Iran ranks highly in tertiary education, productivity growth, knowledge impact and successful patent applications. In many countries, economic hardship has correlated with increased international research collaborations. Some have hypothesised that financial constraint drives scholars to seek outside collaborations for cost and risk sharing, and to access funding, materials and patient populations otherwise unavailable. This paper explores the history and importance of US political sanctions on the health of Iran's academic sector. Although Iran's international research collaborations increased during periods of increased sanctions, the Pearson correlation coefficient between gross domestic product and international research collaborations was not significant (r=0.183, p=0.417). This indicates that other factors are at least in part responsible. Additionally, we found Iran's quantitative (eg, publication number) and qualitative (eg, visibility indices) publishing metrics to be discordant (two-tailed Mann-Kendall trend; p<0.0002 for both). Reasons for this are multifactorial, including increased indexing of Iranian journals, willingness of lower visibility journals to handle manuscripts with Iranian authors, widespread linkage of career advancement to science visibility indices, and others. During periods of increased sanctions, Iranian scholars were increasingly denied opportunities to publish scientific findings, attend scientific meetings, access to essential medical and laboratory supplies and information resources. We conclude that academic boycotts violate researchers' freedom and curtail progress. Free exchange of ideas irrespective of creed is needed to optimize global scientific progress.
    Keywords:  health policy; health services research; international research collaboration; iran; political sanction; public health; united states
    DOI:  https://doi.org/10.1136/bmjgh-2019-001692
  5. PLoS One. 2019 ;14(9): e0222194
    Waqas A, Salminen J, Jung SG, Almerekhi H, Jansen BJ.
      Internet and social media participation open doors to a plethora of positive opportunities for the general public. However, in addition to these positive aspects, digital technology also provides an effective medium for spreading hateful content in the form of cyberbullying, bigotry, hateful ideologies, and harassment of individuals and groups. This research aims to investigate the growing body of online hate research (OHR) by mapping general research indices, prevalent themes of research, research hotspots, and influential stakeholders such as organizations and contributing regions. For this, we use scientometric techniques and collect research papers from the Web of Science core database published through March 2019. We apply a predefined search strategy to retrieve peer-reviewed OHR and analyze the data using CiteSpace software by identifying influential papers, themes of research, and collaborating institutions. Our results show that higher-income countries contribute most to OHR, with Western countries accounting for most of the publications, funded by North American and European funding agencies. We also observed increased research activity post-2005, starting from more than 50 publications to more than 550 in 2018. This applies to a number of publications as well as citations. The hotbeds of OHR focus on cyberbullying, social media platforms, co-morbid mental disorders, and profiling of aggressors and victims. Moreover, we identified four main clusters of OHR: (1) Cyberbullying, (2) Sexual solicitation and intimate partner violence, (3) Deep learning and automation, and (4) Extremist and online hate groups, which highlight the cross-disciplinary and multifaceted nature of OHR as a field of research. The research has implications for researchers and policymakers engaged in OHR and its associated problems for individuals and society.
    DOI:  https://doi.org/10.1371/journal.pone.0222194
  6. Soc Stud Sci. 2019 Sep 27. 306312719880017
    Hamann J.
      How do academics become professors? This paper considers the making of 'professor' as a subject position through which academics are acknowledged in both organizational contexts and disciplinary fields. The paper examines social processes of recognition in 145 appointment procedures for professorships in the discipline of history at sixteen German universities between 1950 and 1985. Based on an analysis of over 1500 documents from archived appointment records, I investigate how academics are acknowledged as professorial in appointment procedures. The procedures invoked both (1) processes of judgement, in which worth and qualities are attributed to candidates, and (2) processes of legitimation, in which said judgements are stabilized and made acceptable. Using insights from the sociology of valuation and evaluation, this paper sheds light on the fundamental processes of recognition and valorization in academia. The findings contribute to the sociology of scientific knowledge and science and technology studies, which have concentrated on academic recognition in the realm of research, but paid less attention to such recognition in organizational contexts. Complementing this literature, the paper allows for a more general understanding of 'professor' as a focal academic subject position.
    Keywords:  Germany; evaluation; judgement; legitimation; positioning; professors
    DOI:  https://doi.org/10.1177/0306312719880017
  7. Asian Pac J Cancer Prev. 2019 09 01. pii: 88722. [Epub ahead of print]20(9): 2789-2794
    Ro DY, Kim DH, Park SH.
      Objective: This study aimed to compare the research trend regarding gynecologic malignancies in North Korean medical journal and South Korean medical journal. Methods: Articles published in the journal of “Pediatrics, Obstetrics, and Gynecology” in North Korea and “Obstetrics & Gynecology Science” in South Korea from 2006 to 2016 were analyzed by using frequency analysis. Studies on gynecologic malignancies were classified by international classification of disease (ICD-10). Results: Out of 3361 reviewed articles, 116 articles published in North Korean journal and 519 articles published in South Korean medical journal were classified as gynecologic oncology. We found a distinct difference between North and South Korean medical journals regarding research trends on gynecologic oncology. The proportions of gestational trophoblastic disease, cervical cancer, and anogenital warts were higher in North Korean medical journal, but proportions of ovarian cancer, fallopian tube cancer, peritoneal cancer, corpus uterine cancer, and vulvar cancer were higher in South Korean medical journal. Conclusion: This study enforced an analysis of research trends on gynecologic malignancies in North Korean and South Korea medical journals, and a distinct difference was observed in this regard. In the future, grand scale cohort study in the genetic identical two Korean population is needed for research of environmental effect on gynecologic cancer.
    Keywords:  North Korea; genetic identity; gynecologic cancer; research trends; socioeconomic disparity
    DOI:  https://doi.org/10.31557/APJCP.2019.20.9.2789
  8. J Extra Corpor Technol. 2019 Sep;51(3): 131-132
    Wong RK.
      
  9. Can J Surg. 2019 Sep 23. 62(6): 016418
    Tran A, Gawad N, Martel A, Manhas N, Allen M, Hameed M, Balaa F.
      Background: Little is known regarding the research and training expectations faced by modern general surgery graduates interested in pursuing academic surgical careers. In this study, we describe the changing face of the Canadian academic general surgeon by outlining the in-residency research productivity and postresidency clinical and academic training trends over time.Methods: Our cross-sectional cohort included Canadian academic general surgeons, defined as those with a university-affiliated appointment as assistant, associate or full professor. Academic surgeons were identified by the Royal College of Physicians and Surgeons of Canada online directory as well as directories of university and hospital websites. Data points included institution, faculty appointment and rank, graduation year, graduate education, fellowship training and research productivity.
    Results: Our cohort included 417 surgeons from 17 Canadian academic institutions. The majority of surgeons were male (72.9%), had completed at least 1 fellowship (72.9%) and had had some form of supplementary research training (51.8%). Surgeons in the cohort had practised a median of 17 (10–27) years. The mean number of total and first-author publications for the participants in this study has increased consistently each decade before the 1980s (p < 0.001). The proportion of academic surgeons completing graduate degrees has increased steadily every decade, reaching a peak of 61.5% for surgeons graduating in the 2010s.
    Conclusion: The Canadian academic surgeon is becoming increasingly productive in research during residency and is pursuing higher levels of graduate education and more fellowships than ever before. These changes probably correspond to an evolving employment and research funding landscape that places tremendous academic pressure on surgical trainees.
    DOI:  https://doi.org/10.1503/cjs.016418
  10. Chemistry. 2019 Sep 26.
    Ross H.
      How can the importance of a journal possibly be pinned down to a single number? The 2018 Impact Factors were released on June 20, 2019. In this editorial Haymo Ross, Editor-in-Chief of the Chemistry-A European Journal, shares some his thoughts on this topic.
    Keywords:  General Chemistry
    DOI:  https://doi.org/10.1002/chem.201903777
  11. Stud Hist Philos Sci. 2019 Aug;pii: S0039-3681(18)30019-0. [Epub ahead of print]76 13-23
    Avin S.
      In 2013 the Health Research Council of New Zealand began a stream of funding titled 'Explorer Grants', and in 2017 changes were introduced to the funding mechanisms of the Volkswagen Foundation 'Experiment!' and the New Zealand Science for Technological Innovation challenge 'Seed Projects'. All three funding streams aim at encouraging novel scientific ideas, and all now employ random selection by lottery as part of the grant selection process. The idea of funding science by lottery emerged independently in several corners of academia, including in philosophy of science. This paper reviews the conceptual and institutional landscape in which this policy proposal emerged, how different academic fields presented and supported arguments for the proposal, and how these have been reflected (or not) in actual policy. The paper presents an analytical synthesis of the arguments presented to date, notes how they support each other and shape policy recommendations in various ways, and where competing arguments highlight the need for further analysis or more data. In addition, it provides lessons for how philosophers of science can engage in shaping science policy, and in particular, highlights the importance of mixing complementary expertise: it takes a (conceptually diverse) village to raise (good) policy.
    DOI:  https://doi.org/10.1016/j.shpsa.2018.11.006
  12. Eur Arch Paediatr Dent. 2019 Sep 27.
    Garcovich D, Marques Martinez L, Adobes Martin M.
      PURPOSE: To perform a bibliometric analysis on the 100 most-cited articles to provide a historical perspective in the scientific advancement of research, and to identify the areas of more intense research. To determine the correlation of the Altmetric Attention Score (AAS) with the number of citations.METHODS: The articles were identified through a search of the Web Of Science (WOS) database. The information relative to the articles was collected. Descriptive statistics was used to describe the articles and the journals. The Pearson's correlation analysis was used to explore the relationship among WOS citations, AAS and dimensions.
    RESULTS: The most-cited articles were published between 1967 and 2013. The articles were published in 18 different journals and cited from 69 to 305 times. The authors belonged to institutions in 20 different countries. The most common study design was observational followed by literature reviews.
    CONCLUSIONS: The journals with a special interest in paediatric dentistry published the majority of the 100 most-cited articles. The most frequent field of study was caries and EEC. MIH and OHRQL displayed a high number of citations per year. A poor correlation exists between the citation count in the WOS and the AAS. Dimension citation count has a strong correlation with the WOS one.
    Keywords:  Altmetric; Bibliometrics; Citation analysis; Dimensions; Paediatric dentistry
    DOI:  https://doi.org/10.1007/s40368-019-00483-z
  13. Rev Saude Publica. 2019 ;pii: S0034-89102019000100271. [Epub ahead of print]53 81
    Barbosa MMAL, Cuenca AMB, Oliveira K, França Junior I, Alvarez MDCA, Omae LY.
      OBJECTIVE: To describe the most-cited articles in public health scientific journals edited in Brazil.METHODS: Articles published between 2008 and 2010 by public health journals edited in Brazil and indexed in the Scopus database were included, and citations received up to five years after publication were ranked. We studied a total of 105 articles, as the last seven articles shared the same number of citations and so were given the same rank.
    RESULTS: The most-cited articles received a median of 28 citations, and the distribution ranged from 22 to 95 citations. These articles describe advances in the areas of Epidemiology (74%), Health Policies, Planning and Administration (19%), and Social and Human Sciences in Health (7%). Only half mentioned that they have received funding. About 75% of the articles were written by three or more authors and 90%, by authors affiliated to public institutions such as universities and government organizations. Fifteen individuals were responsible for authoring or coauthoring three or more of the 105 articles studied. The journals Cadernos de Saúde Pública, Revista de Saúde Pública, and Ciência & Saúde Coletiva have published the vast majority of the most-cited articles in the area (94%).
    CONCLUSIONS: In Brazil, the most-cited articles in public health mainly report Epidemiology research, are written by groups of authors and by researchers affiliated to public institutions and are published in journals with a greater impact. Periodical analyses of these data can show potential changes in the characteristics of articles that most attract public health scientists.
    DOI:  https://doi.org/10.11606/s1518-8787.2019053001554