Progression of citation-based scholarly activity from postgraduate year one to postgraduate year two in a current resident class

Introduction

Whether via the Hirsch index (h-index) (1-3), h-index limited to first author manuscripts (hf) (4,5), h-index limited to first or second author manuscripts (hs) (5), or the g-index (6,7), citation-based scholarly activity has been shown to correlate with resident choice of academic versus private practice jobs (3,5,7). Consequently, further information regarding citation-based scholarly activity among residents may be important for understanding the future trajectory of the academic workforce. The majority of citation-based activity studies involving residents have investigated activity at the culmination of residency, with no previous investigation of activity progression during residency. This study is the first to examine the evolution of citation-based scholarly activity during residency by utilizing a current resident class to evaluate the hypothesis of whether citation-based scholarly activity changes over time. We present the following article in accordance with the MDAR reporting checklist (available at https://cco.amegroups.com/article/view/10.21037/cco-21-147/rc).

Methods

A compilation of United States residents from the graduating radiation oncology class of 2022 [postgraduate year two (PGY-2) academic year of 2018–2019] was obtained as previously described through internet searches and communication with residency program coordinators/directors as needed (8); race/ethnicity was identified from interviews and self-described internet depictions as previously described (9). As this dataset did not involve patient information, it was deemed exempt from Indiana University Institutional Review Board (IRB) review. Citation-based scholarly activity was collected and calculated from searches of the Scopus bibliometric citation database for h-index [defined as the number of manuscripts (h) cited at least h times], hf, hs, and g-index [defined as the largest number of manuscripts (g) by an author which have been cited at least (g × g times)] for each resident as previously described (3,5,7). Calculations were derived in June 2018 for the postgraduate year one (PGY-1) year, and in June 2019 for the PGY-2 year, each within a 30-hour period for all residents to account for the dynamic and organic nature of the h-index, hf, hs, and g-index. Following this, the data were coalesced for statistical analysis, with Fisher’s exact test used to evaluate the progression of each measurement over time. The impact of underrepresented minorities (defined as African-American race and/or Hispanic ethnicity as previously described) on findings was conducted in a subsequent analysis (9). Statistical significance was assigned at P<0.05 (GraphPad Software, San Diego, CA, USA).

Results

The resident class of 2022 involved analysis of 195 radiation oncology residents representing 89 programs; this included 134 men, 61 women, and 12 underrepresented minorities. Analysis revealed that the citation-based scholarly activity significantly increased from PGY-1 to PGY-2 for h-index (2.59 to 3.23; P=0.047) and g-index (4.03 to 5.13; P=0.045) (Figure 1), but not for hf (1.03 to 1.28; P=0.169) or hs (1.48 to 1.88; P=0.0654). From the PGY-1 to PGY-2 year, the proportion of residents with hf of zero (no cited articles as first author) decreased from 43.6% to 38.5%, as did the proportion of residents with hs of zero (no cited articles as first or second author) from 32.3% to 26.7%. Underrepresented minority race/ethnicity (African-American/Hispanic) did not impact the significance of the h-index and g-index progression from PGY-1 to PGY-2, while the significance of these index was lost when solely male (n=134) or female (n=61) residents were analyzed.

Figure 1 Depiction of citation-based activity from PGY-1 to PGY-2 as measured by h-index and g-index for the radiation oncology resident class of 2022. The progression for both indices reached statistical significance. PGY-1, postgraduate year one; PGY-2, postgraduate year two.

Discussion

The results from this study indicate that from the PGY-1 to PGY-2 academic year, residents significantly increase in h-index and g-index measurements of citation-based academic productivity (Figure 1). While the significance in resident progression as measured by the h-index and g-index is not surprising, the lack of significance in hf and hs progression is (although hs did trend towards significance). It is possible that the number of manuscripts on which residents are first or second author is small enough relative to the resident published literature in general that the increase in these indices from PGY-1 to PGY-2 year did not meet statistical significance. Investigation into further years will be insightful to determine whether the progression of hf and hs becomes statistically significant over time.

These findings indicate that the progression of citation-based scholarly activity among residents from PGY-1 to PGY-2 year is significant. Further encouraging data is that the number of residents without a manuscript cited as first or second author decreased between PGY-1 and PGY-2, indicating that active publication in addition to increased duration of time for pre-residency publications to be cited played a role in the h-index and g-index progression significance. Despite the inherent nature of the h-index and g-index naturally increasing over time, the fact that resident activity progression is significant so early in training is an encouraging sign, as the vast majority of residents in radiation oncology have minimal direct interaction with radiation oncology patients, providers, or research mentors during PGY-1 (where most are pursuing a transitional year, internal medicine internship or general surgery internship). It is therefore possible that the progression of citation-based scholarly activity from PGY-2 onward may be even greater than what this study has demonstrated occurs between PGY-1 and PGY-2.

Limitations to this study include the inability to capture 100% of the 2022 radiation oncology resident class, and the fact that compilations occurred prior to the start of PGY-2. As a result, residents who matriculated into radiation oncology but before PGY-2 decided to no longer pursue the field would have been included. Additionally, the number of residents analyzed in PGY-2 (n=195) did not equal those analyzed in PGY-1 (n=177); due to the constantly changing nature of the indices measured we were unable to go back to accurately calibrate the sample size of these groups, and as a result of this inequity the Fisher’s exact test rather than the paired t-test was used for analysis. Furthermore, residency programs existing prior to 2019 but subsequently disbanded would be included in this analysis as well. Another limitation is the lag time in publishing, which may have extended beyond the one-year duration encompassing PGY-1 to PGY-2. Finally, as this analysis was performed on radiation oncology residents, the applicability beyond this subspecialty to all residencies may be limited.

In conclusion, from the PGY-1 to PGY-2 academic year, United States residents in a current radiation oncology class significantly increased in citation-based academic productivity, with an increase in the proportion of residents with a cited first-or-second author manuscript. Following completion of the PGY-2 year, the average radiation oncology resident has published more than three manuscripts cited at least three times. Further study is necessary to determine how this trend persists in future years, and to what degree such a trend impacts scholarly activity in light of the relatively limited data on citation-based academic productivity. This study highlights the continued importance and role of research following the start of residency, and that the increasing pressure for publication facing residency applicants may persist even after the start of residency (10). Furthermore, this study provides a benchmark for further investigation of resident citation-based scholarly activity over time.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the MDAR reporting checklist. Available at https://cco.amegroups.com/article/view/10.21037/cco-21-147/rc

Data Sharing Statement: Available at https://cco.amegroups.com/article/view/10.21037/cco-21-147/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://cco.amegroups.com/article/view/10.21037/cco-21-147/coif). SM serves as an unpaid editorial board member of Chinese Clinical Oncology from August 2021 to July 2023 and serves as a consultant for Gilmartin Capital, a company that evaluates surgically targeted radiation therapies. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 2005;102:16569-72. [Crossref] [PubMed]
2. Rana S, Holliday EB, Jagsi R, et al. Scholastic activity among radiation oncology residents at US academic institutions: a benchmark analysis. J Cancer Educ 2013;28:541-6. [Crossref] [PubMed]
3. McClelland S 3rd, Mitin T, Wilson LD, et al. Relationship Between Citation-Based Scholarly Activity of United States Radiation Oncology Residents and Subsequent Choice of Academic Versus Private-Practice Career. Int J Radiat Oncol Biol Phys 2018;101:46-8. [Crossref] [PubMed]
4. Butson MJ, Yu PK. The first author h-index (h(fa)-index): levelling the field for small and large institute medical and science scholars. Australas Phys Eng Sci Med 2010;33:299-300. [Crossref] [PubMed]
5. McClelland S 3rd, Mitin T, Jagsi R, et al. Importance of First and Second Authorship in Assessing Citation-Based Scholarly Activity of US Radiation Oncology Residents and Subsequent Choice of Academic Versus Private Practice Career. J Am Coll Radiol 2018;15:1322-5. [Crossref] [PubMed]
6. Egghe L. Theory and practice of the g-index. Scientometrics 2006;69:131-52. [Crossref]
7. McClelland Iii S, Mitin T, Nabavizadeh N, et al. Use of the g-index for assessment of citation-based scholarly activity of United States radiation oncology residents and subsequent choice of academic versus private practice career. Rep Pract Oncol Radiother 2019;24:294-7. [Crossref] [PubMed]
8. McClelland S 3rd, Thomas CR Jr, Wilson LD, et al. Association of pre-residency peer reviewed publications with radiation oncology resident choice of academic versus private practice career. Pract Radiat Oncol 2017;7:364-7. [Crossref] [PubMed]
9. McClelland S 3rd, Woodhouse KD, Jaboin JJ, et al. Analysis of pre-residency research productivity, dual degree status, and gender distribution of underrepresented minorities among a current United States radiation oncology junior resident class. Rep Pract Oncol Radiother 2019;24:284-7. [Crossref] [PubMed]
10. Mahalingam J, Khatri C, Fitzgerald JE. Pressure of academic publishing for medical students: a student's perspective. Postgrad Med J 2018;94:367-8. [Crossref] [PubMed]