Variation in Journal Prestige and Usage Metrics Across Kinesiology Subdisciplinary Keywords

NoteAuthor Note

Conflict of Interest: The author declares no conflict of interest.

Correspondence: Correspondence concerning this article should be addressed to Duane Knudson, Department of Health & Human Performance, Texas State University, 601 University Drive, San Marcos, TX 78666-4684. Email: dknudson@txstate.edu

Scientific research and publication are increasingly reliant on computer and internet technologies. Modern electronic publishing, archiving, and indexing of scholarly research have replaced traditional printing of scientific journals and proceedings articles. While specialized databases are available (e.g., CINAHL, PsycINFO, SPORTDiscus), searching for articles and bibliometric indicators (e.g., Impact Factor, SCImago Journal Rank) for journal selection are dominated by multidisciplinary subscription services like Scopus and Web of Science (Pranckutė 2021; Waltman 2016) and free services like Google Scholar. This raises numerous questions about the utility and limitations of multidisciplinary database services, particularly their search capabilities, coverage/indexing, and assigned subject-area categories.

Since scholars search bibliometric databases to plan research and select publication outlets, search engine function is important (Gusenbauer and Haddaway 2020; Pranckutė 2021). Computerized search engines of databases vary in their effectiveness. One study of the search engines of 28 databases found that only about half accurately implemented all 27 search logic (e.g., Boolean operators, full text, literal or expanded queries) tests (Gusenbauer and Haddaway 2020). In addition, search systems employ various relevance-ranking algorithms to rank returned records (Delgado-Lopez-Cozar and Cabezas-Clavijo 2013; Rovira et al. 2023; Yamato et al. 2018). Rovira et al. (2023) reported a reverse-engineering study of search algorithms in free (Google Scholar and Microsoft Academic) and subscription (Scopus & Web of Science) databases. They found that searches of free databases returned results strongly associated with citations, whereas searches of subscription databases did not. Rovira and colleagues did, however, find in one time-limited search (May 2019) that Web of Science may have been experimenting with citations in the search algorithm, given a similar Spearman association (rs = 0.9) seen in the Google Scholar and Microsoft Academic searches. Authors searching databases for journal metrics or rankings to support decisions about publication outlets likely receive biased results from search engine functions, as well as incomplete coverage or indexing of journals and articles in any database service used.

The visibility of journals in interdisciplinary fields like kinesiology also depends on indexing and ranked search results in large, multidisciplinary databases. Even large, curated, and multidisciplinary databases do not provide complete coverage of peer-reviewed publications (Aksnes and Sivertsen 2019; Bar-Ilan 2018; Knudson 2022b; Ramos-Remus et al. 1994; Waltman 2016). Kinesiology journals may be disadvantaged in searches of these databases because of inconsistent coverage/indexing of their broad scope and interdisciplinary nature. Searching databases for kinesiology research may yield biased results because there is no unique kinesiology subject area among the wide variety of subject-area categories created by each database. The growing audit culture in higher education and the proliferation of journal citation metrics that rank journals using inconsistent subject-area categories are problematic for kinesiology and some of its subdisciplines (Cardinal et al. 2014; Knudson 2022b; Knudson et al. 2024; Phillips 2020; Sperka and Phillips 2022). Several studies have begun to explore this visibility problem for kinesiology-related journals and research.

Freeland et al. (2014) reported differences in the 2011 citations reported by Scopus across four research areas (athletic training, movement science, physical education, and sport management) in eleven kinesiology journals. The large number of citations in these journals (> 40,000) is primarily associated with the medical sciences, according to the Ulrichsweb Global Serials Directory and Web of Science. Knudson (2015b) studied the visibility of kinesiology research by documenting the top 40-ranked journals using the two main subject categories of Scopus (Social Sciences; Health and Medical Sciences), Journal Citation Reports (Education, scientific disciplines; Rehabilitation), and the “Physical Education & Sports Medicine” subcategory of Google Scholar Metrics (GSM). This identified 70 to 146 indexed journals in the field.

Despite considerable variation in indexing across databases and subsequent journal rankings, there were moderate-to-strong positive associations among three citation metrics (Google Scholar h-index, Journal Impact Factor, SCImago Journal Rank) for top-ranked kinesiology-related journals. This observation is a consistent result in the bibliometric/scientometric/informetric literature (Bollen et al. 2009; Franceschet 2010; Waris et al. 2017). However, considering the strength of associations, factor analyses consistently indicate that journal citation metrics may serve as proxy estimates of one of two scholarly influence factors: usage/impact or prestige (Bollen et al. 2009; Franceschet 2010; Leydesdorff 2009; Setti 2013; Knudson 2013; Knudson et al. 2024; Zhou et al. 2012). Some of the inconsistent results reported for journal metrics and their inappropriate use in ranking journals stem from inaccurate interpretations of the metrics and the constructs they approximate (Knudson 2024; Knudson et al. 2024). It is not uncommon for study authors to erroneously cite a usage metric, such as the Journal Impact Factor (JIF), as a measure of journal prestige.

Knudson (2022b) updated this work with a study of journal metrics, coverage, and subject area of 100 kinesiology-related journals in the two most prestigious, curated database services: Journal Citation Reports (Web of Science) and Scopus for 2020 (Baas et al. 2020; Pranckutė 2021). Results supported previous studies (Knudson 2015a, 2015b; Freeland et al. 2014) on biased coverage/indexing, and on citation totals for journals reported in the Journal Citation Reports. All these studies report differences in coverage and median citation rates between biomedical and sports medicine journals and the behavioral and social sciences, humanities, physical education, and sport management journals, using Journal Citation Reports subject areas. A recent study confirmed these results across researcher-assigned subject areas (Behavioral/Social Sciences, Education & Humanities, Biophysical & Sports Medicine, and Multi-Disciplinary) using several journal metrics for 114 kinesiology-related journals (Knudson et al. 2024).

Three studies of similar topics support the observation of inconsistent use of subject areas for kinesiology-related journals and research. A study of the use of kinesiology-related keywords as “labels” for research interests by scholars with a Google Scholar Profile reported widely varying numbers of researchers and total citations (Knudson 2022a). Knudson (2023) also reported a 77 to 108% difference in citation rates across 40 subject areas and three databases for biomechanics journals. Knudson (2024) studied four journal prestige metrics of 334 kinesiology-related journals from three major databases (Google Scholar Metrics, Journal Citation Reports, Scopus), finding that metrics were reported to unrealistic precision, with high skew and variation (CV = 66 to 166%) that advantage medical journals over behavioral/social science or specialized journals related to the field. These studies indicate that interdisciplinary fields like kinesiology have variable coverage, subject categories assigned, and citations that differ across different bibliometric database services.

A follow-up study was needed, using the most comprehensive free service (Google Scholar Metrics) and the curated database (Scopus). Specifically, data were needed on both kinesiology-related journals, as defined by Google Scholar’s subject area, and on journal titles that align with kinesiology and its subdisciplinary keywords. This study aimed to document five journal metrics from these two databases and compare them using a comprehensive set of kinesiology-related keywords that represent the field and its many subdisciplinary areas of research. Confirmation was sought for two hypotheses: Journal usage and prestige metrics would differ across common kinesiology subdisciplinary keywords that influence visibility, and the association between metrics would be associated with journal influence factor and size. Some journal metrics are biased in that their calculation benefits journals that publish more articles (Walters 2017).

Method

Searches of Google Scholar Metrics (GSM) were performed in March and April of 2024 (https://scholar.google.com/citations?view_op=metrics_intro&hl=en). Journal metrics from GSM currently return data as of July 2023. GSM is a tool added to the Google Scholar service that ranks journals across 8 categories and 289 subcategories using two Hirsch index metrics (h5-index and h5-median). Google Scholar is the world’s largest scholarly database service (Gusenbauer 2019), consistently finding the most research in all subject areas (Martin-Martin, Orduna-Malea, Thelwall, et al. 2018; Martin-Martin, Orduna-Malea, and Lopez-Cozar 2018; Martin-Martin et al. 2021). This greater coverage of journals, citable articles, subsequent citations, and several subcategories of peer-reviewed literature makes Google Scholar an attractive database service for scholars to both search for research and journal outlets.

Twenty-six searches for journals using kinesiology-related terms were performed, three using general field terms and twenty-three using subdisciplinary terms (Table 1). Given there is no universal organization defining the field of kinesiology, the 26 terms were chosen for their face validity and to expand the previous 20 terms studied (Knudson 2022a). In addition, the top 20 journals in the kinesiology-related GSM subcategory of “Physical Education and Sports Medicine” within the main category of “Health & Medical Sciences” were extracted for comparison. The GSM journal prestige and usage metrics extracted were the journal’s five-year (2018–2022) Google Scholar Hirsch Index (GSh5) and the five-year median citations for the Hirsch Index core (GSh5MeC), respectively. Journal h-indexes indicate the number of a journal’s articles that have received at least h citations in a specific time window. These highly cited journal documents are referred to as the h-core and are then used to calculate a median citation value for GSh5MeC. Note that Google Scholar uses a 5-year time window, so this study uses metric abbreviations with numbers to indicate the time interval used by that measure.

Table 1: Number of Journals Returned by Google Scholar Metrics for Various Kinesiology-Related Terms and Median Associated Citation Metrics

Search Term(s)	N	n	GSh5	GSh5MeC	SJRh3	SJRCPD3	SJRCD3
Human Movement	3	2	23	32	53	1.6	235
Kinesiology	4	3	29	38	20	2.6	246
Physical Activity	20	14	29	51	48	2.5	161
Athletic Training	5	2	30	44	71	2.1	286
Biomechanics	15	15	24	34	31	1.9	213
Exercise Physiology	14	9	22	18	29	1.5	134
Exercise Science	20	18	25	38	35	2.6	191
Fitness	5	4	21	38	27	1.6	139
Motor Behavior	4	1	21	32	77	1.5	210
Motor Development	2	2	26	36	48	1.8	178
Motor Learning	2	2	18	27	34	1.4	99
Physical Education	20	15	29	51	48	2.4	176
Sport Analytics	3	2	22	39	31	2.1	161
Sport Coaching	11	4	28	43	28	2.9	205
Sport History	4	4	9	13	16	0.5	69
Sport Management	13	7	29	53	41	2.5	128
Sport Medicine	20	19	44	61	77	3.6	376
Sport Nutrition	2	2	45	80	75	4.7	192
Sport Pedagogy	3	1	47	66	48	4.8	161
Sport Performance	4	3	31	48	41	2.9	254
Sport Philosophy	2	2	15	22	27	1.3	97
Sport Psychology	14	11	29	40	51	2.8	94
Sport Rehabilitation	9	7	24	35	30	2.0	269
Sport Science	20	17	35	53	56	4.0	269
Sport Sociology	2	2	31	45	66	3.2	140
Strength Conditioning	2	2	55	73	102	3.4	738
Grand Median			29	41	47	3.9	177

Note. Variables were journals returned by Google Scholar (N), number of returned journals analyzed following exclusion (n), five-year (2018–2022) Google Scholar Hirsh Index (GSh5), five-year median citations for h core (GSh5MeC), three-year (2019–2021) SCImago Journal Rank Hirsh Index (SJRh3), three-year SCImago Journal Rank mean citations per document (SJRCPD3), and the three-year SCImago Journal Rank reported journal citable documents (SJRCD3). Google Scholar Metrics reports only the top 20 journals in any search within their assigned 298 subcategories or to open searches like used in this study.

Three additional journal metrics for the world’s largest (Scopus) curated database (Baas et al. 2020) were extracted from SCImago Journal & Country Rank (https://www.scimagojr.com/) for these journals and the year 2022. Two to align with the GSM journal measures reported and the prestige and usage factor structure of journal citation metrics (Bollen et al. 2009; Franceschet 2010; Leydesdorff 2009; West and Rich 2012). The three-year (2019–2021) SCImago Journal Rank (SJRh3) and the three-year mean citations per journal document (SJRCPD3) are proxies for journal prestige and usage, respectively. SJRCPD3 is the mean number of citations per article from the h-core documents from SJRh3. Several journal citation metrics are biased by the journal size from the number of articles published, and references reported/citation density (Barendse 2007; Koelblinger et al. 2019; Roldan-Valadez et al. 2019; Waltman 2016; West et al. 2010), so the third metric documented the size of the journals with the three-year citable documents (SJRCD3). Databases limit the types of articles they index, and SCImago Journal Rank refers to these as citable documents. The number of references for each journal was not available from SCImago Journal Rank. Journal metrics used in this study are summarized in Table 2.

Table 2: Journal Metrics Examined for Kinesiology-Related Keyword Searches

Abbreviation	Metric	Description
GSh5	Google Scholar Hirsh Index	Measure of journal prestige
GSh5MeC	Google Scholar Median Citations for Hirsh Core	Measure of journal usage of highly cited journal articles
SJRh3	SCImago Journal Rank Hirsh Index	Measure of journal prestige but also strongly associated with usage
SJRCPD3	SCImago Journal Rank Mean Citations per Document	Measure of journal usage
SJRCD3	SCImago Journal Rank Citable Documents	Measure of journal size based on Scopus citable documents

Note. Numbers used in abbreviations indicate the time window used by the specific database to calculate each journal citation metric.

Journal data were manually entered in Microsoft Excel and verified. As in a previous study (Knudson 2024), the reliability of data extraction and manual recording is strong, with errors likely to be less than 1 percent. This was verified by the investigator, comparing journal metrics for all journals excluded from further analysis. Excluded journals were 49 duplicates across different kinesiology terms, six for not being a kinesiology-related journal (e.g., engineering proceedings, physics, magazine), and 33 for not having complete Scopus data for SJR3 metrics. There were no errors in duplicate-extracted metrics or in the second check for missing metrics for excluded journals. Interestingly, one of the oldest and most prestigious kinesiology journals (Research Quarterly for Exercise & Sport) did not appear in any of the searches, so its data were directly extracted from GSM and SCImago to support the interpretation of the results.

Descriptive data were calculated for variables from the 26 kinesiology-related terms and the GSM top 20 in “Physical Education & Sports Medicine.” Examination of the top 20 for the kinesiology-related subcategory is a common procedure in bibliometric studies. Bibliometric studies of journals focus on the top percentiles (e.g., 5 to 10%) given their influence from the strong positive skew of citations and uncited articles (Bornmann and Marx 2014, 2015). The associations among the journal metrics were examined for all journals with complete data from both databases, using either Pearson correlations or Kendall’s Tau (\(\tau\)). Associations for non-linear scatterplots with heteroscedasticity and outliers are best measured with Kendall’s Tau (Croux and Dehon 2010). The strength of significant associations was interpreted by using the coefficient of determination (r²) after converting tau to r (Walker 2003). Statistical analyses were performed in JMP Pro 15 (SAS Institute, Cary, NC), with the alpha level set at p < 0.01.

Results

Searches of GSM for the 26 kinesiology terms returned 223 journal and proceedings titles, with repetitions and exclusions, the final sample was 114 journals (Table 1). Inspection of the data in Table 1 for the keyword “kinesiology” indicates wide variation in the number of returned (N = 2 to 20) and analyzed journals (n = 1 to 19) across subdisciplines of the field. Median journal metrics across these 26 kinesiology terms also varied widely. Journals from sport medicine and sport nutrition had the highest prestige metrics (GSh5 and SJR3 between 44 and 77) and usage/citation metrics (GSh5MeC = 61 to 80 and SJRCPD3 = 3.6 to 4.7 citations per article). Terms from fields with lower prestige and usage metrics, such as Sport History and Sport Philosophy, also had the fewest articles published (ranging from 69 to 97) across the three SJR windows. This represents a 5-fold difference in median prestige and a 7-fold difference in median usage between sport history and sport medicine journals.

The top 20-ranked journals in the “Physical Education & Sports Medicine” subcategory of GSM are listed in Table 3. Note that these journals have nominally higher usage and prestige metrics than many of the median values in Table 1. Some of this difference is due to four of the top journals in this GSM subcategory being from the medical field, but were not linked to kinesiology terms in GSM searches.

Table 3: Top 20 Journals Classified by Google Scholar Metrics into the “Physical Education & Sports Medicine” Subcategory for 2023

Journal	GSh5	GSh5MeC	SJRh3	SJRCPD3	SJRCD3
British Journal of Sports Medicine	123	192	209	9.4	596
Sports Medicine	114	170	213	10.2	387
American Journal of Sports Medicine	86	115	252	5.7	1174
Int. J. of Behavioral Nutrition and Physical Activity	74	121	142	8.9	456
Journal of Strength & Conditioning Research	73	90	151	3.8	1308
Medicine & Science in Sports & Exercise	72	114	251	5.3	901
Knee Surg., Sports Traumatology, Arthroscopy	72	95	142	4.7	1409
Journal of Sports Sciences	67	91	153	4.2	951
Psychology of Sport and Exercise	64	85	106	5.2	491
Scand. Journal of Medicine & Science in Sports	63	81	132	4.5	653
Journal of Science and Medicine in Sport	60	88	117	4.0	623
Journal of Sport and Health Science	59	84	56	9.0	227
Arthroscopy	58	70	180	2.8	912
European Journal of Sport Science	57	78	73	4.3	462
Int. J. of Sports Physiology and Performance	56	81	85	3.7	658
Physical Therapy	54	89	169	3.5	576
Gait & Posture	54	75	164	3.0	1128
Orthopedic Journal of Sports Medicine	52	78	53	3.0	1391
Journal of Orthopedic & Sports Physical Therapy	52	77	136	3.9	264
Sport Management Review	51	73	70	6.5	163
Median	62	87	142	4.4	638

Note. Four medical journals in this table were not returned by Google Scholar Metrics for any of the 26 kinesiology terms searched in Table 1 (see body text for discussion). See Table 1 note for variable abbreviations.

Only the scatter plot of GSh5 versus GSh5MeC was linear, and Pearson correlation was used for analysis. The other associations in Table 4 were converted to r from Kendall’s \(\tau\). All variables were significantly and positively associated, given the p < .01 alpha standard. Correlations were larger between the prestige metrics GSh5 and SJRh3 (0.804) and usage metrics GSh5MeC and SJRCPD3 (0.875) across databases than between citable documents (SJRCD3) or between prestige and usage metrics. Journal size (SJRCD3) had small-to-moderate positive associations with all journal metrics, with slightly stronger associations with the prestige metrics.

Table 4: Correlation (r) Matrix of Journal Metrics with Most Calculated from Kendall’s Tau (n = 114)

	GSh5	GSh5MeC	SJRh3	SJRCPD3	SJRCD3
GSh5		0.981	0.804	0.870	0.732
GSh5MeC			0.771	0.875	0.661
SJRh3				0.650	0.630
SJRCPD3					0.353
SJRCD3

Note. All associations are statistically significant at or below an observed p = 0.0003. Kendall’s \(\tau\) values were converted to correlation coefficients r using the formula by Walker (2003).

Discussion

This study of kinesiology subdisciplinary keywords, searched in the world’s largest bibliometric database using the GSM tool, supported previous results and the hypotheses about differences in journal visibility and citation metrics across subdisciplinary areas of the field. Searches of GSM using kinesiology subdisciplinary terms will find a widely varying number of journals, from 2 to the GSM maximum of 20 per search. These results are consistent with previous research on inconsistent use of keywords and subject categories in bibliometric databases (Knudson 2015b, 2022a, 2023; Rovira et al. 2023). Kinesiology scholars should search multiple databases with multiple keywords, given variations in database indexing and inconsistent search engine results.

It is unclear how the GSM search algorithm returns records for keyword searches for journal titles, although search terms in the title appeared to drive results. Searching for “kinesiology” resulted in five journals with this term in the title, and searching for “human movement” returned three journals with these words in the title. A search for “physical activity” returned six of the 20 journals with these two terms, and all 20 with the term “physical”. Some examples of the unusual, returned journals and proceedings from the ones excluded from the study for not being primarily kinesiology-related were Physical Review: Physics Education Research (“Physical Activity” and “Physical Education” searches) and International Conference on Education, Sports, Arts and Management Engineering (“Sport Management” search). This study used the term “sport” consistently rather than “sports,” and found nearly identical search results for both terms in GSM searches. This contrasted with different numbers of returned records for worldwide researchers’ use of the terms sport and sports as research area “labels” in Google Scholar Profiles (Knudson 2022a).

The journal prestige metric GSh5 also varied widely (6 to 123) depending on the subdisciplinary term searched. The Scopus prestige metric (SJR3) for these journals also varied widely from 5 to 252. Journals with higher median prestige metrics were from keyword searches such as “sports medicine” (GSh5 = 44), whereas smaller (fewer published articles) and lower-citation-rate subdisciplines (e.g., “sport history” or “sport philosophy”) were lower (GSh5 = 9 and 15). This supported previous research of large subdisciplinary differences in citations and citation metrics across the diverse scope of kinesiology research (Knudson 2014, 2022b, 2024; Knudson et al. 2024). Similar bias, advantageous to biomedical journals over behavioral science and specialized journals, was evident in the usage metrics (Table 1). Median citation count per document over three years (SJRCPD3) was about 3 to 7 times larger in sport medicine/sport science journals (3.6 to 4.0 citations/document) than in sport history/sport philosophy journals (0.5 to 1.3 citations/document). Comparing the smallest and largest of the five metrics across all journals revealed a 21- to 510-fold range in these kinesiology-related journals, confirming that journal metrics should be compared only within kinesiology subdisciplinary areas.

Even in the most comprehensive database (Google Scholar), with less coverage bias from indexing more journals and disciplines, searches for GSM keywords in kinesiology yielded fewer published articles and lower citation rates in the humanities/social sciences than in larger, faster-moving biomedical science subdisciplines. This bias against the humanities and social/behavioral sciences from coverage and citation density is a common observation in most bibliometric databases (Dorta-González and Dorta-González 2013; Finardi 2014; Freeland et al. 2014; Leta and Rosa 2012; Knudson 2013, 2015a, 2015b; Seglen 1997; Waltman 2016; Waris et al. 2017).

All five journal metrics were positively associated with each other. This is a consistent finding in bibliometric research across many fields (Bollen et al. 2009; Franceschet 2010; Waris et al. 2017) and in kinesiology (Knudson 2013, 2015a, 2015b; Knudson et al. 2024). However, there were stronger associations between the prestige metrics (GSh5 & SJRh3 r² = 65%) and between the usage metrics (GSh5MeC & SJRCPD3 r² = 77%) than across these different journal constructs (r² = 20 to 42%). This supported previous results that identified journal metrics as proxies for two distinct usage and prestige constructs (Bollen et al. 2009; Franceschet 2010; Leydesdorff 2009; Knudson 2013; Knudson et al. 2024; Zhou et al. 2012). The moderate positive associations (12 to 54% shared variance) of these journal metrics with journal size (SJRCD3) also support the observations in this study of higher usage and prestige metrics for larger journals in biomedical sciences than for relatively smaller journals in the humanities/social sciences or with a specialized focus.

All the current results replicate and support recent recommendations of judicious use of multiple journal metrics (clearly aligned with journal quality constructs and noting qualifying factors) as supplementary information about kinesiology journals (Cardinal 2013; Knudson et al. 2024) and research (Knudson 2019). Judicious examination and interpretation of journal metrics within the context of a kinesiology subdiscipline can identify specific journals that might confirm or misinform likely scholar perceptions of prestige. For example, in 2022, the Journal of Applied Sport Psychology had higher-than-median prestige (SJR3 = 85) and usage (SJRCPD3 = 4.9), with a near-median number of published articles over three years (SJRCD3 = 92). A sport psychology scholar might also target the journal Mental Health and Physical Activity, which has a higher SJRCPD3 (5.1) and more published articles (SJRCD3 = 113), even though GSM does not link this to sport psychology. An example of the combined use of journal metrics for one of the most prestigious journals in kinesiology is the Research Quarterly for Exercise and Sport (Cardinal and Thomas 2005; Miranda and Mongeau 1991; Silverman et al. 2014), which, ironically, was not returned by GSM for the 26 searches of kinesiology keywords. Extracting GSM data for this journal indicates near median prestige and usage from GSM (GSh5 = 30, GSh5MeC = 45), but higher than median prestige (SJR3 = 100) and lower than median usage (SJRCPD3 = 2.8) from about a median number of published articles (SJRCD3 = 221). Review journals (e.g., Exercise and Sport Sciences Reviews) often have high prestige (SJR3 = 113) and usage (SJRCPD3 = 5.3) even if they publish relatively fewer articles (85) over three years. Readers interested in data on the rigor, usage, and prestige of kinesiology journals, and on the use of several metrics as proxies for these constructs, are referred to a recent study (Knudson et al. 2024). Kinesiology faculty should consider a variety of factors (e.g., journal mission, kind of scholarship, publication charges, society membership) beyond journal metrics in choosing publication outlets (Hunt and Huttenlock 2024).

Limitations

The present study, while providing confirmatory results from kinesiology and several scientific fields, should be interpreted with some limitations in mind. The kinesiology-related journals studied were limited to those returned by GSM for searches of 26 major keywords related to the field and its subdisciplines and the Research Quarterly for Exercise and Sport. There was also potential investigator subjectivity in journals excluded for not being kinesiology-related. There were small differences in journal metrics estimating prestige and usage between the two databases due to differences in target year, time windows, and algorithms, although the metrics are highly correlated within the prestige or usage construct. The accuracy of journal metrics is generally poorly understood and exhibits large positive skewness, outliers, variation, and biases due to database coverage (e.g., Hicks et al. 2015; Seglen 1992, 1997). Given that these limitations are present in all bibliometric studies, the results of the present study accurately confirm previous observations of differences in journal metrics across subdisciplines of kinesiology in the world’s largest free (Google Scholar) and curated (Scopus) databases.

Conclusion

The results of this study confirm recent studies indicating that bibliometrics used as proxies for the impact or prestige of kinesiology journals are likely biased by confounding factors such as journal size and subdisciplinary reporting and citation traditions (Knudson 2024; Knudson et al. 2024). These large differences (3 to 7 times) in median prestige and usage metrics between kinesiology subdisciplinary journal areas usually disadvantage humanities/behavioral science and specialized journals (e.g., lower visibility in page-rank search results and underestimated usage and prestige relative to larger subdisciplines) compared to biomedical science journals. These consistent results in kinesiology, recommendations from the bibliometric community (Declaration on Research Assessment 2013; Hicks et al. 2015; Wilsdon et al. 2015), and commentaries (Cardinal 2013; Olive et al. 2022; Phillips 2020; Sperka and Phillips 2022; Wagner 2016; Welk et al. 2014) all argue against the misuse of single citation metrics or their comparison across subdisciplines in evaluating kinesiology-related journals or the articles they publish. There is a clear consensus in the evidence that kinesiology research is too diverse and interdisciplinary to rely on a single database or journal metric as a proxy for research quality. Judicious use of multiple databases and journal metrics may provide only limited supplementary information on likely subdiscipline-specific peer perceptions for selecting potential publication outlets. Research published in journals should be evaluated on its own merits, not journal metrics. Multiple articles and author-level citation metrics also provide only proxy, supplementary information on peer perceptions of two constructs: likely usage and prestige (Knudson 2019).

License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

References

Aksnes, Dag W., and Gunnar Sivertsen. 2019. “A Criteria-Based Assessment of the Coverage of Scopus and Web of Science.” Journal of Data and Information Science 4 (1): 1–21. https://doi.org/10.2478/jdis-2019-0001.

Baas, Jeroen, Michiel Schotten, Andrew Plume, Grald Cote, and Reza Karimi. 2020. “Scopus as a Curated, High-Quality Bibliometric Source for Academic Research in Quantitative Science Studies.” Quantitative Science Studies 1 (1): 377–86. https://doi.org/10.1162/qss_a_00019.

Barendse, W. 2007. “The Strike Rate Index: A New Index for Journal Quality Based on Journal Size and the h-Index of Citations.” Biomedical Digital Libraries 4: Ar3. https://doi.org/10.1186/1742-5581-4-3.

Bar-Ilan, Judit. 2018. “Tale of Three Databases: The Implication of Coverage Demonstrated for a Sample Query.” Frontiers in Research Metrics and Analytics 3: 6. https://doi.org/10.3389/frma.2018.00006.

Bollen, Johan, Herbert Van de Sompel, Aric Hagberg, and Ryan Chute. 2009. “A Principal Component Analysis of 39 Scientific Impact Measures.” PLoS One 4 (6): e6022. https://doi.org/10.1371/journal.pone.0006022.

Bornmann, Lutz, and Werner Marx. 2014. “How to Evaluate Individual Researchers Working in the Natural and Life Sciences Meaningfully? A Proposal of Methods Based on Percentiles of Citations.” Scientometrics 98 (1): 487–509. https://doi.org/10.1007/s11192-013-1161-y.

Bornmann, Lutz, and Werner Marx. 2015. “Methods for the Generation of Normalized Citation Impact Scores in Bibliometrics: Which Method Best Reflects the Judgements of Experts?” Journal of Informetrics 9 (2): 408–18. https://doi.org/10.1016/j.joi.2015.01.006.

Cardinal, Bradley J. 2013. “Judicious Use of Journal Impact Factors and the Preservation of Our Fields of Study.” Journal of Physical Education, Recreation and Dance 84 (2): 7–9. https://doi.org/10.1080/07303084.2013.757172.

Cardinal, Bradley J., David P. Schary, and Moosong Kim. 2014. “In Search of Kinesiology’s Core Journals: An Elusive and Potentially Value-Laden Enterprise.” Comprehensive Psychology 3: 13. https://doi.org/10.2466/11.03.CP.3.13.

Cardinal, Bradley J., and Jerry R. Thomas. 2005. “The 75th Anniversary of the Research Quarterly for Exercise and Sport: An Analysis of Status and Contributions.” Research Quarterly for Exercise and Sport 76 (Suppl. 2): S122–34. https://doi.org/10.1080/02701367.2005.10599294.

Croux, Christophe, and Catherine Dehon. 2010. “Influence Functions of the Spearman and Kendall Correlation Measures.” Statistical Methods & Applications 19: 497–515. https://doi.org/10.1007/s10260-010-0142-z.

Declaration on Research Assessment. 2013. San Francisco Declaration on Research Assessment. https://sfdora.org/.

Delgado-Lopez-Cozar, Emilio, and Alvaro Cabezas-Clavijo. 2013. “Ranking Journals: Could Google Scholar Metrics Be an Alternative to Journal Citation Reports and Scimago Journal Rank?” Learned Publishing 26 (2): 101–14. https://doi.org/10.1087/20130205.

Dorta-González, Pablo, and María I. Dorta-González. 2013. “Comparing Journals from Different Fields of Science and Social Science Through a JCR Subject Categories Normalized Impact Factor.” Scientometrics 95 (2): 645–72. https://doi.org/10.1007/s11192-012-0929-9.

Finardi, Ugo. 2014. “On the Time Evolution of Received Citations in Different Scientific Fields: An Empirical Study.” Journal of Informetrics 8 (1): 13–24. https://doi.org/10.1016/j.joi.2013.10.003.

Franceschet, Massimo. 2010. “The Difference Between Popularity and Prestige in the Sciences and in the Social Sciences: A Bibliometric Analysis.” Journal of Informetrics 4: 55–63. https://doi.org/10.1016/j.joi.2009.08.001.

Freeland, Melissa, Linda Howes, Elizabeth Hamstra, and Sonya Bahnmaier. 2014. “Citation Analysis to Assist Selection in Kinesiology.” Serials Librarian 67: 307–41. https://doi.org/10.1080/0361526X.2014.939327.

Gusenbauer, Michael. 2019. “Google Scholar to Overshadow the All? Comparing the Sizes of 12 Academic Search Engines and Bibliographic Databases.” Scientometrics 118: 177–214. https://doi.org/10.1007/s11192-018-2958-5.

Gusenbauer, Michael, and Neal R. Haddaway. 2020. “Which Academic Search Systems Are Suitable for Systematic Reviews or Meta Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources.” Research Synthesis Methods 11 (2): 181–217. https://doi.org/10.1002/jrsm.1378.

Hicks, Diana, Paul Wouters, Ludo Waltman, Sarah de Rijcke, and Ismael Rafols. 2015. “The Leiden Manifesto for Research Metrics.” Nature 520: 429–31. https://doi.org/10.1038/520429a.

Hunt, Benjamin E., and Travis Huttenlock. 2024. “Publication Rates for Tenure-Track Faculty Members in Kinesiology and Exercise Science Departments at Liberal Arts Colleges and Universities.” International Journal of Kinesiology in Higher Education 8 (1): 15–23. https://doi.org/10.1080/24711616.2023.2176800.

Knudson, Duane. 2013. “Impact and Prestige of Kinesiology-Related Journals.” Comprehensive Psychology 2: 13. https://doi.org/10.2466/50.17.CP.2.13.

Knudson, Duane. 2014. “Citation Rates for Highly-Cited Papers from Different Sub-Disciplinary Areas Within Kinesiology.” Chronicle of Kinesiology in Higher Education 25 (2): 9–17. https://www.nakhe.org/_Library/journal_archives/25_2.pdf.

Knudson, Duane. 2015a. “Citation Rate of Highly-Cited Papers in 100 Kinesiology-Related Journals.” Measurement in Physical Education and Exercise Science 19: 44–50. https://doi.org/10.1080/1091367X.2014.988336.

Knudson, Duane. 2015b. “Influential Kinesiology Journals: The View from Outside of the Field.” The Physical Educator 72: 149–59.

Knudson, Duane. 2019. “Judicious Use of Bibliometrics to Supplement Peer Evaluations of Research in Kinesiology.” Kinesiology Review 8: 100–109. https://doi.org/10.1123/kr.2017-0046.

Knudson, Duane. 2022a. “Citations in Google Scholar Profiles by Kinesiology Subdiscipline.” Journal of Kinesiology and Wellness 11: 25–33. https://jkw.wskw.org/index.php/jkw/article/view/102.

Knudson, Duane. 2022b. “What Kinesiology Research Is Most Visible to the Academic World?” Quest 74 (3): 285–98. https://doi.org/10.1080/00336297.2022.2092880.

Knudson, Duane. 2023. “Publication Metrics and Subject Categories of Biomechanics Journals.” Journal of Information Science Theory and Practice 11 (4): 40–50. https://doi.org/10.1633/JISTaP.2023.11.4.3.

Knudson, Duane. 2024. “Are There Meaningful Prestige Estimates of Kinesiology-Related Journals?” Measurement in Physical Education and Exercise Science 28 (4): 316–37. https://doi.org/10.1080/1091367X.2024.2341849.

Knudson, Duane, Bradley Cardinal, and Patrice McCullagh. 2024. “Synthesis of Publication Metrics in Kinesiology-Related Journals: Proxies for Rigor, Usage, and Prestige.” Quest 76 (1): 93–112. https://doi.org/10.1080/00336297.2023.2237150.

Koelblinger, Doris, Georg Zimmermann, Sascha B. Weineck, and Tobias Kiesslich. 2019. “Size Matters! Association Between Journal Size and Longitudinal Variability of the Journal Impact Factor.” PLoS ONE 14 (11): e0225360. https://doi.org/10.1371/journal.pone.0225360.

Leta, Jacqueline, and Suely Rosa. 2012. “Trends in Scientific Publications in Physical Education: A Multifaceted Field?” Journal of Science Communication 11 (1): 1–9.

Leydesdorff, Loet. 2009. “How Are New Citation-Based Journal Indicators Adding to the Bibliometric Toolbox?” Journal of the American Society for Information Science and Technology 60: 1327–36.

Martin-Martin, Alberto, Enrique Orduna-Malea, and Emilio D. Lopez-Cozar. 2018. “Coverage of Highly-Cited Documents in Google Scholar, Web of Science, and Scopus: A Multidisciplinary Comparison.” Scientometrics 116 (3): 2175–88. https://doi.org/10.1007/s11192-018-2820-9.

Martin-Martin, Alberto, Enrique Orduna-Malea, Mike Thelwall, and Emilio D. Lopez-Cozar. 2018. “Google Scholar, Web of Science, and Scopus: A Systematic Comparison of Citations in 252 Subject Categories.” Journal of Informetrics 12 (4): 1160–77. https://doi.org/10.1016/j.joi.2018.09.002.

Martin-Martin, Alberto, Mike Thelwall, Enrique Orduna-Malea, and Emilio D. Lopez-Cozar. 2021. “Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations.” Scientometrics 126: 871–906. https://doi.org/10.1007/s11192-020-03690-4.

Miranda, Mark A., and Diane Mongeau. 1991. “An Evaluation of Journals in Physical Education, Athletics, and Sports.” Serials Librarian 21: 89–113.

Olive, Rebecca, Sara Townsend, and Murray G. Phillips. 2022. “‘Not Everything That Can Be Counted Counts, and Not Everything That Counts Can Be Counted’: Searching for the Value of Metrics and Altmetrics in Sociology of Sport Journals.” International Review for the Sociology of Sport, ahead of print. https://doi.org/10.1177/10126902221107467.

Phillips, Murray G. 2020. “Sizing up Sport History Journals: Metrics, Sport Humanities, and History.” International Journal of the History of Sport 37 (8): 692–704. https://doi.org/10.1080/09523367.2020.1796652.

Pranckutė, Raminta. 2021. “Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Todays’ Academic World.” Publications 9 (1): 12. https://doi.org/10.3390/publications9010012.

Ramos-Remus, Cesar, Maria Suarez-Almazor, Martha Dorgan, Augusto Gomez-Vargas, and Anthony S. Russell. 1994. “Performance of Online Biomedical Databases in Rheumatology.” Journal of Rheumatology 21 (10): 1912–21.

Roldan-Valadez, Ernesto, Sergio Y. Salazar-Ruiz, Ibarra-Contreas, and Camilo Rios. 2019. “Current Concepts on Bibliometrics: A Brief Review about Impact Factor, Eigenfactor Score, CiteScore, SCimago Journal Rank, Source-Normalized Impact Per Paper, H-Index, and Alternative Metrics.” Irish Journal of Medical Science 188: 939–51. https://doi.org/10.1007/s11845-018-1936-5.

Rovira, Cristofol, Lluis Codina, Frederic Guerrero-Sole, and Carlos Lopezosa. 2023. “Ranking by Relevance and Citation Counts, a Comparative Study: Google Scholar, Microsoft Academic, WoS and Scopus.” Future Internet 11: 202. https://doi.org/10.3390/fi11090202.

Seglen, Per O. 1992. “The Skewness of Science.” Journal of the American Society for Information Science 43 (9): 628–38. https://doi.org/10.1002/(SICI)1097-4571(199210)43:9<628::AID-ASI5>3.0.CO;2-0.

Seglen, Per O. 1997. “Why the Impact Factor of Journals Should Not Be Used for Evaluating Research.” British Medical Journal 314: 497–97. https://doi.org/10.1136/bmj.314.7079.497.

Setti, Gianluca. 2013. “Bibliometric Indicators: Why Do We Need More Than One?” IEEE Access 1: 232–46. https://doi.org/10.1109/ACCESS.2013.2261115.

Silverman, Stephen, Pamela H. Kulinna, and Sherri R. Phillips. 2014. “Physical Education Pedagogy Faculty Perceptions of Journal Quality.” Journal of Teaching in Physical Education 33: 134–54. https://doi.org/10.1123/jtpe.2013-0052.

Sperka, Leigh, and Murray G. Phillips. 2022. “Vistas of the Field: Examining Quality Indicators of Health and Physical Education Journals.” Sport Education and Society, ahead of print. https://doi.org/10.1080/13573322.2022.2096585.

Wagner, Ulrik. 2016. “The Publishing Game: The Dubious Mission of Evaluating Research and Measuring Performance in a Cross-Disciplinary Field.” Scandinavian Sport Studies Forum 7: 63–88.

Walker, David A. 2003. “JMASM9: Converting Kendall’s Tau for Correlation or Meta-Analytic Analyses.” Journal of Modern Applied Statistical Methods 2 (2): 525–30. https://doi.org/10.22237/jmasm/1067646360.

Walters, William H. 2017. “Do Subjective Journal Ratings Represent Whole Journals or Typical Articles? Unweighted or Weighted Citation Impact?” Journal of Informetrics 11 (3): 730–44. https://doi.org/10.1016/j.joi.2017.05.001.

Waltman, Ludo. 2016. “A Review of the Literature on Citation Impact Indicators.” Journal of Informatics 10 (2): 354–91. https://doi.org/10.1016/j.joi.2016.02.007.

Waris, Abdul, Shujaat Ahmad, Chaudhry Isam, Mohammed Abdel-Magid, and Abrar Hussain. 2017. “Comparison Among Journal Quality Indicators of Sports Science Journals.” Library Herald 55 (3): 338–51. https://doi.org/10.5958/0976-2469.2017.0032.X.

Welk, Gregory, Mark G. Fischman, Christy Greenleaf, et al. 2014. “Editorial Board Position Statement Regarding the Declaration on Research Assessment (DORA) Recommendations with Respect to Journal Impact Factors.” Research Quarterly for Exercise and Sport 85 (4): 429–30. https://doi.org/10.1080/02701367.2014.964104.

West, Jevin, Theodore Bergstrom, and Carl T. Bergstrom. 2010. “Big Macs and Eigenfactor Scores: Don’t Let Correlation Fool You.” Journal of the American Society for Information Science and Technology 61 (9): 1800–1807. https://doi.org/10.1002/asi.21374.

West, Richard E., and Peter J. Rich. 2012. “Rigor, Impact, and Prestige: A Proposed Framework for Evaluating Scholarly Publications.” Innovative Higher Education 37: 359–71. https://doi.org/10.1007/s10755-012-9214-3.

Wilsdon, James, Liz Allen, Eleonora Belfiore, et al. 2015. The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. Higher Education Funding Council for England.

Yamato, Tiê P., Mona Arora, Matthew L. Stevens, Mark R. Elkins, and Anne M. Moseley. 2018. “Quality, Language, Subdiscipline and Promotion Were Associated with Article Access on Physiotherapy Evidence Database (PEDro).” Physiotherapy 104: 122–28. https://doi.org/10.1016/j.physio.2017.08.003.

Zhou, Yue B., Linyuan Lu, and Meng Li. 2012. “Quantifying the Influence of Scientists and Their Publications: Distinguishing Between Prestige and Popularity.” New Journal of Physics 14: 033033. https://doi.org/10.1088/1367-2630/14/3/033033.