Corps de l’article

1. Introduction

1.1. The evolution of open access

The first open access (OA) peer-reviewed journal dates back to the early 1990s, but the widespread use of the concept as we understand it nowadays appeared in 2002, when the Open Society Institute launched the Budapest Open Access Initiative (BOAI). Since then, a series of declarations, such as the Bethesda Statement on Open Access Publishing and the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities, both in 2003, have been published to promote its adoption worldwide. A working definition satisfying the needs of the present study would be that of the BOAI, which defined academic OA as:

[…] free availability on the public internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. The only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited.

Chan, Cuplinskas, et al. 2002

Swan (2012: 11) affirmed that OA provides multiple and various kinds of benefits: improving the efficiency and efficacy of research, speeding up the dissemination of academic works, enabling interdisciplinarity and computation upon research literature (for example, mining for indexing or software analysis), increasing its visibility, usage and impact, and allowing professionals, practitioners, business communities, as well as the interested public, to benefit from it. In other words, providing universal access to information with no barriers of any kind is not only “an encouraging trend for free flow of information in the scientific worlds,” but also “ensures true democratization of knowledge” (Madalli 2015: 13) and has, therefore, an ethical dimension that must not be neglected.

With initiatives like Plan S (calling for publishing in fully OA journals only), launched by Science Europe in September 2018, more and more research funders and policymakers issue formal mandates demanding that publicly funded research be disseminated in OA. However, many have not yet adopted measures to guarantee that the OA mandate is fully obeyed and many scholars do not feel committed to making their work openly accessible. In this context, commercial publishers, academic social media, as well as predatory journals, have taken note and have started adapting their business models to take advantage of this new situation, contributing to increasing the panoply of OA modes (hybrid, delayed, green, black, etc.).[1]

It is worth noting that gold OA is very frequently used as a synonym and prototype for full OA in the literature. However, as we will presently see, diamond and hybrid OA are far more widespread within TIS. Whereas gold OA journals depart from a publishing model based on full OA, hybrid journals publish their contents under a subscription model, but offer authors the option of publishing their articles in OA by charging article processing charges (APCs). The business model of hybrid journals is generally not sustained by APCs but rather by the subscriptions paid by institutions, and the price of these subscriptions is maintained even if the journals include OA articles. In gold OA journals, APCs are meant to cover all the costs associated with editing and publishing the journal. Conversely, in diamond OA neither authors nor readers need to pay a fee. Most scholars do not differentiate gold from diamond OA, not only because the terminology has been subject to constant change, in parallel with the evolution of publishing models, but also because the diamond publishing model is almost non-existent in most disciplines. In this regard, references to “gold” OA that we find in the literature must be understood as a vague synonym of “full” OA.

Promoters of free access to science are working hard towards a goal that, for the first time in history, seems feasible in terms of both availability and cost. Although the transition to OA is slower than expected,[2] all studies show a steady increase in research outputs in OA since the late 1990s. Lewis (2012) suggested that gold OA might account for 90% of articles in 2020, or 2025 at latest, while Antelman (2017: 413) foresaw that there would still be 37% toll-access-only (TA) journals by 2025. According to Mikki (2017: 1533), the total share of freely available articles is around 70%; however, “the remaining 30% represent a crucial part of the scholarly output and belong to the most prestigious publishers.” Conversely, Piwowar, Priem, et al. (2018) estimate that only 28% of scholarly literature is OA, although the ratio is growing and the data corresponding to the latest year analysed (2015) recorded the highest percentage (45%). Wang, Cui, et al. (2018: 573) affirm that one quarter of the articles from Web of Science (WoS) is gold OA. However, Green (2019: 13) points out that so far this increase mainly derives from authors paying APCs (that is, they belong to the gold publishing model rather than the diamond model). It is worth mentioning that backfile conversion (the digitalizing and sharing on the internet of documents originally only available as hard copies) is also accelerating over time, which contributes to OA availability (Archambault, Amyot, et al. 2014: II).

The distribution of OA research is uneven amongst disciplines and countries. For instance, based on 2017 non-published data from Piwowar, Van Noorden (2019) reports that the world leaders in OA are Indonesia (>80%), Colombia (>60%), Bangladesh (>60%) and Brazil (>60%), all surpassing the 2017 world average of 41%. Regarding discipline-dependent differences, Archambault, Amyot, et al. (2014: IV) affirm that, although all the fields derive an OA citation advantage, “OA availability is greatest in general S&T (58% of the sampled papers) and lowest in general arts, humanities & social sciences (2.6%).” Mikki (2017: 1533) also reports a 72% ratio of OA for the natural sciences and technology, whereas the ratios for the social sciences and humanities are 58% and 55%, respectively. As far as translation and interpreting studies (TIS) is concerned, Franco Aixelá, Olalla-Soler, et al. (2021: 21) carried out a large-scale overview of the state of OA in TIS, including doctoral dissertations, books and book chapters, and concluded that “the evolution of open access has been remarkable in TIS, with a sustained growth from 18.0% in 1996-2000 to 46.6% in 2011-2015, almost reaching a balance with toll-access publication.” We also found that the share of OA journal articles reached 67.5% in the period 2011-2015, which places TIS data close to science and technology in Archambault, Amyot, et al. (2014: IV) and above Piwowar, Priem, et al.’s (2018) world average for the year 2015.

1.2. The open access citation advantage postulate

Lately, the pressure to publish OA has increased even more, since target journals should be both top-tier high-impact and OA, which notably narrows down the possibilities, especially in younger and smaller disciplines. To take TIS as an example, in 2019, out of the 146 active TIS journals detected (RETI 2013-2020), 91 (62.3%) were OA. However, there were only two (1.4%) OA journals included in the Journal Citation Reports (JCR), namely JoSTrans and Linguistica Antverpiensia (both in Q3). As for the other international prestigious index, the Scopus-based Scimago Journal Rank (SJR), only eleven out of the 146 (7.5%) in its first two quartiles were OA journals, namely JoSTrans and Translation & Interpreting in Q1, and Hermes, Hikma, Íkala, Linguistica Antverpiensia, Meta (delayed OA), MonTI, Mutatis Mutandis, New Voices in Translation Studies, Sendebar and Skase in Q2. In the last few years, some of the top-tier TIS subscription journals have been switching to a hybrid model, giving authors the choice to have their papers published OA if they pay article processing charges (APCs).

OA promoters have recently added a new argument to the debate that directly affects visibility and academic recognition: the open access citation advantage (OAA). The rationale is that increasing access to research will necessarily increase usage, thus boosting citations. In other words, OA research will have a greater citation impact compared to similar TA. If this was so, it would introduce a more pragmatic dimension for scholars going OA: “even if the true OAA turns out to be only 10-15%, this would still be a major incentive for scholars to choose an OA publishing option” (Wagner 2010: 4).

To this end, many scholars have embarked on a bibliometric crusade, taking a great variety of methodological approaches to adhere to, or reject, the OAA hypothesis. In the following we will summarize a representative sample of the main trends and findings.

In their literature reviews, Swan (2010), Hitchcock (2004/2013), Tennant, Waldner, et al. (2016) and Yan and Li (2018) found a general consensus favouring OAA, with the size of the effect depending on the object of research and the discipline. The SPARC Europe (2020) website lists 70 studies devoted to OAA up to 2015, and reports that 66% of them found an OAA, 24% found no OAA and 10% were inconclusive.

Lawrence (2001) discovered that the most highly cited articles in computer science were significantly more likely to be freely available online, with the mean number of citations to offline articles being 2.74, and the mean number of citations to online articles showing an increase of 157%. Antelman (2004: 376) showed that citation rates for OA publications exceed TA publications by 91%, 86%, 51%, and 45% in mathematics, political science, electrical and electronic engineering, and philosophy, respectively. In the same vein, Hajjem, Harnad, et al. (2005: 3) observed through a ten-year longitudinal study that OA articles are 36% to 172% more likely to be cited than their TA counterparts across a broad array of knowledge domains. Based on a total sample size of 209,000 papers, Archambault, Amyot, et al. (2014: II) also found that on average, the citation advantage of OA papers was 40.3% while the citation disadvantage was 27% for non-OA papers. Tang, Bever, et al. (2017) found an OAA in ecology journals, accumulating to approximately five more citations per article five years after publication. Ottaviani (2016: 1) affirmed that “an OAA as high as 19% exists, even when articles are embargoed during some or all of their prime citation years.” Fukuzawa (2017) also found that papers published in OA journals were cited in more countries and obtained more foreign citations, whereas Mikki (2017: 1529) concluded that there is a clear OAA in Norwegian research as a whole, since, on average, OA documents earned twice as many citations. Li, Wu, et al. (2018) also confirmed that OA increases journal citations, its magnitude depending on “characteristics of the journal such as the field, rank, and discipline of the journal, as well as the tendency of similar journals prone to open access.” Piwowar, Priem, et al. (2018: 1) corroborated that “OA articles receive 18% more citations than average, an effect driven primarily by green and hybrid OA.” Using a corpus of 47 Elsevier journals with different access types, Sotudeh (2019: 9) observed that TA “papers exhibited [a] significant disadvantage compared with their OA peers in each of the Green and APC-Green models.” Finally, Bautista-Puig, Lopez-Illescas, et al. (2020: 19) concluded that the journals in their study, which flipped from TA to OA, tended to have an OAA compared to non-gold OA journals.

However, Xu, Liu, et al. (2011, cited by Li, Wu, et al. 2018: 845) observed that, in contrast to other broad knowledge domains, OA papers in the humanities were at a disadvantage. Similarly, Sotudeh, Ghasempour, et al. (2015) report a lower OAA in the social sciences and humanities (3.14% higher than the TA set) as compared with the natural sciences (35.95%) in papers published in author-pay models of journals from Springer and Elsevier. Wray (2016) highlights that out of the 70 studies included in the above-mentioned list of SPARC Europe (2020), only 15 include data on the social sciences and humanities, from which only seven (47%) report an unequivocal OAA.

Some studies have also presented results against an OAA, such as Craig, Plume, et al. (2007: 247), who claim that no evidence has been found to support it. They also denounce that most studies fail “to determine accurately the date of earliest dissemination of each article, and then to impose a defined citation window, which must be used if citation analysis of open access status is to yield definitive results.” Davis and Walters (2011) also criticize the studies which indicate large citation advantages because these studies fail to control for confounding variables. A number of authors argue that although there is often a correlation between OA and more citations, the increase in citations is not necessarily caused by OA, that is, that there might be confounding factors not taken into account in the analysis. In their multidisciplinary study, Dorta-González, González-Betancor, et al. (2017) found no general OAA of gold OA, be it at the journal or at the article level. They contend that although in some scientific disciplines the average impact of an article is higher in the case of OA (36 categories), in most cases the opposite happens (173 categories) (Dorta-González, González-Betancor, et al. 2017: 9). Fukuzawa (2017: 1019) pointed out that what he calls international non-OA journals (that is, those journals not based in the country of the researchers publishing in them) attracted more attention from researchers working in countries that publish a significant number of papers than international OA journals did, which could have contributed to the higher citation counts for the former. In other words, OA might be an important condition for higher citation rates, but not the only one. Actually, Tahamtan, Safipour, et al. (2016), after carrying out a literary review, identified 28 different factors that influence the number of citations (related not only to the document itself but also to the journal and the author).

These are the main confounding factors mentioned in the literature—some of them partly overlapping—that might be influencing citation frequency:

  • intrinsically high quality or interest in the paper (Kurtz, Eichhorn, et al. 2005; Moed 2007; Craig, Plume, et al. 2007; Jabbour, Jabbour, et al. 2013, cited by Tahamtan, Safipour, et al. 2016: 1198; Sotudeh, Ghasempour, et al. 2015: 589; Ottaviani 2016: 10);

  • self-selection bias, whereby authors may choose OA for only their best quality publications (Kurtz, Eichhorn, et al. 2005);

  • the early viewership (Kurtz, Eichhorn, et al. 2005) or early exposure to draft versions of a manuscript (Moed 2007), according to which OA articles have more citations because they have been available for longer (specific to the green or black OA model);

  • conflation of pre-publication vs. published versions of articles (Ottaviani 2016);

  • absence of an appropriate control group of non-OA articles with which to compare citation figures (Ottaviani 2016);

  • availability at multiple access points or document versions (Xia, Myers, et al. 2011);

  • inclusion in repositories indexed in various alerting and search services, such as arXiv (Kurtz, Eichhorn, et al. 2005) or Academia.edu (Niyazov, Vogel, et al. 2016), which improve their discoverability;

  • language of the contribution (Franco Aixelá and Rovira-Esteva 2019);

  • publication’s global orientation (Fukuzawa 2017; Franco Aixelá and Rovira-Esteva 2019) or prestige (Gargouri, Hajjem, et al. 2010);

  • geographical proximity of researchers (Lee, Brownstein, et al. 2010)

However, Gargouri, Hajjem, et al. (2010: 8) confirmed that OAA “is a statistically significant, independent positive increase in citations, even when independent contributions of many other salient variables,” such as article age, journal impact factor, number of authors, number of pages or references cited, field, article type or country, were controlled. They affirm that OAA is real, independent and causal, and its size is necessarily correlated with quality, just as citations themselves are (Gargouri, Hajjem, et al. 2010: 1). In sum,

OA itself will not make an unusable (hence uncitable) paper more used and cited. However, wherever there are subscription-based constraints on accessibility, providing OA will increase the usage and citation of the more usable and citable papers, probably in proportion to their importance and quality, hence citability.

Gargouri, Hajjem, et al. 2010: 9

McCabe and Snyder (2014: 1297), after removing the confounding effect of author selection, reported a small but still meaningful 8% OAA. Niyazov, Vogel, et al. (2016) also estimated a statistically significant citation advantage of black OA papers after controlling for the selection bias.

OAA varies between disciplines or research fields (Antelman 2004; Moed 2007; Norris 2008; Archambault, Amyot, et al. 2014; Sotudeh, Ghasempour, et al. 2015). For Yan and Li (2018: 845) “disciplinary patterns of OA advantages are reflections of how OA is interwoven into the academic workflows” in each field. Niyazov, Vogel, et al. (2016), affirm that “[s]ince the citation frequency differs across disciplines, a citation advantage estimate that doesn’t control for academic discipline might over- or underestimate the true advantage.” Tahamtan, Safipour, et al. (2016: 1215) conclude that most factors influencing the frequency of citations are related to the topic of a paper and the field’s sub-disciplines. In the same vein, Li, Wu, et al. (2018: 3) also found that “knowledge domain is arguably the most important factor in determining the outcome.” However, none of the 33 reference papers quantifying the OAA listed by Tennant, Waldner, et al. (2016: 7-8), for example, focuses specifically on data from the humanities. All in all, according to our literature review, most studies on OAA do not include data on language-related research and even less so on TIS.

Davis and Walters (2011: 214) call for more empirical research dealing with the impact of OA on the use of scholarly literature. As Sotudeh, Ghasempour, et al. (2015: 592) conclude, it is “necessary to carry out further researches [sic] to analyse the differences in citation performances of OA and TA papers across specific subject areas.” This is precisely what we intend to do in the present study—to empirically research the impact of OA on citation behaviour in TIS.

To the best of our knowledge, the first general critical analysis of the presence and evolution of OA in TIS is provided in Franco Aixelá, Olalla-Soler, et al. (2021). For reasons of space, this publication did not address the specific topic of OAA in TIS, which is the main aim of the present article. In this regard, we will try to answer the following general research question: Is there an OAA in TIS in terms of impact or, in other words, is OA TIS research more highly cited than its TA counterparts?

Since this question is very general and includes many issues that should be singled out to be properly analysed, the following five more specific research questions have also been drafted:

  1. Has there been an evolution of OAA over time?

  2. Does OA decrease citation latency?

  3. Does OA increase the probabilities of being cited with the passing of time?

  4. Are there differences in OAA in the cited half-life?

  5. Is there a relationship between document access type and citing/cited documents?

Section Two will be devoted to explaining the data and methodological approach. Results, together with the discussion, will be presented in Section Three. This article ends with some concluding remarks, summarising the main findings, the limitations of this study and some ideas for future research.

2. Data and methods

2.1. Data collection and preparation

A copy of BITRA (Franco Aixelá 2001-2021) was exported to a txt file on May 29, 2020. At that time, BITRA contained 81,003 entries. The txt file was processed in Excel to create a derived database that could be used for statistical analysis in SPSS (v. 25). In Excel, data were manually checked to identify errors and inconsistencies.

We used BITRA citations as a proxy in TS, since BITRA had mined citation data from over 10% of its documents when this study was carried out, and over 100,000 citations had been assigned to the cited documents. Thus, the results obtained in our analyses should be interpreted with caution. Despite this limitation, we chose to use BITRA as a source of citation data for the following reasons: a) BITRA excludes self-citations, which have been found to distort results (Cooke and Donaldson 2014); b) BITRA is a TIS-specific database and it does not index or extract citations from or to documents outside the discipline. This is essential when carrying out citation and latency analyses, since field-normalised data are required to avoid distortions in the results (De Bellis 2009). c) BITRA only indexes documents that have been subject to academic filters (peer or publisher review). Other sources of citation data, such as Scopus, WoS or Google Scholar (GS), do not meet some of these criteria and their use for citation and latency analysis purposes in TIS would bias the results obtained. Moreover, extracting TIS citation data from Scopus, WoS or Google to at least partly triangulate with the citation data extracted from BITRA would need to be done manually, since these databases do not contain a specific category for TIS. Such a procedure would be practically impossible, as it would involve mining individual citation scores for thousands of documents.

In our article, impact is measured in citation counts as an indicator of the attention a document has received since its publication and not as an indicator of its quality. From its very inception, impact has been a moot point (for example, Bornmann and Daniel 2008), and it is important to be aware that documents are cited for numerous reasons which are not always related to their quality (Garfield 1965). In other words, impact, as measured by citation counts, is influenced by many variables. In our study, we have controlled for the following variables so as to establish as direct a relationship as possible between access type and impact:

  • The “early view effect.” BITRA only mines citations that occur after the document has been officially published.

  • Document format (see below).

  • Elapsed time between citation and publication (see below).

  • Field-normalised citations. BITRA only mines citations from and to TIS documents.

  • Self-citations. BITRA excludes authorial self-citations.

In order to work with a homogeneous sample, a period of analysis ranging from 1996 to 2015 (after which a lesser portion of the citations has been detected as of yet) was agreed upon. We established 1996 as a starting year (as in Archambault, Amyot, et al. 2014) because OA was virtually unheard of before 1993. Given that the 1996-2015 twenty-year period (containing 52,914 documents) is rather broad, it was divided in four five-year periods to better observe any possible changes or trends: 1996-2000 (11,884 documents), 2001-2005 (13,690 documents), 2006-2010 (14,642 documents), and 2011-2015 (12,698 documents). Since our analyses here are based on citation counts and the elapsed time between the year of citation and the year of publication of a document, only documents with at least one citation were used.[3] Hence, calculations were performed using 20,368 documents, that is, 38.5% of the total documents for this period (1996-2000: 4,956 documents; 2001-2005: 5,603 documents; 2006-2010: 5,627 documents; 2011-2015: 4,182 documents).

The two main indicators in this study were citation counts (65,894 citations registered for the 1996-2015 period) and the elapsed time between the citation and publication years. These two indicators cannot be employed as raw data when conducting bibliometric analyses. In terms of impact, a document that has been published for 10 years and has accrued 10 citations cannot be equated to a document with 10 citations but published five years ago, since the latter will have received twice as many citations per year.

Rovira-Esteva, Franco Aixelá, et al. (2019) determined that some publication formats in TIS tend to accrue more citations than others. Hence, we normalised our citation data, that is, we applied a corrective factor to adjust for differences in the density of publications (De Bellis 2009: 116), in terms of access type (OA or TA), time (publication year) and document type (article, book, book chapter, journal special issue or Ph.D. thesis). We calculated the mean number of citations accrued by all documents published in a specific year, publication format and access type (expected citations) and we divided the observed citations of each document with the same characteristics by its expected citations. For the analyses based on the elapsed time between citation and publication, we applied to each document Moed, Van Leeuwen, et al.’s (1999) correction by taking into account its publication year, access type and format.

2.2. Data analysis

The analyses performed in this article regarding the performance of OA documents vs. TA documents are grouped into three sections: citation patterns (2.2.1.), citation latency and cited half-life (2.2.2.), and specific citation patterns of TIS journals and articles (2.2.3.).

2.2.1. Citation patterns

In this section, we first compared the median number of citations accrued by OA and TA documents both for the entire period of the study (1996-2015) and by five-year periods (1996-2000, 2001-2005, 2006-2010, and 2011-2015). For this analysis, and for all the others in which we compared the median of citations accrued by OA vs. TA documents, we computed central-tendency statistics and measures of dispersion for OA and TA separately. Given that bibliometric distributions of citation data are highly skewed and non-normal (De Bellis 2009), the median was used as the main central-tendency statistic, and the median absolute deviation (MAD) as a measure of dispersion, since it is more robust. The 5% trimmed mean (that is, a mean that excludes outliers, made up by the lowest 5% and the highest 5% of the data) and the standard deviation (SD) were also computed. For these three analyses, the Mann-Whitney U test was applied to compare OA vs. TA, and r is the reported effect size. Given that our research questions aim to identify a possible citation advantage for OA vs. TA, the tests carried out here were one-sided tests.

We finally compared the access type (OA or TA) of the documents citing OA or TA documents. In order to obtain an overview, this was performed only for the entire period. Significance was tested with the chi-squared test and the Fisher’s exact test, given that both variables are dichotomous. Phi was used as an effect size.

2.2.2. Citation latency and cited half-life

In this section, we first compared the citation latency, that is, the time between a document being published and first being cited, of OA and TA publications from 1996 to 2015. The analysis was performed for the entire period and for five-year periods (1996-2000, 2001-2005, 2006-2010, and 2011-2015).

Our second analysis was aimed at comparing the cited half-life, that is, “the time during which half the total use of a given literature has been made” (De Bellis 2009: 114), for OA and TA documents. For obvious reasons, this was performed only for the entire period.

2.2.3. Specific citation patterns of TIS journals and articles

In the first analysis, we compared the normalised number of citations accrued by OA vs. TA TIS-specific journals for the period 1996-2015. To normalise our citation data for each journal, we divided the number of citations accrued by a journal from 1996 (or its foundation year when later than 1996) until 2015 by the number of articles published by that journal during the same period. The resulting figure is a normalised metric for both the journal’s productivity and its years of activity. For the period 1996-2015, BITRA included articles published by 2,693 different journals. We needed to establish a comparable subset of TIS-specific journals so as to reduce the possible bias of including journals from many different disciplines, most of them with very few TIS articles. To identify TIS-specific journals that is, disciplinary journals, we used two sources: RETI and BITRA. The journals listed in both as TIS-specific (that is, with at least 50% of their articles devoted to TIS) were included in our analysis. In all, 146 journals were selected.[4]

Our second analysis aimed to establish the ratio of cited articles published by the ten most cited OA and TA journals identified in the previous analysis through the normalised citation value.

In our third analysis we compared the median number of citations accrued by OA vs. TA articles published by disciplinary journals. This was performed for: a) the entire period of study (1996-2015); b) five-year periods (1996-2000, 2001-2005, 2006-2010, and 2011-2015); and c) disciplinary journals founded from 1996 onwards.

In our last analysis, we compared the access type (OA or TA) of the citers to check whether there was a relationship between the access type of the citer and the cited documents. In order to obtain an overview, this was performed only for the entire period.

For all tests, both significant and non-significant results are reported, and they are also interpreted based on the magnitude of the difference (that is, the effect size), that is (Ellis 2010): 0.1 = small; 0.3 = medium; 0.5 = large.[5] This rule is applied for both r and phi.

3. Results and discussion

The results obtained from the analysis are presented in three subsections: citation patterns of OA and TA documents (3.1.), citation latency and cited half-life of OA and TA documents (3.2.), and specific citation patterns of OA and TA TIS journals and articles (3.3.). It is worthwhile to mention that the following results correspond to those documents for which citations have been detected in BITRA. As in most fields of knowledge, not receiving any citation is also the norm in TIS (61.8% of cases, according to Rovira-Esteva, Franco Aixelá, et al. 2019).

3.1. Citation patterns of OA and TA documents

For the entire period of the study (1996-2015), OA documents tend to receive slightly more citations than TA documents (Table 1), although the difference is rather small (U = 38,172,284.5; p < 0.001; r = 0.121). Concluding that OA poses a citation advantage over TA would be too strong and we can only hint at some tendency toward this advantage.

Table 1

Citations accrued by OA documents vs. TA documents (1996-2015)

Citations accrued by OA documents vs. TA documents (1996-2015)

-> Voir la liste des tableaux

From a diachronic perspective (Table 2), OA documents published in the periods 1996-2000 (U = 1,585,817.5; p < 0.001; r = 0.135), 2001-2005 (U = 3,150,468.5; p < 0.001; r = 0.119), and 2011-2015 (U = 1,949,604.5; p < 0.001; r = 0.065) accrue slightly more citations than TA documents.[6] This means that the differences are small, and it is not possible to conclude that a diachronic OAA exists. The number of citations accrued by OA documents slightly increases as time advances, but so do citations to TA documents.

Table 2

Citations accrued by OA documents vs. TA documents by period

Citations accrued by OA documents vs. TA documents by period

-> Voir la liste des tableaux

Almost 64% of OA documents are cited by other OA documents (Table 3). In the case of TA documents, there is a balance between OA and TA citing documents (47% vs. 53%). There is a significant, although small, relationship between the access type of cited and citing documents (χ2 [1] = 566,616; p < 0.001; p-value of the Fisher’s exact test < 0.001; φ = 0.094).

Table 3

Access type of citing documents based on access type of cited documents

Access type of citing documents based on access type of cited documents

-> Voir la liste des tableaux

It is interesting to observe that each access type seems to reinforce its own access type, most clearly in the case of OA. To our knowledge, there are no large-scale studies on the type of access of cited and citing documents as a whole, so our results could inspire future studies analysing this relationship in other disciplines.

3.2. Citation latency and cited half-life of OA and TA documents

For the twenty years under study (1996-2015), 50% of OA documents receive their first citation (citation latency) in the first 5.10 years after publication (Table 4), whereas TA documents receive their first citation slightly faster (4.05 years after publication). Hence, according to our data, OA does not decrease citation latency over TA (U = 41,142,588; p < 0.001), although the difference is smaller than what we have termed as a small effect (0.1) (r = 0.05).

Table 4

OA vs. TA citation latency (1996-2015)

OA vs. TA citation latency (1996-2015)

-> Voir la liste des tableaux

From a diachronic perspective (Table 5), the citation latency of OA documents is slightly higher than that of TA documents in 1996-2000 (U = 1,553,591; p = 0.999; r = 0.157), 2001-2005 (U = 2,854,371; p = 0.999; r = 0.108), and 2006-2010 (U = 3,075,847; p = 0.999; r = 0.142), but once more the differences are small. Both in OA and TA documents, the citation latency decreases as time advances, and in the period 2011-2015 the median citation latency is very similar for OA and TA documents (U = 1,758,559; p = 0.999; r = 0.158). Therefore, currently publishing in OA or TA does not seem to have any bearing on how long it takes to receive the first citation.

Table 5

OA vs. TA citation latency by period

OA vs. TA citation latency by period

-> Voir la liste des tableaux

The cited half-life of OA and TA documents is also similar (7.10 vs. 7.35). The likelihood of being cited as time advances (Figure 1) is almost identical for OA and TA documents. The curves of both access types overlap until the 50% likelihood of being cited is reached seven years after publication. Therefore, OA does not seem to extend a document’s impact further into the long tail of the citation curve.

Figure 1

Likelihood of being cited as time advances for OA and TA documents

Likelihood of being cited as time advances for OA and TA documents

-> Voir la liste des figures

Emery, Lucraft, et al. (2017: 20) measured the performance of Springer Nature OA books compared with TA books only to find that the former had an increased performance, based on downloads (seven times more), citations (50% higher) and mentions (ten times more). Although they found variations by subject area, OA books always outperformed TA books. For reasons of space, in this paper we have mainly analysed the performance of TIS documents as a whole and specifically for journals, leaving for a future study the possible influence of other confounders such as document type or language.

3.3. Specific citation patterns of OA and TA journals and articles

Table 6 shows the total number of citations detected in BITRA for each journal (1996-2015), the share they represent in relation to that access type, as well as the normalised number of citations per year in descending order. Both in TA and OA journals, citations tend to accumulate in few journals. In OA journals, citations tend to concentrate in what, according to RETI’s (2013-2020) data, are top-tier journals. In the case of TA journals, all those included in this table can be considered the most prestigious within TIS. Only one OA journal surpasses 10% of citations accrued by that access type (Meta[7]), while this ratio is reached in three TA journals.

Table 6

Journal citations (normalised per year)

Journal citations (normalised per year)

Note. To ensure a comparable set of journals, only journals with at least fifty citations and at least fifty articles recorded in BITRA are included. Normalised citations have been calculated dividing the number of citations accrued in the period 1996-2015 by the number of articles published in the same period. If a journal was founded after 1996, the year of foundation was set as the initial year.[8] According to Scopus’ data, during the period 1996-2015, only ten articles were published OA by TA TIS journals.

-> Voir la liste des tableaux

Eleven OA journals receive at least one normalised citation per year, while this is the case in eight TA journals. One journal (Meta) accrues more than 40% of all citations to OA journals, thus its two-year embargo does not seem to have any negative effect on its impact. Despite having a very small share of the total number of citations to OA journals, Hermes performs very well in terms of normalised citations per year, equalling that of Meta. Amongst TA journals, The Translator and Target take almost one quarter of the cake each, with about six normalised citations per year, as does Interpreting, with a much smaller share for the total citations to TA journals.

It is worth highlighting that all the TA journals in Table 6 follow a hybrid business model. In the case of OA journals, except for Meta and TTR (with delayed access), all are diamond OA. In our discipline, Springer, John Benjamins, and Taylor & Francis/Routledge can be considered the big three publishers, since they concentrate most of the high-ranking subscription and hybrid journals in TS (ten out of the eleven in Table 6).

Many TIS diamond journals are younger and smaller than TA journals, and this may have an adverse effect on citation rates. As Archambault, Amyot, et al. (2014: V) point out, “authors frequently prefer reading and citing established journals, and it is therefore a challenge to start a journal from scratch, and to have authors submit high-quality articles. It takes time to build a reputation and to attract established authors.” Moreover, quite a few of the diamond TIS OA journals are from regions of the world not historically favoured by WoS (Brazil, Italy or Spain), are published in languages other than English, or might be considered less prestigious because they have not had time to become established or to accumulate citations.

Björk and Solomon (2012) also found that “OA journals that fund publishing with article processing charges (APCs) are on average cited more than other OA journals.” However, in TIS this only applies to the hybrid business model, since out of the existing two gold or fully APC-funded OA journals (RETI 2013-2020), only one citation has been detected in BITRA so far. Therefore, in our case, gold OA journals are cited much less than both diamond and subscription journals, likely due to the wide offer and relatively good indexing, in the former, and hybrid journals occupying top-tier positions, in the latter. Delayed OA is a small, special group occupying a middle position between both groups as regards impact.

As in Björk and Solomon’s study (2012), “OA journals are much more numerous in categories that have low overall impact factors which may explain some of the difference in average impact between OA and subscription journals.” However, our results seem to indicate that OA journals included in prestigious indexes are approaching the same scientific impact or even outperforming some subscription and hybrid ones.

So, what we might be witnessing here is the so-called Matthew or snow-ball effect. McCabe and Snyder (2014: 1295-6) also provide “evidence of a ‘superstar’ effect of open access, that is, open access benefits higher quality journals more than lower-quality.”

Our results suggest that a journal’s history and prestige seem to play a key role within TIS research practices. Since subscription and hybrid journals are more established within the discipline and readily available through many university libraries, researchers will tend to cite such journals in their fields, regardless of their business models. Therefore, OA might be a beneficial but not sufficient condition for a journal’s visibility and appeal. Visibility has also to do with outreach, having a wide readership and, more importantly, the position in international prestigious rankings. This is in line with Dorta-González, González-Betancor, et al’s (2017: 9) findings, who also concluded that OA per se does not guarantee higher visibility in relation to the subscription or hybrid model.

Migheli and Ramello (2014: 1253) denounce that academia is experiencing an OA paradox, that is, “scholars generally express strong support for OA, but do not subsequently transfer this enthusiasm to their publication choices,” which might have its “roots in the perceived low quality of OA journals.”

Moreover, TIS scholars often work under the pressure of publishing in journals indexed in the first two quartiles of SJR (Scopus) or JCR (WoS). As of now, the offer of OA TIS top-tier journals able to absorb this demand is very scarce, and TIS scholars seem to prefer publishing in better indexed subscription or delayed journals, rather than publishing in lower ranking diamond OA journals. Because TA journals make up the foundation of scholarly communication in TIS, it is very hard for OA journals to challenge their position. It is our contention that, as more diamond or gold OA journals establish themselves in the top tiers, authors will probably see them as a better alternative. One of the problems here is that, despite the fact that around 70% of TIS disciplinary journals have been diamond OA from their inception, most of them survive thanks to philanthropic support, either from professional associations or universities. Their scarcity of resources, both in terms of money and professionals devoted to run them, hinders their inclusion in most prestigious indexes, but also their ability to issue more than one or two volumes per year and, thus, to administer and publish submitted papers with fluency.

Also, scholars in the humanities and the social sciences often have meagre funding to invest money in APCs to get their papers published in OA, which is not felt to be worthwhile from the point of view of citations (Wray 2016).

The ratio of cited articles (Table 7) is slightly lower in the case of the 10 most cited journals publishing in OA than that of the 10 most cited ones in TA (49.0% vs. 58.6%; χ2 with continuity correction [1] = 49.659; p < 0.001; p-value of the Fisher’s exact test < 0.001; φ = 0.096), although the difference is small.

Table 7

Ratio of cited articles published during the period 1996-2015 by the ten OA and TA journals with the highest value of normalised citations

Ratio of cited articles published during the period 1996-2015 by the ten OA and TA journals with the highest value of normalised citations

Table 7 (suite)

Ratio of cited articles published during the period 1996-2015 by the ten OA and TA journals with the highest value of normalised citations

Note. As the value of normalised citations (Table 6) is the same for Meta and Hermes (2.7) and for Quaderns and Tradumàtica (1.0), Table 7 includes twelve OA journals that correspond to the 10 journals with the highest value of normalised citations.

-> Voir la liste des tableaux

Generally speaking, TA journals outperform OA journals in terms of citation counts and percentage of cited articles. The percentage of cited articles in most cited OA journals (as shown in Table 7) is above 35%, which is only the case in eight journals in the TA group. However, there are no OA journals above 60% of cited articles, whereas there are four TA that satisfy this condition. Therefore, publishing in OA journals per se does not seem to compromise or decrease the possibilities of being cited.

For the entire period under study (1996-2015), OA articles published by TIS disciplinary journals (Table 8) accrue almost as many citations as TA articles and therefore there is no OAA over TA ones (U = 2134492; p = 0.989; r = 0.041).

Table 8

Citations accrued by OA articles vs. TA articles in TIS disciplinary journals (1996-2015)

Citations accrued by OA articles vs. TA articles in TIS disciplinary journals (1996-2015)

-> Voir la liste des tableaux

Diachronically (Table 9), TA articles accrue slightly more citations than OA articles for all periods except for 2006-2010 and thus no OAA over TA ones is detectable as time advances (1996-2000: U = 73,862; p = 0.816; r = 0.037; 2001-2005: U = 112,468; p = 0.999; r = 0.215; 2006-2010: U = 164,475; p = 0.996; r = 0.092; 2011-2015: U = 152,557; p = 0.875; r = 0.040).

Table 9

Citations accrued by OA articles vs. TA ones published by TIS disciplinary journals by period

Citations accrued by OA articles vs. TA ones published by TIS disciplinary journals by period

-> Voir la liste des tableaux

Despite the fact that the volume of TIS OA articles has increased over time up to 2015, no increasing tendency to cite OA documents has been detected. It is worth noting that the median for citations of both access types became precisely equal in the period 2006-2010 when the number of OA articles doubled that of TA. However, there is no generalizable OAA at the article level.

In the case of journals published after 1996, when some journals started their shift from paper to electronic format (Table 10), we detect a similar trend: there is no OAA for OA articles over TA articles (U = 396,205; p = 0.999; r = 0.108), but again the difference is very small.

Table 10

Citations accrued by OA articles vs. TA articles published by TIS disciplinary journals since 1996

Citations accrued by OA articles vs. TA articles published by TIS disciplinary journals since 1996

-> Voir la liste des tableaux

Although some journals started going openly available online as of 1996, business models have continuously changed over the last decade. The shift from paper to digitalisation has implied either the shift from a subscription model to a diamond open access model (mainly in the case of journals run by universities and associations) or to a hybrid business model (commercial publishers). Some journals have simply kept the subscription business model providing paper and/or online access.

In Table 11, the same tendency identified in Table 3 is detected: OA articles are more frequently cited by OA documents (63%) than by TA documents (37%). In the case of TA articles, there is a balance between OA and TA citing documents (48% vs. 52%). Again, there is a significant although small relationship between the access type of cited articles and citing documents (χ2 [1] = 527,927; p < 0.001; p-value of the Fisher’s exact test < 0.001; φ = 0.153).

Table 11

Access type of cited articles published by TIS disciplinary journals and citing documents

Access type of cited articles published by TIS disciplinary journals and citing documents

-> Voir la liste des tableaux

4. Conclusions

4.1. Major findings

This is the first study carried out in TIS comparing the performance of documents in terms of accrued citations according to their access type to find out whether OA TIS research is more highly cited than its TA counterpart (OAA). The analysis is based on a total sample of over 20,000 TIS-related documents extracted from BITRA covering a time span of 20 years.

Although OA publications tend to receive slightly more citations than TA documents in our period of study (1996-2015), this difference is too small to confirm or reject the OAA hypothesis in TIS. Therefore, the first general conclusion is that the OAA detected in some disciplines cannot be clearly ascertained in TIS.

Apart from this general research question, we also formulated five specific ones. Firstly, we wanted to analyse if there has been an evolution of OAA over time. The total number of citations accrued by OA documents has slightly increased as time has advanced, but so have citations to TA documents. Therefore, our results do not show an OAA over time.

Secondly, we wondered if OA decreased citation latency. According to our data, OA does not decrease the time elapsed from publication to first citation over TA, although the difference is very small, since OA and TA documents receive their first citation in the first 5.10 and 4.05 years after publication, respectively. Both in OA and TA documents, the citation latency decreases over time, and in the last period (2011-2015) the median citation latency is almost the same for OA documents as it is for TA documents (3.32 vs. 3.30).

Thirdly, we searched for differences in the cited half-life and tried to discover if OA increased the likelihood of being cited as years go by, but the cited half-life of OA and TA documents is very similar (7.10 vs. 7.35), as well as the probabilities of being cited as years go by.

Fourthly, we looked for a relationship between document access type and citing and cited documents. It is worth noting that almost 64% of OA cited documents are cited by OA documents, whereas in the case of TA documents, there is a balance between OA and TA citing documents (47% vs. 53%). There is a significant, although small, relationship between access type of cited and citing documents. When it comes to articles, almost the same percentages are found.

Last, we searched for differences contingent to the journal’s access mode. Although the difference is small, TA articles published by TIS disciplinary journals from 1996 to 2015 accrue more citations than OA articles and therefore no OAA has been found. The same tendency was observed for disciplinary journals published after 1996. Citations in TA journals tend to accumulate in fewer journals than in OA journals, whereas in OA journals, citations tend to concentrate in top-tier journals.

The proportion of cited articles is higher in the case of the ten most cited TA journals compared with the ten most cited OA ones (55.0% vs. 50.7%), although the difference is small. The proportion of OA journals accruing up to one citation per year is higher than TA journals. However, from three citations upwards, TA journals accrue more citations.

In sum, after conducting different kinds of analyses, we cannot adhere to, nor reject the OAA hypothesis in TIS, since at present there seems to be a technical draw in this regard between both access types. However, our results reinforce the future of OA in our discipline in general and of OAA in particular, since publishing in OA in TIS has already equalled the probabilities of being cited.

4.2. Limitations and future lines of research

This study is innovative not only because it is the first of this kind in TIS, but also because it has included different parameters of analysis seldom seen in the literature.

As Kurtz and Brody (2006: 7) stated, “[t]he ‘open access advantage’ is a promising author-incentive to promote free access to the scholarly literature (with all the public benefit that comes with that), but establishing clear evidence that free access increases citation impact is beset with technical difficulties.”

Comparing our results with those of previous studies is not a straightforward task, not only because academic practices differ regarding document types, citation patterns and accessibility practices, but also due to terminological confusion. For instance, in TIS most OA journals follow the diamond OA model, gold OA journals are very rare and scholars publishing in hybrid journals tend to avoid paying APCs to make their papers OA, while delayed OA journals form a very small but important group in terms of indexation and impact. Therefore, the recurrent dichotomy found in the literature between the gold and subscription business models is not applicable to our case.

Our study has overcome some of the limitations of other studies, such as: a) being restricted to a short citation window; b) including only one kind of document type (mainly journals and journal articles); and c) being based on data from generalist bibliographical databases or on an ethnocentric view of research (because they focus almost only on research carried out or published in the West), to name just a few. Nevertheless, it is not exempt from its own limitations. We tried to control for the following confounding variables: field specificities, document format, early view effect, self-citations and time elapsed from publication to citation. It is, however, very difficult to sort out the effects of various underlying factors. As Kurtz and Brody (2006) report, OAA “is the most difficult factor to test independently, since it demands comparisons between samples of articles that are similar in all other aspects.” Therefore, there might be other variables affecting the OAA which we could not control for, such as reputation, co-authorship, or the impact of green or bronze access models.

TIS scholars’ patterns of OA culture are interwoven into their academic workflow and publishing values. Nevertheless, our findings only portray a part of the complex picture of ongoing changes in TIS. Since there is no complete index of open available documents, studies on OAA mostly involve case studies, which tend not to be representative. Follow-up studies and additional investigations are required, together with complementary analysis of the effects of the interaction of different factors on OAA.

It is also important to take into account that we are working with the citations detected in just one database, BITRA, which can only be a proxy (no database can ever be exhaustive in its coverage of a living and worldwide discipline). This is a limitation and it involves a bias due to the operative difficulty of working from Western Europe with languages and socio-cultural spaces which are unfamiliar to the compilers of BITRA and often use writing systems not based on the Latin alphabet. Nevertheless, as aforementioned, BITRA is the only bibliography to combine the TIS label with citation mining, and the largest and most inclusive database as regards TIS (with 52% of publications in English, as compared, for instance, with over 80% in WoS or 78% in Scopus according to their own self-presentations on the internet in 2020).

There are many other aspects and approaches that could be included in the future to contribute to the body of literature documenting OAA in TIS. For instance, as already mentioned, we have used BITRA’s detected citations as a proxy for scientific impact in TIS, but different citation sources lead to different numbers of citations. Therefore, our data could be compared with those obtained from other TIS non-specific bibliographical databases such as WoS and Scopus, although the corpus of journals they include is much smaller and less representative of the discipline. Further studies could also employ altmetrics, including but not limited to the number of downloads and web page visitors or social media outreach, to investigate if they are correlated with increased citations. The discipline also needs mixed or more qualitative approaches analysing whether OA documents increase their range of citing countries, gender differences, citations from neighbouring disciplines, if those journals which have flipped from TA to gold or hybrid OA tend to have an OAA compared to subscription journals or have improved their performance in terms of citations accrued, not to mention interviews with scholars to explore their attitudes towards OA. Since OA in TIS is on the increase, conducting longitudinal analyses of citations to papers published in a particular journal would also be interesting. Last, but not least, research on this topic would also very much benefit from the existence of a TIS repository of OA documents with good quality metadata that enabled new and updated bibliometric studies.

By empirically researching the impact of OA on citation behaviour in TIS, we hope to have contributed to this much debated topic, for which there is no academic consensus yet. OA is a dynamic and complex multifaceted phenomenon within the ever-changing landscape of scholarly practices. Therefore, this study needs to be further expanded, not only including other variables of study, but also updated and more exhaustive data to confirm or contradict current trends.