How Long It Takes to Review a Paper
Abstract
To gain insight into the duration and quality of the scientific peer review process, we analyzed data from 3500 review experiences submitted by authors to the SciRev.sc website. Aspects studied are duration of the commencement review round, full review elapsing, firsthand rejection time, the number, quality, and difficulty of referee reports, the time information technology takes authors to revise and resubmit their manuscript, and overall quality of the experience. We find clear differences in these aspects between scientific fields, with Medicine, Public wellness, and Natural sciences showing the shortest durations and Mathematics and Computer sciences, Social sciences, Economics and Concern, and Humanities the longest. One-third of journals have more 2 weeks for an immediate (desk) rejection and one sixth even more than 4 weeks. This suggests that besides the time reviewers take, inefficient editorial processes likewise play an of import role. As might exist expected, shorter peer review processes and those of accepted papers are rated more positively by authors. More surprising is that peer review processes in the fields linked to long processes are rated highest and those in the fields linked to short processes everyman. Hence authors' satisfaction is plainly influenced by their expectations regarding what is mutual in their field. Qualitative information provided past the authors indicates that editors can enhance author satisfaction past taking an independent position vis-Ã -vis reviewers and past communicating well with authors.
Introduction
The scientific peer review procedure is 1 of the weakest links in the process of scientific knowledge production. While it is possible to review a paper in less than a day (Ware and Mabe 2015), it may often lie untouched on reviewers' desks and in editorial offices for extended periods before it is evaluated. This ways a substantial loss of time for the scientific process, which has otherwise become much more efficient in the last decades. There are even indications that the duration of the peer review process may take increased in the concluding decades (Ellison 2002a; Azar 2007). Hence in that location are expert reasons for a disquisitional look at this procedure.
To gain insight into the duration and other key aspects of the peer review process, we clarify data from 3500 review experiences submitted by authors to the SciRev.sc website (world wide web.scirev.sc). On this website, researchers can share their experiences with the peer review process regarding manuscripts they have submitted to scientific journals. This information can subsequently be used by their colleagues when selecting a journal to submit their work. Information is available on several important aspects of the peer review process, including the duration of the commencement review round, full review duration, the time editors accept to inform authors about an immediate (desk-bound) rejection of a manuscript, the number and quality of referee reports, the fourth dimension authors take to revise and resubmit their manuscript, and the overall quality of the procedure as experienced by the authors.
Elapsing of the first review round—or offset response time (Azar 2007)—is probably near important for scientific authors as it determines how much time may be lost if the outcome is negative (Solomon and Björk 2012). The number of review rounds and the time journals have to manage these rounds are also important, as these aspects significantly bear upon the time that elapses until writer(southward) are informed of the concluding editorial determination. Another of import elapsing indicator is the immediate (desk) rejection time, i.east., the time taken by an editor to inform authors that the manuscript is not considered fitted for the journal. If this only takes a few days, authors tin can without much time loss send the manuscript to some other journal. However, quite often, editors may have weeks or fifty-fifty months for a desk rejection. This seems unacceptable and may point to a less than efficient organization of the editorial process. If editors take much time to inform authors that they are not interested in the manuscript, they probably volition likewise be rather slow in other aspects of manuscript treatment, such as assigning reviewers and processing review reports. The firsthand rejection time is thus a major indicator of a journal'south performance.
As well by the elapsing of the different steps of the peer review process, total publication time is also influenced by revision time, i.eastward., the time taken past authors to revise and resubmit the manuscript. This cistron is therefore besides included in our analysis. Information technology is influenced by the time authors are able and prepared to spend on the revision of the manuscript and past the difficulty of the revisions required. In this connection, it is important also to include aspects of the referee reports. Constructive comments by reviewers may substantially contribute to the quality of scientific papers, while low quality and contradictory referee reports may exist a major source of frustration among authors (Nicholas et al. 2015). In the SciRev questionnaire, authors are asked most the number of reports they received and how they experienced the quality of the reports and the difficulty of the changes they were required to make.
Too the measurable factors, such as the duration of the dissimilar phases of the peer review process and the number of referee reports, there are besides aspects of the process that are more difficult to quantify. Does the editor take questions of the author(s) seriously? Is a reasonable motivation for a (desk) rejection given? Does the editor take an independent position vis-Ã -vis reviewers when making of import decisions? Does the editor propose authors on the importance of specific reviewer comments? Together these aspects affect the author'due south experience with the journal and to a certain extent may turn a rejection into a good experience or an acceptance into a bad one. We therefore also clarify the authors' overall evaluation scores given to the journals for their peer review performance likewise as the motivations given by authors for their scores. Considering an author's review experience is influenced past many factors (e.g., the outcome of the review process, the impact cistron of the journal, and differences in expectations betwixt scientific fields), nosotros study the overall scores in a multivariate way and also clarify the authors' scoring motivations.
Background
There are around 28,000 scientific journals worldwide, which publish ii.5 million scientific manufactures annually, produced by a research customs of 6–nine 1000000 scientists (Ware and Mabe 2015; Jinha 2010; Björk et al. 2009; Plume and Van Weijen 2014; Etkin 2014). Many of the published manufactures accept been rejected at least once earlier they reached the editor's desk of the journal in which they were published. This means that each year many more manuscripts pass through peer review than are published.
Although in that location is some variation among journals, the peer review procedure typically starts with a kickoff evaluation of the manuscript past the editor, followed by a conclusion to accept the manuscript for peer review or immediately (desk) turn down it. If desk rejected, the corresponding author receives a message from the editor that the manuscript is considered not fit for publication in the journal, with or without a brief motivation given for the rejection. A manuscript that has passed this first stage will then exist send out for peer review, whereby experts in the field (peers of the authors) evaluate the manuscript and write a referee written report. On the ground of these reports, the editor decides either to reject the manuscript or gives the author(south) an opportunity to revise and resubmit it, or—in exceptional cases,—directly accepts it. In case of a revise-and-resubmit, several additional review rounds may follow earlier a final decision regarding acceptance or rejection is made. If the process takes exceptionally long, the writer may make up one's mind to withdraw the manuscript and submit information technology to another journal.
Process too slow
Given the fact that reviewers are ofttimes overloaded with bookish piece of work, that they are mostly non paid for their review work, and that reviews are mostly anonymous, there are few incentives to requite high priority to this piece of work (Azar 2007; Moizer 2009). Hence, while the actual time it takes to write a referee report may vary between a few hours and a day (Ware and Mabe 2015), reviewers tend to have several weeks to several months to submit their reports. Apart from the time reviewers take to deliver their reports, the total manuscript processing fourth dimension of journals is influenced by the duration of the diverse stages of manuscript handling at editorial offices. Given that these offices oftentimes have limited resources and many editors do this work as well busy academic careers, waiting times at the different stages are oft (much) longer than strictly necessary.
It is therefore not surprising that one of the most of import criticisms of the peer review organization is that it is much too dull (Lotriet 2012). There are even indications that is has been getting slower in recent decades (Alberts et al. 2008). Ellison (2002a, 2002b) documents a slowdown since the 1970s in submission-acceptance duration in economics and suggests a like slowdown in other fields. A major crusade for this is that authors are required to revise their manuscripts more than often and more extensively (Ellison 2002a, 2002b; Azar 2007; Cherkashin et al. 2009; Björk and Solomon 2013). According to Ellison (2002a), review rounds are of quite recent appointment. In the early 1950s, 'almost all submissions were either accepted or rejected: the noncommittal "revise-and-resubmit" was reserved for exceptional cases (p. 948).'
From the writer'due south perspective, first response time is particularly important, i.e., the time that elapses between submission and offset response from the editor, exist it rejection, acceptance, or a revise-and-resubmit. Start response time is important because it ofttimes delays the publication of an commodity more one time, equally many manuscripts are rejected once or several times before credence (Azar 2007; Etkin 2014; Pautasso and Schäfer 2010). There are indications that duration of the first review round has increased, at to the lowest degree in some fields. Azar (2007) finds that first response time for economic journals "grew from about 2 months circa 1960 to about 3–6 months in the early 2000s (Azar 2007, p. 182)". However, as Azar points out, a longer starting time response fourth dimension is in itself not necessarily negative. Economics manuscripts accept become longer over time and accept more mathematical content, which means it is more time-consuming to evaluate them.
Field departure
Durations vary essentially between scientific fields and even within the same broader subject field. Kareiva et al. (2002), for instance, studying conservation biology, found that the procedure from submission to publication took on boilerplate 572 days for conservation and applied ecology journals compared to 249 days for genetics and evolution journals.
With respect to the number of times the boilerplate manuscript is rejected before it reaches the journal that volition publish it, Azar (2004) arrives at a figure of three to vi rejections. Similar to an increase in beginning response time, there as well seems to be an increase in the number of rejections prior to publication. Thomson Reuters (in Ware and Mabe 2015, p. 51) reports an increase in the rejection rate from 59 to 63% between 2005 and 2010. Regarding the desk rejection charge per unit, Lewin (2014) reports an increase of up to three times for some journals. Lewin attributes this to increased publication pressure level, whereby "governments in countries exterior of the USA appoint in a process of quantifying the scholarship of scientists in their countries equally a way of rationalizing the allocation of national resources to institutions of higher learning in their countries. The unsurprising effect has been a dramatic increase in submissions to the top journals by scholars from emerging economies likewise as from European countries" (Lewin 2014, p. 169).
Editors are likewise worried virtually these developments. 'Amidst periodical editors there are growing concerns that the quality—and duration—of the review process is being negatively afflicted equally "referees are stretched thin past other professional commitments". This often leads to "challenges in finding sufficient numbers of reviewers in a timely manner" (Lotriet 2012, p. 27).' One time reviewers take been establish, other issues may emerge, such every bit poor reviewer understanding on submissions (Peters and Ceci 1982; Onitilo et al. 2014) or upstanding issues (Resnik et al. 2008). Reviewers who brand contradictory comments are a major source of frustration for authors likewise as editors. Regarding unethical practices, Resnik et al. (2008) mention (in order of frequency) reviewers asking authors to include 'unnecessary references to their publication(s), personal attacks, reviewers delaying publication to publish a paper on the same topic, breach of confidentiality and using ideas, information, or methods without permission (p. 305)'.
Ways to better
Several suggestions accept been washed to make it more bonny for scientists to act every bit reviewers. Free subscription to journal content, annual acknowledgement on the journal's website, more feedback about the consequence of the submission and quality of the review, engagement of reviewers to the journal'southward editorial board and financial incentives (Tite and Schroter 2007). A noteworthy initiative in this respect is Publons (world wide web.publons.com), a website where reviewers can upload information on bearding review work they performed. This information is then verified with the journals and can subsequently be used as 'proof' of the peer review work done by the reviewer. This initiative provides a solution to the recognition problem. However, it does not assistance solve the problems of duration and quality every bit neither the time reviewers spent writing the reports nor the quality of their reports are registered.
Equally to financial incentives, Thompson et al. (2010) establish a statistically pregnant reduction in review duration when referees were paid for their efforts. 'Median get-go response time was reduced from 90 to 70 days, a 22% reduction in the presence of payments. With payments, but ane% of first response times exceeded vi months; without payments, 16% exceeded 6 months (Thompson et al. 2010, p. 678).' Although it was not possible to compare the quality of referee reports submitted with or without payment, they idea information technology likely that if the length of referee reports was an indication of quality, payment might even have led to an increase in referee reports' quality: "[r]eferees did not nuance off shorter reports to come across the deadline for payment; in fact, reports were statistically significantly longer with payments than they were prior to payments" (Thompson et al. 2010, p. 690).
Previous studies by Hamermesh (1994) for seven journals in 1989 too found an increase in timely referee reports for journals offer payments. However, since "some empirical evidence suggests that when voluntary economic activities—giving blood, volunteering to work for public or individual institutions, and collecting donations for clemency, for example—are rewarded with relatively low payment levels, low-paid functioning is inferior to voluntary functioning" (Thompson et al. 2010, p. 680), most likely reviewers would have to receive a realistic rather than a symbolic payment for their efforts.
It seems natural to expect that authors of papers that have been accepted are happier with the review experience, when they wait dorsum at it in hindsight. Authors tend to endure from attributional bias. If their newspaper is rejected, many authors tend to blame this on situational factors, such as incompetent reviewers or uninterested editors, but in case of credence tend to aspect this to their own expertise and competence in writing high-quality papers (Garcia et al. 2016). The difference in ratings between authors of accepted and rejected manuscripts might also be greater, the longer the elapsing of the peer review procedure. The more time and energy authors invest in a manuscript, the more than likely it is they volition be disappointed by a rejection, and even more so if rejection follows afterwards several review rounds.
Methods
The data used in this paper are based on 3500 review experiences, reported by authors between 2013 and 2016, past filling in a questionnaire on the SciRev.sc website. The SciRev questionnaire contains questions on the duration of the unlike phases of the peer review procedure of inquiry articles, on the number, quality, and difficulty of the received referee reports, on the result of the peer review process, and on whether the manuscript has previously been submitted to another periodical. It also asks authors to provide an overall rating of the review experience and gives them the opportunity to motivate their rating. Research articles may include any paper submitted to a scientific journal (regular enquiry papers, review articles, rapid communications, research notes, etc.), provided it has been subjected to peer review.
Authors who submitted a review to SciRev.sc were asked about their affiliation, which was checked by request them for their institutional email accost and sending a confirmation link to that address. Authors who registered with a noninstitutional e-mail address, because for various reasons they could not provide an institutional one (e.yard., chore modify or working in a not-Western institute without proficient ICT services), were asked for additional information to bank check their identity. Reviews were only accustomed if the writer's identity was confirmed. Reviews of accepted papers were additionally checked at the journals' websites; these reviews were only included if the writer had indeed published a paper in the periodical during the menstruation mentioned.
Although the information are not based on a representative sample of writer experiences, they are interesting because they paint a broad picture of the range of author experiences from different fields of study. Each submitted review represents the experience of an writer and is important as such. If other authors report similar experiences, this would betoken toward a specific pattern. And if the resultant patterns differ among scientific fields, this would indicate that the prevalence of specific experiences differs among those fields.
At that place is piddling reason to expect authors from different fields to be fundamentally different in the way they experience the different aspects of the peer review process. All the same, there might be unlike expectations between fields almost review duration and hence about what is considered a long process. Besides past field differences, experiences may also be colored by the process outcome and the journal's bear upon cistron. Nosotros therefore split the figures presented in this paper according to scientific field and process outcome (accepted/rejected) and as well study relationships with the journal's impact factor. Information on the bear on cistron was derived from the periodical'southward website and other Internet sources. This information could exist found for 3126 reviews. In our assay, we utilise the natural logarithm of the impact gene, as more journals are full-bodied in the lower ranges of the impact factor.
Of the 3500 review experiences, 572 (16.3%) referred to manuscripts that were rejected without being sent to reviewers, 693 (xix.viii%) that were rejected after the offset review round, 2128 (60.viii%) that were accustomed after one or more than review rounds, 43 (one.2%) that were immediately accepted without peer review procedure, and 64 (1.8%) that were withdrawn past the author. Given the relatively small number of reported cases of manuscripts that were withdrawn or immediately accepted, these were not included in our analysis. Nosotros also removed some farthermost cases regarding firsthand rejection time (>62 days; 53 cases), elapsing of offset review round and total review elapsing (>100 weeks; fifteen cases), and elapsing of revision after first review round (>300 days; six cases). The extreme cases were not concentrated in specific fields.
Data on the diverse aspects of the peer review procedure is presented for all review experiences, separately for accustomed and rejected papers and for ten major scientific fields: (1) General journals (n = 172), (2) Natural sciences (n = 1408), (3) Engineering (including technology; n = 518), (4) Mathematics and Computer sciences (northward = 375), (5) Medicine (n = 640), (six) Public health (including health professions; due north = 348), (7) Psychology (including education; due north = 355), (8) Economics and Concern (including police force; n = 318), (9) Social sciences (north = 553), and (10) Humanities (north = 178). Given that a substantial number of journals accept a wide scope and therefore include more than than one scientific field, the sum of the reviews in the dissimilar fields is higher than the full number of reviews.
At the end of the SciRev questionnaire, authors are asked to requite an overall rating of their review experience. Because this experience is influenced past many aspects of the peer review procedure, also providing descriptive figures, also a multivariate regression analysis is performed. In this assay, the variation in the rating is explained on the ground of relevant characteristics of the process, i.e., whether or not the newspaper was accepted or rejected, the duration of the first review circular, the number of review rounds, the number of referee reports received in the commencement review round, whether the author is from an English language-speaking country, and the scientific field of the periodical. We present both directly effects of these factors and significant interactions between them. For journals roofing several scientific fields, we merely included the journal'southward main field in this analysis.
In the multivariate analysis, we excluded reviews of papers that were withdrawn, immediately accustomed, or desk rejected. Among the remaining 2821 reviews, there were some missing values. 5 reviews for which elapsing of the first review round was missing were given the average duration of the get-go review round. Two reviews where the language of the reviewer was missing were included in the non-English (biggest) category. For 289 cases the impact factor was missing. These missings were addressed using the dummy variable adjustment procedure [imputing the mean and including a dummy indicating the missings (cf. Allison 2001)]. Results of the analysis with missing values dealt with in this way were substantially the aforementioned as those with all missings removed from the information.
The overall rating of the review experience is measured on a calibration running from 0 (very bad) to 5 (excellent). The event of the peer review process is a dummy indicating whether the paper was accustomed (1) or rejected (0). The duration of the first review round is measured in days. To indicate language groundwork, we included a dummy indicating whether (ane) or not (0) the organisation where the author works is located in a state where English is the main language used in daily life (i.due east., United Kingdom, Ireland, U.s.a., Canada, Australia, New Zealand, Due south Africa, and British Indian Ocean Territory). Of the 3500 reviews, 2516 were submitted by authors from not-English-speaking countries. Regarding the distribution of reviews over continents, 557 were obtained from Canada and the USA, 96 from Latin America and the Carribean, 2099 from Europe, 470 from Asia and the Pacific, 190 from the Middle Eastward, 83 from Africa, and five of which the continent is not known. For the dummies for scientific field, deviation from hateful (effects) coding is used. The dummies therefore indicate to what extent the overall rating within the field is higher or lower than the mean of the fields (Hardy 1993).
Afterward rating the overall review feel, authors are given the opportunity to motivate their rating in a few words or sentences. These motivations are published online with the reviews, if permission is given past the author. They paint a sometimes revealing picture of what researchers feel in their attempts to get their work published. To supplement the figures presented in this paper with qualitative information, we analyzed the 1879 motivations available in the 3500 reviews studied.
Results
Kickoff response fourth dimension
For authors, the elapsing of the outset review circular, or first response time, is probably the factor they are by and large interested in, as this takes up a substantial part of the full manuscript evaluation fourth dimension and to a large extent determines how much time is lost if the effect is negative. Beginning response time includes the time taken past the journal for a first evaluation of the manuscript, finding reviewers, the time the latter require to do their work, and the time the editor so requires to evaluate the manuscript in light of the referee reports and to inform authors about the determination.
Equally can be seen in Table 1, the reported first response time in the SciRev data is on average xiii weeks and varies considerably among scientific fields. Information technology took 8–9 weeks in Medicine and Public health related journals, 11 weeks in Natural sciences and General journals, 14 in Psychology, and 16–18 weeks in Social sciences, Humanities, Mathematics and Reckoner sciences, and Economics and Business. These figures differ betwixt accustomed and rejected manuscripts, with first response time of rejected manuscripts taking, on average, iv weeks longer.
While writing a peer review may accept between iv and 8 h, in only nineteen% of all reported cases authors were informed about the outcome in less than a calendar month. In about one tertiary of the cases (32%) authors had to expect 3 months or more and in 10% of the cases even more than than 6 months before being informed. Duration differs widely between scientific fields. In Social sciences and Humanities, just 7–8% of the authors were informed within ane month versus 25% in Natural sciences and 27–28% in Medicine and Public wellness. In Economics and Business and Mathematics and Computer sciences over one sixth (eighteen%) of authors had to wait 6 months or longer.
It is yet unclear to what extent the long duration of the starting time review round is the issue of the peer review process every bit such and to what extent it is due to (in)efficient manuscript handling at editorial offices. Given that firsthand rejection times are often long (run into Table iii and its discussion below), it seems that inefficiencies at editorial offices also play an important role. The finding that in Medicine and Public Health—where professionalization of journals is relatively loftier—beginning response times are the shortest, also points in this direction.
To test this idea further, nosotros looked at the relationship between the periodical's impact gene and first response time. Every bit highly ranked journals generally accept more resources at their disposal and thus probably better organized editorial offices, and as reviewers are more motivated to review for those journals, nosotros expected to discover a negative relationship. Pearson correlations between showtime response fourth dimension and impact factor indeed confirm this expectation. These correlations are significantly negative for all scientific fields combined (P = −0.29) too as for all scientific fields separately, with Full general journals (P = −0.51), Mathematics and Reckoner sciences (P = −0.27), and Natural sciences (P = −0.26) having the highest correlations. The simply exception was Humanities, where no significant correlation between first response time and impact cistron was found. This might be considering this field traditionally values publishing books more than publishing in journals (Ware and Mabe 2015).
Total review duration
Full review duration refers to the time a manuscript is nether responsibility of the periodical. As well by the elapsing of the first review round, total review duration is as well determined by the number and duration of subsequent review rounds. Full review duration does non include the fourth dimension taken by authors to revise and resubmit their manuscript. Given that rejected manuscripts accept on average less review rounds, nosotros restrict this analysis to accustomed papers.
Table two shows that the reported total review duration of accustomed manuscripts is on average 17 weeks. Again there are substantial differences between scientific fields. With 12–14 weeks, average total review elapsing is shortest in Medicine, Public health, and the Natural sciences. It is longest in Economics and Business organization, where the procedure takes on average 25 weeks and is twice as long. In Mathematics and Figurer sciences, Social sciences and Humanities, total review elapsing is also long, i.e., 22–23 weeks. Hence the differences in the elapsing of the review processes we observed for the first review round are too present in the other aspects of the process.
If we divide out the data further, we note that in Natural sciences, Medicine, and Public health 13–16% of the manuscripts pass through the unabridged peer review process inside i calendar month, that this applies to about two thirds of the manuscripts after 3 months, and to 87–92% of the manuscripts after 6 months. In Mathematics and Calculator sciences, Social sciences, and Humanities, these figures are iii–4%, one third and slightly higher up two thirds, respectively. Whereas only 8% of the authors in Medicine had to await more six months, this applies to one third of authors in Social sciences and Economic science and Concern.
The total time a manuscript is with the journal is determined by the time a journal takes for a review round and by the number of review rounds. As mentioned in the Background-department, there are indications that the number of review rounds has increased in recent years. In our data, the number of review rounds on average amounts to 2.03, with Psychology (2.23), General journals (2.18), Economics and Business (2.16), and Social sciences (2.15) showing a higher average number of review rounds.
Total review duration correlates significantly and negatively (−0.27) with a journal's impact cistron, thus indicating that total review duration is shorter for college impact factor journals.
Immediate (desk) rejection fourth dimension
Immediate rejection time is the time an editor takes to inform authors that he or she is not interested in the manuscript (and volition therefore not send information technology to reviewers). Our figures conspicuously show that firsthand rejection time is a major source of unnecessary time loss in the peer review process (Table 3). On average, an immediate rejection in Medicine takes 10 days, closely followed by Natural sciences, Public health, and Applied science, taking 11–12 days. Journals in Psychology, Social sciences and Mathematics and Computer sciences take half equally long, i.e., 15–17 days. These are relatively high averages, given that in many cases an inspection of the abstract is sufficient to decide that a newspaper does not fit.
On the positive side, in one-half (50%) of the reported immediate rejection cases, the editor informed the author(s) inside i week. However, the information likewise show that in 17% of cases authors had to wait more than 4 weeks to exist informed of the rejection. Several authors even had to look for more than iii months, or withdrew their manuscripts after hearing aught for an even longer catamenia. These are conspicuously unacceptable practices.
The state of affairs is all-time in Medicine, where 62% of authors are informed about an immediate rejection within 7 days, followed by Natural sciences and Public health where this effigy is 54%. Immediate rejection time is longest for authors in the Social sciences and Mathematics and Calculator sciences, where in about 30% of reported cases it took the editor iv weeks or more to inform writer(southward) that he or she was not interested in the manuscript and would non to ship it to reviewers. There is a significant negative correlation (−0.eighteen) between immediate rejection time and the journal's impact factor, which indicates that journals with a higher touch gene have editors who work faster and editorial offices that are more professionally organized.
Reviewers are generally blamed for long processing times, but our findings indicate that manuscript treatment at editorial offices plays an important office also. If editors have a calendar month for an immediate rejection determination, they are probably also slow in finding reviewers and processing referee reports.
Referee reports
The boilerplate number of referee reports is nearly 2.2 in all scientific fields (come across Table four). This correspondence is remarkable, given the substantial differences betwixt fields in other respects. There is slight variation in the experienced quality of the referee reports between the fields [as indicated on a scale running from 0 (very bad) to v (excellent)]. Authors report the quality of the reports to be somewhat higher in Natural sciences, Engineering science, and Public health (3.vii), and lower in General journals, Psychology, and Economic science and Business (3.4). It is interesting that the long review duration in Economics and Business did not interpret into referee reports experienced of college quality.
Authors who were given the opportunity to revise and resubmit their papers were besides asked to what extent they perceived the requested changes every bit difficult and whether they thought their manuscript had improved equally a event of the revision. At that place is a meaning positive correlation (0.40) between these factors. When the revision was experienced as more difficult, authors were as well more satisfied with the improvement. Regarding the difficulty experienced, revision processes were perceived as easiest in Mathematics and Computer sciences and in Public wellness (ii.half-dozen), and as most hard in Economics and Business (3.3). Regarding the experienced improvement of the manuscript as a upshot of the revision, authors from Social sciences, Economics and Business, and Humanities reported somewhat college figures (3.8 and 3.nine) compared to the other scientific fields (3.vii).
There is a minor positive correlation (0.07) betwixt the difficulty experienced regarding the referee reports and the impact factor of the journal. Thus, reviewers of more than highly ranked journals tend to make somewhat greater demands on the authors. The degree of comeback experienced regarding the manuscript is non significantly related to touch factor.
Revision time
The time from the get-go submission date to the concluding decision appointment is not only influenced by the time the manuscript is at the editorial part or being reviewed, but also by the time authors take to revise their manuscript. Information technology is therefore important to look also at the duration of the revision time. Table five shows that authors who received a revise-and-resubmit on boilerplate take 39 days to revise their manuscript, but there is substantial variation among the fields. Authors in Economics and Business accept longest to revise their manuscripts: on average 64 days to prepare and submit a revised version. This is essentially longer than authors in Natural sciences, Technology and Mathematics and Computer sciences (32–34 days) and in Public health (29 days). Plain, in Economics and Business it is not just the editors who take more than fourth dimension.
Table 5 also shows the pct of manuscripts revised inside a specific number of days. While xviii% of authors in Engineering, Mathematics and Computer sciences and Public health revise their manuscript within seven days, this applies to 9–10% of authors in Social sciences and Humanities and just 3% of authors in Economics and Business.
Regarding the relationship between the journal'southward touch cistron and the time authors accept to revise their manuscript, nosotros expected authors who received a revise-and-resubmit from a high-level journal to be more motivated to complete the revision of their manuscript quickly. However, no significant correlation was found between revision time and the journal's impact factor.
Rating of peer review experience
The SciRev questionnaire gives authors the opportunity to provide an overall rating of the review experience on a scale from 0 (very bad) to 5 (excellent); see Table half dozen for details. Authors of accepted manuscripts give the peer review process a much higher rating (4) than authors of rejected manuscripts (2.2). Moreover, the rating of the peer review procedure is negatively related to total review duration. This correlation is −0.43 for both accepted and rejected manuscripts.
To make up one's mind how the various factors might bear on the satisfaction of authors with the peer review procedure, we turn to the results of multivariate analyses (encounter Table 7). The offset columns show the results of Model i, which contains all relevant variables. Model 2 contains the same variables only also the significant interactions betwixt the variables.
As tin be seen in Model 1, all variables, except impact gene, are significantly related to authors' rating of the peer review procedure of their manuscript. As expected, authors of accustomed manuscripts rate the process significantly more positive than authors of rejected manuscripts. Authors tend to suffer from attributional bias: if their newspaper is rejected, they often blame this on situational factors such as incompetent reviewers and uninterested editors; only if it is accepted they tend to attribute this to their own expertise and competence in writing high-quality papers (Garcia et al. 2016).
Authors also value speed of the peer review process. When the elapsing of the showtime review round is shorter and there are fewer review rounds, authors requite the procedure a significantly higher rating. Authors who receive more referee reports also tend to exist more positive about the process. Their perception might exist that their manuscript has been dealt with more seriously and thoroughly. Authors from countries where English language is the first language rate the peer review process less positive than authors from other countries. It is possible that these authors have higher expectations of the process and are more critical regarding aspects that do non meet their expectations.
Taking into business relationship other factors, authors in Economics and Business, Social sciences, Psychology, and Mathematics and Computer sciences are more positive about the peer review process than authors in Natural sciences, Medicine, Public health, and specially General journals.
When we include the significant interactions in the model (Model 2), the sign and significance of the main effects stay the aforementioned. The interaction analysis shows that the negative issue of a longer duration of the offset review round and the negative result of more review rounds are less profound for accustomed papers. Hence it seems that authors are willing to have extensive revision work if this is rewarded with the acceptance of their paper. At the same time, they seem especially disappointed if the manuscript is notwithstanding rejected afterwards a long review procedure.
The negative interaction betwixt a paper being accustomed and the number of referee reports indicates that authors of rejected papers may consider a higher number of reports every bit a sign that their newspaper was taken seriously and might exist content with extensive feedback. For obvious reasons, authors of accustomed papers are more positive when the journal has a higher bear upon cistron. Authors from English-speaking countries are less negative about the peer review process when their paper is accustomed and when they receive more than referee reports but find a long process more problematic. This might reflect that they have higher expectations that their paper will be accepted and that the peer review process will be short and efficient compared to authors from non-English-speaking countries.
When the elapsing of the start review round is longer, or when the touch cistron of the journal is higher, authors are more concerned about a higher number of review rounds. In those cases, they might await a smooth continuation of the procedure and be more disappointed when this proves not to be the case. A longer elapsing of the first review round is considered less negative past authors who receive more referee reports.
Qualitative findings
The motivations authors requite for their rating of the peer review process on SciRev.sc contain important qualitative information on author experiences. We analyzed these motivations and registered the author's major business concern(south). A first important observation is that well-nigh half (918) of the 1879 comments is positive. Many authors, in item of accepted papers, are satisfied with the process and limited their gratitude in their motivations. Of the 961 comments with a negative connotation, 371 (39%) limited concerns nigh the duration of the review procedure. This attribute of long review duration is included in the quantitative outcomes and has been discussed in the preceding sections.
A more informative source of discontent, mentioned 437 times (45%), concerns the function of editors and editorial offices. Poor advice of editors/offices—in particular not reacting to information requests—are a major source of frustration mentioned by authors. We received reports of authors who waited over six months without hearing anything of the journal or receiving reactions to data requests. Also editors who 'hide' backside reviewers and do non take an independent position vis-Ã -vis them are perceived as problematic. In detail when referee reports are contradictory—as often happens—it is important that editors provide guidance and indicate the comments on which authors should focus in their revision.
Poor quality of referee reports is mentioned in 141 (15%) of the critical comments. Referee reports are frequently perceived to be superficial, contradictory, unreadable, inquire unreasonable modifications, or convey the impression that the reviewer did non read or sympathize the newspaper. Some other issues mentioned are the addition of completely new comments in the second review round, the theft of ideas, or asking for unnecessary references.
Conclusion
In this newspaper we study various aspects of the peer review procedure on the footing of 3500 review experiences reported in the last 3 years on the SciRev.sc website. Aspects discussed include the start response fourth dimension (duration of the beginning review round), full review duration (the fourth dimension the manuscript is at the editorial part or with reviewers), the firsthand rejection time, the fourth dimension authors take for their first revision (revision time), the number, quality, and difficulty of referee reports received, and the overall rating of the process.
We find considerable variation between the ten scientific fields distinguished. Whereas the reported first response fourth dimension is eight–9 weeks for Medicine and Public health, information technology is 11–xiv weeks in Natural sciences, Technology, Psychology, and General journals and 16–18 weeks in Economics and Business, Social sciences, Mathematics and Computer sciences, and Humanities (Table one). There is too considerable variation around these averages. While 27–28% of authors in Medicine and Public wellness were informed within a month, 18% of authors in Mathematics and Computer sciences and Economics and Concern had to wait more than 6 months for a decision. As expected, these figures too interpret into longer total review durations reported for the scientific fields with longer starting time review rounds (Tabular array 2).
The long elapsing of the peer review process is often blamed on reviewers taking much time to complete their reports. However, our figures point that inefficient editorial processes are also of import. The reported firsthand rejection time (Table 3), which is not influenced by reviewers, shows substantial variation among the fields and is often unreasonably long. Whereas in half of the immediate rejection cases authors were informed within a week, in about ane sixth of these cases authors had to wait for more than than 4 weeks. Medicine performs best with an average of x days, Natural sciences, Public health and Engineering come second with 11–12 days. Psychology, Social sciences, and Mathematics and Computer sciences have longest with 15–17 days. If editors take much fourth dimension for a desk rejection, it is likely they too take much fourth dimension finding reviewers and processing incoming referee reports. Immediate rejection time is therefore a powerful indicator of the overall functioning of editorial offices.
The full time betwixt submission of a manuscript and the last decision of the editor is non only influenced by the time reviewers take to submit their reports and the time editorial offices accept to handle the manuscript, but also past the time authors take to revise and resubmit their manuscript (Table five). In this respect, the situation is similar to that of the other durations. While, on average, authors take 39 days to revise their manuscript, authors in Psychology and Social sciences take 50 days, and those in Economics and Business even 64 days. On the other hand, authors in Public Wellness, Engineering, Mathematics and Computer sciences, and Natural sciences accept but 29–34 days for a revision. The longer elapsing in some fields is not associated with a college number of referee reports (two.0–two.three) nor with more difficult referee reports (ii.6–3.3).
Most characteristics of the peer review process studied are related to the journal'south affect factor. More highly ranked journals have a shorter duration of the commencement review round (P = −0.29), total review duration (P = −0.27), and immediate rejection time (P = −0.18), all indicating that review processes of more highly ranked journals are more efficient. We also found a pocket-size but significant positive correlation (P = 0.08) between experienced difficulty of the referee reports and impact factor, indicating that reviewers of more than highly ranked journals are somewhat more enervating.
As expected, authors of accepted manuscripts are more satisfied with the peer review experience than authors of rejected papers (Tabular array half-dozen). On a calibration from 0 (very bad) to v (excellent), they rate the process a iv, compared to a 2.two for authors of rejected manuscripts. A longer duration of the process is negatively associated with the rating, independent of the process upshot. For both accustomed and rejected manuscripts the Pearson correlation coefficient between total review duration and rating is −0.37.
To assess the independent associations between the characteristics of the process and the satisfaction of authors, a multivariate regression analysis was performed with the overall rating of the process as dependent variable (Table 7). This analysis shows that even when the other variables are taken into account, all three aspects, i.e., a shorter duration of the outset review circular, a lower number of review rounds, and acceptance of the paper, are associated with a significantly higher overall rating of the feel. Interestingly, information technology also shows that, in spite of the longer duration in Economics and Business organisation, Social sciences, and Mathematics and Computer sciences, authors in those fields are more positive about the process than authors in the General journals, Medicine and Public wellness, where processes are shorter. Expectations thus clearly play a function.
Equally expected, authors of accustomed papers are even more than positive if the journal has a higher impact factor. They are (after) also less bothered by a longer duration of the first review round and by more 1 review round. Nosotros also discover that authors rate the process more positive if they receive more than referee reports, in item after a long first review round and when the manuscript is rejected. This indicates that authors appreciate the work of reviewers and the feedback given on their manuscripts. Compared to authors from non-English-speaking countries, those from English-speaking countries are more often than not less satisfied with the procedure, specially when their manuscript is rejected or in case of more than one review round. This suggests that authors from English-speaking countries have higher expectations of the peer review process.
References
-
Alberts, B., Hanson, B., & Kelner, K. 50. (2008). Reviewing peer review. Science, 321, fifteen.
-
Allison, P. (2001). Missing information. London: Sage Publications Ltd.
-
Azar, O. H. (2004). Rejections and the importance of first response times. International Journal of Social Economics, 31(3), 259–274.
-
Azar, O. H. (2007). The slowdown in first-response times of economics Journals: Tin information technology exist beneficial? Economic Enquiry, 45(ane), 179–187.
-
Björk, B., Roos, A., & Lauri, K. (2009). Scientific journal publishing: Yearly volume and open access availability. Information Research, 14, 1.
-
Björk, B., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Informetics, 7, 914–923.
-
Cherkashin, I., Demidova, S., Imai, S., & Krishna, Thou. (2009). The inside scoop: Credence and rejection at the journal of international economic science. Journal of International Economics, 77, 120–132.
-
Ellison, G. (2002a). The slowdown of the economics publishing process. Journal of Political Economy, 110(5), 947–993.
-
Ellison, G. (2002b). Evolving standards for academic publishing: A q-r theory. Journal of Political Economy, 110(5), 994–1034.
-
Etkin, A. (2014). A new method and metric to evaluate the peer review process of scholarly journals. Pub Res Q, thirty, 23–38.
-
Garcıa, J. A., Rodriguez-Sanchez, Rosa, & Fdez-Valdivia, J. (2016). Why the referees' reports I receive as an editor are and so much better than the reports I receive equally an author? Scientometrics, 106, 967–986.
-
Hamermesh, D. South. (1994). Facts and myths about refereeing. Journal of Economic Perspectives, 8(1), 153–163.
-
Hardy, M.A. (1993). Regression with dummy variables. Sage.
-
Jinha, A. E. (2010). Article 50 million: An estimate of the number of scholarly articles in being. Learned Publishing, 23, 258–263.
-
Kareiva, P., Marvier, M., West, S., & Hornisher, J. (2002). Deadening-moving journals hinder conservation efforts. Nature, 420, 15.
-
Lewin, A. Y. (2014). The peer-review process: The good, the bad, the ugly, and the extraordinary. Management and System Review, 10(2), 167–173.
-
Lotriet, C. J. (2012). Reviewing the review process: Identifying sources of delay. Australasian Medical Journal, 5(i), 26–29.
-
Moizer, P. (2009). Publishing in accounting journals: A fair game? Bookkeeping, Organizations and Society, 34, 285–304.
-
Nicholas, D., Watkinson, A., Jamali, H. R., Herman, E., Tenopir, C., VolentineK, R., et al. (2015). Peer review: Still king in the digital age. Learned Publishing, 28(1), 15–21.
-
Onitilo, A. A., Engel, J. Yard., Salzman-Scott, S. A., Stankowski, R. V., & Suhail, A. R. (2014). A cadre-item reviewer evaluation (CoRE) system for manuscript peer review. Accountability in Research: Policies and Quality Balls, 21, 109–121.
-
Park, I.-U., Peacey, M. W., & Munafo, One thousand. R. (2014). Modelling the effects of subjective and objective decision making in scientific peer review. Nature, 506, 93–98.
-
Pautasso, 1000., & Schäfer, H. (2010). Peer review filibuster and selectivity in ecology journals. Scientometrics, 84, 307–315.
-
Peters, D., & Ceci, South. (1982). Peer-review practices of psychological journals: The fate of published manufactures, submitted over again. The Behavioral and Brain Sciences, five, 187–255.
-
Plume, A., & van Weijen, D. (2014). Publish or perish? (p. 38). The ascent of the fractional author: Research Trends.
-
Resnik, D. B., Gutierrez-Ford, Ch., & Peddada, Southward. (2008). Perceptions of ethical problems with scientific journal peer review: An exploratory study. science eng Ideals, 14, 305–310.
-
Solomon, D., & Björk, B. (2012). Publication fees in open admission publishing: Sources of funding and factors influencing choice of journal. Journal of the American Society for Computer science and Technology, 63(1), 98–107.
-
Thompson, G. D., Aradhyula, S. V., Frisvold, G., & Frisvold, R. (2010). Does paying referees expedite reviews?: Results of a natural experiment. Southern Economic Journal, 76(3), 678–692.
-
Tite, 50., & Schroter, S. (2007). Why practise peer reviewers decline to review? A Survey, Cintinuing Professional Education, 61, 9–12.
-
Ware, M., & Mabe, Thou. (2015). The STM report: An overview of scientific and scholarly journal publishing. The Hague: International Association of Scientific, Technical and Medical Publishers. http://www.stm-assoc.org/2012_12_11_STM_Report_2012.pdf.
Acknowledgement
This article is based upon work from COST Activeness TD1306 "New Frontiers of Peer Review", supported by COST (European Cooperation in Scientific discipline and Technology).
Author information
Affiliations
Corresponding writer
Rights and permissions
Open Access This article is distributed under the terms of the Artistic Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided yous give appropriate credit to the original author(s) and the source, provide a link to the Creative Eatables license, and point if changes were made.
Reprints and Permissions
Nigh this article
Cite this article
Huisman, J., Smits, J. Duration and quality of the peer review process: the author'south perspective. Scientometrics 113, 633–650 (2017). https://doi.org/10.1007/s11192-017-2310-v
-
Received:
-
Published:
-
Upshot Appointment:
-
DOI : https://doi.org/10.1007/s11192-017-2310-5
Keywords
- Peer review process
- Duration
- Quality
- Author's experience
Source: https://link.springer.com/article/10.1007/s11192-017-2310-5
Post a Comment for "How Long It Takes to Review a Paper"