...
- When asked about the importance of improving a specific range of services, more than 70% of respondents said that improving search functions to allow more refined results was very important/important across all groups by years of use, age groups, number of articles published, country groups, and subject areas. Many commenters requested enhanced functions such as author search, date-limited searching, and searching non-English languages. Search was equally problematic regardless of whether the user searched for a known paper, was browsing a subject category, or looking for specific authors.
- A series of questions asked users about improving the submission process specifically with (1) support for submitting research data, code, slides and other materials; (2) improving support for linking research data, code, slides, etc., with a paper; and (3) updating the TeX engine and various other enhancements. Support About 40% of respondents rated each one as very important/important. The open text responses also displayed considerable interest in better support for supplemental materials, although respondents disagreed as to whether they should be hosted by arXiv or another party. Many respondents are supportive of integrating or linking to other services (especially GitHub), while a significant number of respondents also indicated doubts about long-term availability and link rot for content not hosted within arXiv. Some expressed concerns regarding the resources required for arXiv to improve this. There was some interest in including the data underlying figures in arXiv papers.
- Among other services and improvements recommended by respondents were:
- Consistent inclusion of information and links about the published versions of the papers.
- More refined options for alerting, both email and RSS. Several respondents specifically requested email alerts for works by a particular author, and there was some interest in HTML-formatted email with live links.
- Updating and keeping current arXiv’s TeX engine and provide TeX templates or style files to make submission easier.
- Linking papers to each other via citations and actionable links in bibliographies.
- Ability to submit a PDF, an increase in the file size limit (often with specific request to link to figures), and the ability to upload multiple files at once.
- Allowing submission directly from authoring platforms (such as Overleaf or Authorea).
- Providing use statistics such as paper downloads and views
- A much larger percentage of recent arXiv users (five years or less) selected the “no opinion” option about current service upgrades. For all the questions in this category, the same trend is visible: a higher percentage of recent users expressed that they had no opinion and this percentage of respondents decreased with each level of increase in years of use. Interestingly, this same trend is not visible by age group; i.e., our data do not show that a higher percent of younger users have no opinion.
Importance of Quality Control Measures
- arXiv’s users were asked a series of questions regarding quality-control measures. Based on the 26,430 responses to specific controls, the most important of these (ranked very important/important) were:
- Check papers for text overlap, i.e., plagiarism 77%
- Make sure submissions are correctly classified 64%
- Reject papers with no scientific value 60%
- Reject papers with self-plagiarism 58%
- A large percentage of all demographic groups found checking for plagiarism to be important and a slightly smaller group found checking for self-plagiarism as important. There was no discernible difference across demographic groups for the other measures. Similarly, self-plagiarism was also mentioned as another area for improvement. Some noted that context is the key; for example, conference papers are a common and typical area where self-plagiarism could occur in an otherwise scientifically sound submission.
- Several respondents said they were unaware of precisely what quality-control measures were already in place, and felt that the process is too opaque. Others acknowledged the difficult balance between rejecting papers that are clearly unworthy—“crackpot”—and rejecting papers for other, perhaps less obvious, and anonymized reasons. However, even in the face of such criticisms there was a strong thread of satisfaction with arXiv’s current quality-control process and users cautioned against going too far in the other direction.
- Some users would prefer that arXiv embrace a more open peer review and/or moderation process, while others were adamant that current controls allow arXiv the freedom and speed of access that is otherwise unobtainable through traditional publishing.
- Overall, the feeling was that quality control matters but user comments varied greatly in relation to how arXiv could practically achieve these goals. As one respondent wrote, “Judgment about quality control is a very relative issue."
...