Workflow - Goals of Peer Review

Should peer reviewers evaluate a submitted paper for the pursuitworthiness or acceptability of the content of the paper?

Ideally, a workflow needs to clearly articulate the goals of its peer review process. In the traditional workflow, the answer to this question is far from obvious. While some reviewers review submissions for pursuitworthiness, others review for acceptability, yet others do both.1 As a result, it is often unclear whether the content of published articles is to be taken as accepted by the respective community or as or merely as considered pursuitworthy by the editors and reviewers? As noted by Shaw and Barseghyan,

Many of our practices suggest that it might be the former. For example, in Naomi Oreskes’ widely cited study on the consensus on climate change, she uses the content found in publications as a measure of acceptance.2 Moreover, many reviews reject papers due to their purported flaws suggesting that they should not be published because they are not acceptable.3 Yet, at other times, the fact that something was published is only taken to mean that it was considered to be worthy of further attention. Philosophers, for example, will be acutely familiar with this view: no one reads the newest paper on realism in Philosophy of Science to see what the community believes about realism. They merely search for stimulating and interesting ideas.4p. 3

Thus, it is vital to clearly state the goals of the peer review process.

In the scientonomic context, this question was first formulated by Hakob Barseghyan and Jamie Shaw in 2019. The question is currently accepted as a legitimate topic for discussion by Scientonomy community.

In Scientonomy, the accepted answer to the question is:

• The goal of peer reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability.

Scientonomic History

Acceptance Record of the Question

Here is the complete acceptance record of this question (it includes all the instances when the question was accepted as a legitimate topic for discussion by a community):

Community	Accepted From	Acceptance Indicators	Still Accepted	Accepted Until	Rejection Indicators
Scientonomy	22 December 2019	The publication of Shaw and Barseghyan (2019) is and indication of the acceptance of the question.	Yes

All Direct Answers

The following direct answers to the question have been suggested:

Theory	Formulation	Formulated In
Goals of Peer Review - Pursuitworthiness (Shaw-Barseghyan-2019)	The goal of peer reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability.	2019

If a direct answer to this question is missing, please click here to add it.

Accepted Direct Answers

The following theories have been accepted as direct answers to this question:

Community	Theory	Formulation	Accepted From	Accepted Until
Scientonomy	Goals of Peer Review - Pursuitworthiness (Shaw-Barseghyan-2019)	The goal of peer reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability.	25 February 2023

Suggested Modifications

Here is a list of modifications concerning direct answers to this question:

Modification	Community	Date Suggested	Summary	Date Assessed	Verdict	Verdict Rationale
Sciento-2019-0001	Scientonomy	22 December 2019	Accept that the goal of peer-reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability.	25 February 2023	Accepted	The decision was made during the 2023 scientonomy workshop. The modification was summarized by Paul Patton as essentially a ratification of current scientonomic practice. Jamie Shaw raised some concerns about how we don’t have adequately defined norms that must be satisfied for pursuitworthiness, which may make this modification trivial. Discussion about how peer-reviewers’ notions of pursuitworthiness may veer close to acceptability ensued. Nevertheless, the modification passed with 83% of the votes to accept (10/12).

Current View

In Scientonomy, the accepted answer to the question is Goals of Peer Review - Pursuitworthiness (Shaw-Barseghyan-2019).

Goals of Peer Review - Pursuitworthiness (Shaw-Barseghyan-2019) states: "The goal of peer reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability."

In the scientonomic workflow, the goals of peer review are to assesses a paper for pursuitworthiness of the modifications suggested in the paper. Thus, peer reviewers should not evaluate submissions for acceptability, but only for pursuitworthiness.

Related Topics

This question is a subquestion of Scientonomic Workflow.