Workflow - Closure Mechanism

How should verdicts on suggested modifications be achieved? If modifications are accepted as a result of a communal consensus, then what constitutes such a consensus?

To ensure the advancement of knowledge, an academic workflow requires some closure mechanism, i.e. some idea as to what counts as a communal consensus and when and how it is to be achieved. This is far from obvious, as different workflows often have different procedures for consensus formation. In some workflows, the closure happens with the publication of the paper, while in others it happens long after the initial publication, if at all. In the latter case, workflows can differ in their understanding of what counts as a consensus (e.g. unanimity, lack of explicit objection, majority opinion, etc.). "While views differ as to how closure should proceed, they agree that we need some way to close debates or discussions such that they do not extend indefinitely."1 Thus, it is important for the scientonomic workflow to have a proper closure mechanism in place to ensure that suggested modifications are evaluated and verdicts are reached communally in an inclusive and transparent fashion.

In the scientonomic context, this question was first formulated by Hakob Barseghyan and Jamie Shaw in 2019. The question is currently accepted as a legitimate topic for discussion by Scientonomy community.

In Scientonomy, the accepted answer to the question is:

• Scientonomic knowledge is best advanced by:

1. documenting the body of accepted communal knowledge knowledge in an online encyclopedia;
2. scrutinizing this accepted knowledge, identifying its flaws, and formulating open questions at seminars, conferences, publications, and other in-person or online formats;
3. publishing journal articles that propose modifications to our current knowledge and documenting these suggestions;
4. evaluating the suggested modifications with the goal of reaching a communal consensus and changing the respective encyclopedia pages when a verdict is reached.

Scientonomic History

Acceptance Record

Here is the complete acceptance record of this question (it includes all the instances when the question was accepted as a legitimate topic for discussion by a community):

Community	Accepted From	Acceptance Indicators	Still Accepted	Accepted Until	Rejection Indicators
Scientonomy	22 December 2019	The publication of Shaw and Barseghyan (2019) is and indication of the acceptance of the question.	Yes

All Theories

The following theories have attempted to answer this question:

Theory	Formulation	Formulated In
Closure Mechanism - Time Limit and Communal Vote (Shaw-Barseghyan-2019)	The verdict on a suggested modification should be decided by a communal vote that will follow the discussion period.	2019
Closure Mechanism - Acceptance by Default (Shaw-Barseghyan-2019)	A modification should be accepted by default if there are no objections within a 90-day period following its publication.	2019

If an answer to this question is missing, please click here to add it.

Accepted Theories

According to our records, no theory on this topic has ever been accepted.

Suggested Modifications

Here is a list of modifications concerning this topic:

Modification	Community	Date Suggested	Summary	Verdict	Verdict Rationale	Date Assessed
Sciento-2019-0007	Scientonomy	22 December 2019	Accept that the verdict on suggested modifications is to be decided by a communal vote that will follow the discussion period. Have a communal discussion and decide as to what percentage of votes it should take for a modification to be accepted - a simple majority (50% +1), or supermajority of three fifths (60%), two thirds (67%), or three quarters (75%). Also discuss to decide as to how long the discussion period and the voting period should be.	Accepted	Prior to the 2023 workshop, Ameer Sarwar argued against the modification. First, he noted that voting is not an appropriate mechanism in science where the goal is to unearth truth.c1 Second, it is unclear how we can ensure informed voting given that some members of the community could be inactive for several years. He thus suggested that we should keep this modification open and wait until after the resolution of modifications 2019-0002 and 2019-0003 that suggest alternative ways to increase participation. During the workshop, the modification was generally well received. Before voting, there was some concern about our voting process – who can vote and when can they vote? – voiced by Josh Allen and Paul Patton. Additionally, Deivide Garcia and Amirali Atrli raised concerns about who are “allowed” to function as part of the scientonomy community. Patton also suggested introducing quorum in addition to the 2/3rds stipulation to avoid potential modifications to the scientonomic body of knowledge introduced by a small number of participants. Gregory Rupik along with Jamie Shaw indicated that while quorum makes perfect sense for larger decision-making bodies, our capacity to vote should not hinge on who is absent, but rather on who is present. It was also determined that even though there is always a risk of a small group of people making big changes, or with people being unsatisfied with a modification they were not allowed to vote on, the iterative nature of our process ensures that there are easy solutions here; in addition, as Spenser summarized, most people in academic environments can be trusted to self-police. Notably, since this proposal represents an attempt to formalize a voting system and closure mechanism, and itself was not subject to a specific voting system, it was decided by those present that we would accept this modification with a minimum of 2/3rds assent. The modification was accepted with overwhelming support.	25 February 2023
Sciento-2019-0008	Scientonomy	22 December 2019	Accept that a countdown mechanism is to be introduced, where a modification is accepted by default if there are no objections within a 90-day period following its publication.	Not Accepted	It has been agreed that the idea of accepting a modification by default after a fixed time period might have several negative consequences. First, it may lead to the automatic acceptance of an otherwise unacceptable modification that just happened to be suggested at a time when most researchers interested in the topic were exceptionally busy.c1 It was emphasized that if we were to allow for modifications to become accepted simply "because no one said anything" we would be giving "undue power to the mechanism of what gets accepted".c2 This might "allow some modifications to garner more discussion than others depending on when they are published and lead to an incorrect understanding of the Scientonomic community’s evaluation of a particular modification", so we might end up with a mosaic that is not representative of the communal views.c3 It was also agreed that acceptance by default fails to address the concern that some members of the community may be reluctant to object to a modification for a variety of reasons. It is unlikely that “having time limits, even if they are apparent and made known within our community, will incentivize explicit objection”.c4 It was suggested that "researchers may be even more reluctant to “impede the modification’s acceptance” now that it would be an automatic process”.c5 Finally, it was mentioned that "the implementation of this modification may result in yet another unwanted consequence: some researchers may end up submitting a negative comment simply for the sake of preventing the automatic acceptance of the modification and stopping the countdown".c6	18 October 2022

Current View

In Scientonomy, the accepted answer to the question is Scientonomic Workflow (Barseghyan et al.-2016).

Scientonomic Workflow

Scientonomic Workflow (Barseghyan et al.-2016) states: "Scientonomic knowledge is best advanced by:

1. documenting the body of accepted communal knowledge knowledge in an online encyclopedia;
2. scrutinizing this accepted knowledge, identifying its flaws, and formulating open questions at seminars, conferences, publications, and other in-person or online formats;
3. publishing journal articles that propose modifications to our current knowledge and documenting these suggestions;
4. evaluating the suggested modifications with the goal of reaching a communal consensus and changing the respective encyclopedia pages when a verdict is reached."
   
   The key stages of the workflow are:

• Pose Questions: The goal of this stage is to scrutinize the current state of the scientonomic theory and our knowledge of scientific change and identify as many open questions as possible. The annual seminar on scientonomy hosted by the University of Toronto's Institute for the History and Philosophy of Science and Technology is currently the main venue facilitating this stage of the workflow.
• Suggest Modifications: The goal of this stage is to advance our knowledge of scientific change by proposing modifications to our current body of knowledge. These suggested modifications are published and properly documented. These modifications are currently published in the Journal of Scientonomy but, in principle, they can be published in any journal which makes use of the scientonomic mechanism of modifications. Once a modification is published, this encyclopedia documents that suggestions and invites experts to review it.
• Evaluate Modifications: The goal of this stage is to assess the suggested modifications and decide which of them are acceptable and which are not. This is done by the community of scientonomists on the respective discussion pages of this encyclopedia. If a consensus emerges, the fate of the modification is documented. If a modification causes disagreement among scientonomists, it becomes a topic of discussion during scientonomic workshops, which aim at bridging the gaps between opposing parties and arriving at consensus.
• Document Changes: The goal of this stage is to document all the changes in our communal body of knowledge. If a modification is considered acceptable by the community, then the respective articles of this encyclopedia are modified to reflect that change. If a modification is considered unacceptable, then the respective verdict is documented for that modification.

The primary role of this encyclopedia in the scientonomic workflow is to document the current state of scientonomic knowledge, trace all suggested modifications, and list open questions.

Here is an outline of the main stages of the scientonomic workflow:

This workflow gives researchers a simple way of knowing where the community stands on different topics, i.e. what theories it currently accepts, what open questions it tries to answer, what modifications have been proposed and how they have been assessed. It ensures that our communal knowledge is advanced in a piecemeal and transparent fashion:

• Piecemeal: modifications to the communal mosaic are suggested one by one, which allows for a sober critical evaluation of these suggestions by the community.
• Transparent : suggested modifications and their evaluations are properly documented, so that there is no mystery as to whether, when, or why a certain modification was or wasn't accepted.

The workflow is scalable, as it can - in principle - be implemented in other fields of digital humanities and beyond.

Related Topics

This question is a subquestion of Scientonomic Workflow.