Shaw and Barseghyan (2019)

Shaw, Jamie and Barseghyan, Hakob. (2019) Problems and Prospects with the Scientonomic Workflow. Scientonomy 3, 1-14. Retrieved from https://scientojournal.com/index.php/scientonomy/article/view/33509.

Title	Problems and Prospects with the Scientonomic Workflow
Resource Type	journal article
Author(s)	Hakob Barseghyan, Jamie Shaw
Year	2019
URL	https://scientojournal.com/index.php/scientonomy/article/view/33509
DOI	10.33137/js.v3i0.33509
Journal	Scientonomy
Volume	3
Pages	1-14

Abstract

While the scientonomic workflow guiding the development of a general theory of scientific change has been practiced for nearly four years, it has yet to be formally evaluated. The goal of this paper is to fill this gap with a critical appraisal of the practice and theoretical underpinnings of the workflow currently used in scientonomy. First, we consider the traditional workflow which uses publications as the primary vehicle for substantive epistemic change and find that it fails to be sufficiently transparent or inclusive and is ambiguous at decisive points. Conversely, as we argue, the scientonomic workflow has the potential to succeed where the traditional workflow fails and thus provides a promising alternative workflow. We then go on to note a number of practical and theoretical problems that have arisen upon reflection on the scientonomic workflow and suggest some modifications to the workflow and to our practices. This paper takes the first steps in improving the workflow to reach its maximum potential.

Theories

Here are all the theories formulated in Shaw and Barseghyan (2019):

Theory	Type	Formulation	Formulated In
Closure Mechanism - Acceptance by Default (Shaw-Barseghyan-2019)	Normative	A modification should be accepted by default if there are no objections within a 90-day period following its publication.	2019
Scientonomic Workflow (Barseghyan et al.-2016)	Normative	Scientonomic knowledge is best advanced by: documenting the body of accepted communal knowledge knowledge in an online encyclopedia; scrutinizing this accepted knowledge, identifying its flaws, and formulating open questions at seminars, conferences, publications, and other in-person or online formats; publishing journal articles that propose modifications to our current knowledge and documenting these suggestions; evaluating the suggested modifications with the goal of reaching a communal consensus and changing the respective encyclopedia pages when a verdict is reached.	2016
Publishing Modification Comments (Shaw-Barseghyan-2019)	Normative	The discussions concerning a suggested modification are to be published once a communal verdict is available. The discussions are to be published in the journal as special commentary articles co-authored by all participants of the discussion or in special edited collections.	2019
Goals of Peer Review - Pursuitworthiness (Shaw-Barseghyan-2019)	Normative	The goal of peer reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability.	2019
Allow Modification Reformulations (Shaw-Barseghyan-2019)	Normative	The commentators of suggested modifications are allowed to suggest reformulations of the original formulations in the comments. By default, the new formulation should bear the original author’s name, unless the author decides to give credit to those who significantly contributed to the new reformulation.	2019
Handling Ripple Effects - Editorial House Keeping (Shaw-Barseghyan-2019)	Normative	The encyclopedia editors should be granted official housekeeping rights to handle the ripple effects. If the additional required changes are implicit in the suggested modification, the editors should create and alter encyclopedia pages to ensure that the accepted body of scientonomic knowledge is properly documented; if it is conceivable to accept the modification without accepting the ripple effect change in question, the editors should register these changes as new suggested modifications so that the community can discuss and evaluate them in an orderly fashion.	2019
Closure Mechanism - Time Limit and Communal Vote (Shaw-Barseghyan-2019)	Normative	The verdict on a suggested modification should be decided by a communal vote that will follow the discussion period.	2019

Suggested Modifications

Here are all the modifications suggested in Shaw and Barseghyan (2019):

• Sciento-2019-0001: Accept that the goal of peer-reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0002: Accept that the discussions concerning a suggested modification are to be published once a communal verdict is available. The discussions are to be published in the journal as special commentary articles co-authored by all participants of the discussion or in special edited collections. The modification was suggested to Scientonomy community by Hakob Barseghyan, Jamie Shaw and Karen Yan on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0003: Accept that the commentators of suggested modifications are allowed to suggest reformulations of the original formulations. Also accept that, by default, the new formulation should bear the original author’s name, unless the author decides to give credit to those who significantly contributed to the new reformulation. This should be decided collegially by the author, the commentators, and the editors on a case-by-case basis. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0004: Accept that an annual book prize is to be offered for extensive participation on the encyclopedia. The winner(s) are to be decided by the encyclopedia editors. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0005: Accept that star-ratings are to be introduced for commentators who comment on suggested modifications on the encyclopedia. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0006: Accept that the encyclopedia editors are to be granted official housekeeping rights to handle the ripple effects. Also accept that if the additional required changes are implicit in the suggested modification, the editors should create and alter encyclopedia pages to ensure that the accepted body of scientonomic knowledge is properly documented; if it is conceivable to accept the modification without accepting the ripple effect change in question, the editors should register these changes as new suggested modifications so that the community can discuss and evaluate them in an orderly fashion. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0007: Accept that the verdict on suggested modifications is to be decided by a communal vote that will follow the discussion period. Have a communal discussion and decide as to what percentage of votes it should take for a modification to be accepted - a simple majority (50% +1), or supermajority of three fifths (60%), two thirds (67%), or three quarters (75%). Also discuss to decide as to how long the discussion period and the voting period should be. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
• Sciento-2019-0008: Accept that a countdown mechanism is to be introduced, where a modification is accepted by default if there are no objections within a 90-day period following its publication. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The discussion was closed on 18 October 2022 and the modification was not accepted. It has been agreed that the idea of accepting a modification by default after a fixed time period might have several negative consequences. First, it may lead to the automatic acceptance of an otherwise unacceptable modification that just happened to be suggested at a time when most researchers interested in the topic were exceptionally busy.^c1 It was emphasized that if we were to allow for modifications to become accepted simply "because no one said anything" we would be giving "undue power to the mechanism of what gets accepted".^c2 This might "allow some modifications to garner more discussion than others depending on when they are published and lead to an incorrect understanding of the Scientonomic community’s evaluation of a particular modification", so we might end up with a mosaic that is not representative of the communal views.^c3 It was also agreed that acceptance by default fails to address the concern that some members of the community may be reluctant to object to a modification for a variety of reasons. It is unlikely that “having time limits, even if they are apparent and made known within our community, will incentivize explicit objection”.^c4 It was suggested that "researchers may be even more reluctant to “impede the modification’s acceptance” now that it would be an automatic process”.^c5 Finally, it was mentioned that "the implementation of this modification may result in yet another unwanted consequence: some researchers may end up submitting a negative comment simply for the sake of preventing the automatic acceptance of the modification and stopping the countdown".^c6