Shaw and Barseghyan (2019)
Shaw, Jamie and Barseghyan, Hakob. (2019) Problems and Prospects with the Scientonomic Workflow. Scientonomy 3, 1-14. Retrieved from https://scientojournal.com/index.php/scientonomy/article/view/33509.
Title | Problems and Prospects with the Scientonomic Workflow |
---|---|
Resource Type | journal article |
Author(s) | Hakob Barseghyan, Jamie Shaw |
Year | 2019 |
URL | https://scientojournal.com/index.php/scientonomy/article/view/33509 |
DOI | 10.33137/js.v3i0.33509 |
Journal | Scientonomy |
Volume | 3 |
Pages | 1-14 |
Abstract
While the scientonomic workflow guiding the development of a general theory of scientific change has been practiced for nearly four years, it has yet to be formally evaluated. The goal of this paper is to fill this gap with a critical appraisal of the practice and theoretical underpinnings of the workflow currently used in scientonomy. First, we consider the traditional workflow which uses publications as the primary vehicle for substantive epistemic change and find that it fails to be sufficiently transparent or inclusive and is ambiguous at decisive points. Conversely, as we argue, the scientonomic workflow has the potential to succeed where the traditional workflow fails and thus provides a promising alternative workflow. We then go on to note a number of practical and theoretical problems that have arisen upon reflection on the scientonomic workflow and suggest some modifications to the workflow and to our practices. This paper takes the first steps in improving the workflow to reach its maximum potential.
Theories
Here are all the theories formulated in Shaw and Barseghyan (2019):
Theory | Type | Formulation | Formulated In |
---|---|---|---|
Closure Mechanism - Acceptance by Default (Shaw-Barseghyan-2019) | Normative | A modification should be accepted by default if there are no objections within a 90-day period following its publication. | 2019 |
Scientonomic Workflow (Barseghyan et al.-2016) | Normative | Scientonomic knowledge is best advanced by:
| 2016 |
Publishing Modification Comments (Shaw-Barseghyan-2019) | Normative | The discussions concerning a suggested modification are to be published once a communal verdict is available. The discussions are to be published in the journal as special commentary articles co-authored by all participants of the discussion or in special edited collections. | 2019 |
Goals of Peer Review - Pursuitworthiness (Shaw-Barseghyan-2019) | Normative | The goal of peer reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability. | 2019 |
Allow Modification Reformulations (Shaw-Barseghyan-2019) | Normative | The commentators of suggested modifications are allowed to suggest reformulations of the original formulations in the comments. By default, the new formulation should bear the original author’s name, unless the author decides to give credit to those who significantly contributed to the new reformulation. | 2019 |
Handling Ripple Effects - Editorial House Keeping (Shaw-Barseghyan-2019) | Normative | The encyclopedia editors should be granted official housekeeping rights to handle the ripple effects. If the additional required changes are implicit in the suggested modification, the editors should create and alter encyclopedia pages to ensure that the accepted body of scientonomic knowledge is properly documented; if it is conceivable to accept the modification without accepting the ripple effect change in question, the editors should register these changes as new suggested modifications so that the community can discuss and evaluate them in an orderly fashion. | 2019 |
Closure Mechanism - Time Limit and Communal Vote (Shaw-Barseghyan-2019) | Normative | The verdict on a suggested modification should be decided by a communal vote that will follow the discussion period. | 2019 |
Suggested Modifications
Here are all the modifications suggested in Shaw and Barseghyan (2019):
- Sciento-2019-0001: Accept that the goal of peer-reviews in the scientonomic workflow is evaluation for pursuitworthiness rather than acceptability. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0002: Accept that the discussions concerning a suggested modification are to be published once a communal verdict is available. The discussions are to be published in the journal as special commentary articles co-authored by all participants of the discussion or in special edited collections. The modification was suggested to Scientonomy community by Hakob Barseghyan, Jamie Shaw and Karen Yan on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0003: Accept that the commentators of suggested modifications are allowed to suggest reformulations of the original formulations. Also accept that, by default, the new formulation should bear the original author’s name, unless the author decides to give credit to those who significantly contributed to the new reformulation. This should be decided collegially by the author, the commentators, and the editors on a case-by-case basis. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0004: Accept that an annual book prize is to be offered for extensive participation on the encyclopedia. The winner(s) are to be decided by the encyclopedia editors. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0005: Accept that star-ratings are to be introduced for commentators who comment on suggested modifications on the encyclopedia. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0006: Accept that the encyclopedia editors are to be granted official housekeeping rights to handle the ripple effects. Also accept that if the additional required changes are implicit in the suggested modification, the editors should create and alter encyclopedia pages to ensure that the accepted body of scientonomic knowledge is properly documented; if it is conceivable to accept the modification without accepting the ripple effect change in question, the editors should register these changes as new suggested modifications so that the community can discuss and evaluate them in an orderly fashion. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0007: Accept that the verdict on suggested modifications is to be decided by a communal vote that will follow the discussion period. Have a communal discussion and decide as to what percentage of votes it should take for a modification to be accepted - a simple majority (50% +1), or supermajority of three fifths (60%), two thirds (67%), or three quarters (75%). Also discuss to decide as to how long the discussion period and the voting period should be. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The modification is currently being evaluated; a verdict is pending.
- Sciento-2019-0008: Accept that a countdown mechanism is to be introduced, where a modification is accepted by default if there are no objections within a 90-day period following its publication. The modification was suggested to Scientonomy community by Hakob Barseghyan and Jamie Shaw on 22 December 2019.1 The discussion was closed on 18 October 2022 and the modification was not accepted. It has been agreed that the idea of accepting a modification by default after a fixed time period might have several negative consequences. First, it may lead to the automatic acceptance of an otherwise unacceptable modification that just happened to be suggested at a time when most researchers interested in the topic were exceptionally busy.c1 It was emphasized that if we were to allow for modifications to become accepted simply "because no one said anything" we would be giving "undue power to the mechanism of what gets accepted".c2 This might "allow some modifications to garner more discussion than others depending on when they are published and lead to an incorrect understanding of the Scientonomic community’s evaluation of a particular modification", so we might end up with a mosaic that is not representative of the communal views.c3 It was also agreed that acceptance by default fails to address the concern that some members of the community may be reluctant to object to a modification for a variety of reasons. It is unlikely that “having time limits, even if they are apparent and made known within our community, will incentivize explicit objection”.c4 It was suggested that "researchers may be even more reluctant to “impede the modification’s acceptance” now that it would be an automatic process”.c5 Finally, it was mentioned that "the implementation of this modification may result in yet another unwanted consequence: some researchers may end up submitting a negative comment simply for the sake of preventing the automatic acceptance of the modification and stopping the countdown".c6
References
- a b c d e f g h Shaw, Jamie and Barseghyan, Hakob. (2019) Problems and Prospects with the Scientonomic Workflow. Scientonomy 3, 1-14. Retrieved from https://scientojournal.com/index.php/scientonomy/article/view/33509.