Accept that star-ratings are to be introduced for commentators who comment on suggested modifications on the encyclopedia.
In the present-day academic climate of "publish-or-perish", academics are required to publish and present at prestigious venues. In contrast, spending time and effort commenting on the encyclopedia pages does little to further one's academic career. Thus, to address the problem of the lack of commenting, it is necessary to restructure the incentive structures.
One possible way to incentivize commenting is to introduce star-ratings. Studies suggest that when commenting helps people achieve some kind of a status as a result, the people are strongly encouraged to comment more. In addition, a 5 Gold Star rating may provide a CV line. While this suggestion may seem far-fetched, it is still think worth discussing.
Note that this suggestion requires no formal changes to the workflow, but requires changes to our practices.
The discussion was closed on 25 February 2023 and the modification was not accepted. At the 2023 workshop, this modification was met with concerns from the community. Firstly, the question of the equitability of star ratings was highlighted by Rebecca Muscant and Izzy Friesen. Amirali Atrli also wondered whether the problem of incentives becomes further stretched out by the five-star scale. Kye Palider noted that the up-or-down arrow that we already have on our Wiki seems simpler and more democratic than star-ranking. Alessandra Castino also mentioned that on online forums, the basis of their rating systems can discourage new commenters, and that we might see this here too. Some rating systems are also better than others – it was suggested by Joshua Allen, for example, that StackExchange’s system could provide a good model, as it incentivizes participation. Friesen highlighted that larger forums where rating systems for commentary are popular include anonymity by default, which makes this kind of recognition important, but this is not so much of an issue in the scientonomy community. Palider suggested simply listing a user’s number of comments but, as Castino emphasized, such ratings might not reflect the quality of the comments. Jamie Shaw noted that this need not necessarily be an issue since all participation grades are almost inevitably subjective and don’t necessarily reflect the quality. Deivide Garcia suggested that more thoughts needs to be put into this before any of the suggestions could be implemented. Barseghyan agreed and noted that the mediawiki platform has limitations on what can be done here, so it is unclear which of the new suggestions could be possibly implemented. Ultimately, though, the modification was rejected.
Click on the Discussion tab for comments.
- Shaw, Jamie and Barseghyan, Hakob. (2019) Problems and Prospects with the Scientonomic Workflow. Scientonomy 3, 1-14. Retrieved from https://scientojournal.com/index.php/scientonomy/article/view/33509.