-6
$\begingroup$

Since some time there has been discussions about having a place for community reviews; see for example

Community Peer-Review in Mathoverflow

Such a system already working at PhysicsOverflow with considerable success.

My attempt to start something similar in MO was quickly rejected for reasons that I well understand: MO is a place for questions and not for reviews. Therefore I intend to open an area 51 site.

I would keep it simple: Users (anyone, not only authors) can post links to freely available preprints together with some info (authors and abstract). The only acceptable answers would be classical journal-type reviews (not mathscinet type reviews). Comments should only address points that need further clarification. The site should not a place for discussing the paper's impact. Anonymous reviews are of course accepted. The flagging and point system should work well enough for "accepting" or "rejecting" preprints/reviews. Sectioning could be done via AMS subject classification.

But for all these I need at least 5 users to support the site and 5 example questions. I am willing to put my 3 current unpublished preprints as questions.

It would be great to have a discussion, not just upvotes/downvotes. If you support this, please let me know. If you think that such an effort is wrong, please state your argument.

EDIT1: It could be considerable having a list of members/moderators acting as "editors", that is, they would receive preprints from authors that are not members of the site, post their preprints and recruit reviewers. I am willing to act as an editor for pdes and calculus of variations.

EDIT2: Following Gerhard Paseman's suggestion, it makes sense to ask those that want to have a paper reviewed, to provide with at least 3 classes for which they are willing to act as possible reviewers. I am willing to review for: 35A01/2 (PDE - Existence/Uniqueness problems) 35A15 (PDE - Variational methods) 35Bxx (PDE - Qualitative properties of solutions) 35Gxx (PDE - General higher-order equations and systems) 49Jxx (Calculus of variations - Existence theories)

EDIT3: In the review section of PO (3) The review section, a mechanism is described how manuscripts, may be accepted for review. We should also do something similar here in order to avoid submissions that contain work-in-progress whose authors expect to get community advice. For example let the question reach 10 upvotes (including 1 from an subject editor) before it is eligible for review.

EDIT4: Just to be clear, I am willing to write down the first 5 reviews needed to keep things going.

$\endgroup$

closed as off-topic by Stefan Kohl, Joonas Ilmavirta, Lucia, András Bátkai, Jeremy Rickard Aug 14 at 13:27

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to be about MathOverflow or the software that powers the Stack Exchange network within the scope defined in the help center." – Stefan Kohl, Joonas Ilmavirta, Lucia, András Bátkai, Jeremy Rickard
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ @MartinSleziak This proposal was for finding errors which I understand the many reasons why it failed. It was meant that people post mistakes. In this site people would post preprints. They maybe excellent, good, mediocre, flawed or rubbish. I consider this as genuinely different. The reviews section on PO is working: they have 165 requests, around 1 per month and almost every second request has a review. $\endgroup$ – Thanasis Stylianou Aug 13 at 14:35
  • 1
    $\begingroup$ It might be better to call it public review rather than peer review. I imagine most of the participants (who will have something useful and positive to contribute anyway, but) will not be sufficiently expert on the subject matter of a paper to be considered a peer reviewer for that paper. Indeed, I am concerned that papers written for specialized audiences would only fare well under this system when the reviewer population has a significant percentage of the specialized audience. Gerhard "Considers Himself Really Special Audience" Paseman, 2019.08.13. $\endgroup$ – Gerhard Paseman Aug 13 at 15:31
  • 1
    $\begingroup$ You might get a better foothold if you recruit participants by topic. Suggest that anyone who wants a preprint (of their own or authored by someone giving permission) reviewed state two or three topics that the person is willing to review for. When a topic gets three or more willing reviewers, then papers on that topic are accepted for purposes of public review. As a fake example (I'm not ready to commit at this time), I might say I am willing to review papers in algorithmic number theory, or in distribution of smooth numbers. Gerhard "Should Review My Own Suggestions" Paseman, 2019.08.13. $\endgroup$ – Gerhard Paseman Aug 13 at 15:39
  • 3
    $\begingroup$ I'm not convinced that the SE-style QA system is a good platform for that. We're supposed to have clear and specific questions, and "please review this whole article" is not one. Having such a service might be good indeed, but I doubt this is the way to go. For a concrete example, I'd much rather try to get Peerage of Science to expand to mathematics. $\endgroup$ – Joonas Ilmavirta Aug 13 at 19:23
  • 1
    $\begingroup$ @JoonasIlmavirta This looks quite good actually. However there are two problems. First, there is no organized community supporting such a system: the success of MO (and other such sites) lies as far as I understand on the "size of the community" - either too large or too small and the community will eventually die out. On the other hand, I am currently comparing community peer reviewing to open source apps. There anyone (a user, part of the community, not only confirmed programmers) is able to submit a code snippet which is digested by the community. Open source has won the battle. $\endgroup$ – Thanasis Stylianou Aug 13 at 19:41
  • 2
    $\begingroup$ "either too large or too small and the community will eventually die out." Die out from being too large? I'm reminded of Yogi Berra's remark aboout a certain restaurant, "Nobody goes there anymore. It's too crowded." $\endgroup$ – Gerry Myerson Aug 13 at 23:27
  • 1
    $\begingroup$ @GerryMyerson :-) I think that was the message of the merging section of the area 51 blog: stackoverflow.blog/2010/09/17/merging-season $\endgroup$ – Thanasis Stylianou Aug 14 at 8:17
  • 1
    $\begingroup$ @NoahSnyder There are many differences: posted manuscripts will not have been published anywhere - MR does this for published articles which have already been peer reviewed. The reviews on MR are more of an extended abstract style. This is not the goal here: the idea is to let the community do the peer reviewing itself and make the results public. MR does not allow anonymous reviews, the reviews and the articles do not get any up/downvotes. The community has of a completely different character. MR is not run by the community itself - a few editors handle all the work. $\endgroup$ – Thanasis Stylianou Aug 14 at 8:23
  • 1
    $\begingroup$ @GerhardPaseman Concerning your remark on specialized audience, this is where the moderators/editors should contribute, namely in recruiting reviewers, as it is the case in standard journals. $\endgroup$ – Thanasis Stylianou Aug 14 at 11:50
  • 1
    $\begingroup$ Why is this question considered off-topic? For an area 51 project one needs to find stackexchange users and a site who will support the project. How is this possible without posting to meta? $\endgroup$ – Thanasis Stylianou Aug 14 at 13:34
  • 3
    $\begingroup$ I think this is a reasonable meta post (given earlier one time requests from other proposals) and will vote to reopen. Again I am watching the situation some before committing much time to it. Gerhard "Wonders About Automating Scientific Review" Paseman, 2019.08.14. $\endgroup$ – Gerhard Paseman Aug 14 at 13:50
  • 2
    $\begingroup$ @ThanasisStylianou I am not sure how familiar you are with the area51 process, but only example questions are posted there (no answers). If I understand your proposal correctly, the reviews would be posted as answers. So there aren't "5 reviews needed to keep things going." The reviews (or answers) will be needed only after the site gets from area51 into beta phase. $\endgroup$ – Martin Sleziak Aug 14 at 14:27
  • 2
    $\begingroup$ I have created the proposal....let's see what happens! meta.mathoverflow.net/q/4321/40644 $\endgroup$ – Thanasis Stylianou Aug 14 at 15:05
  • 1
    $\begingroup$ By my count 35 of the 165 requests on PO currently have a review, for a 21% success rate. A total of 39 reviews have been submitted (4 articles have 2 reviews), for a rate of about 1 review every 50 days. Add in the fact that the 6 most recent reviews are by the same user, and it seems like PO's review system is barely working at all, let alone "working ... with considerable success". $\endgroup$ – the thesaurus saurus Aug 15 at 8:27
  • 1
    $\begingroup$ @thethesaurussaurus it is a bit unfair to look at it like that. In the last year the have 1 submission per month and every second one has a review. Also they accepted all subjects, so a level of dispersion is expected. The system is active since 2015, I consider this a success. $\endgroup$ – Thanasis Stylianou Aug 15 at 8:40

Browse other questions tagged .