5
$\begingroup$

The review system is rigged to nudge users into action rather than non-action. This means that when someone opens a review, if they want to review, there is a good chance that they will act on a low quality post, or a late post, and review it.

But since the community is not very large in terms of reviewers (there are only 114 Reviewer badges at the time of this post, and this comes with much repetition, as one can obtain the Reviewer badge up to 6 times), this means that the likelihood of the action being taken by someone who is not an expert is not negligible.

Sure, some stuff is clearly nonsense and should be acted on (e.g. spam, or other clearly not-answers). But there are times that I, for example, would flag something as low quality or not an answer, because I can parse the content of the post and I recognize it to be irrelevant, or sufficiently far from the actual topic. And there is nothing more frustrating to see this flag cleared by someone who has no actual experience in set theory clearing the post as "Looks OK".

Yes, on other websites, the ideal way would be to downvote or leave a comment that this answer is nonsensical in hope that reviewers would take that into heart, or other users would take that into heart. On MathOverflow, I sometimes find that to be unnecessary. Expressing open dissent can lead to unnecessary conflict, which can be easily avoided when the answer is deleted from the review queue or by the moderators. (And history had shown that at least on a few occasions people whose answers were deleted in such way are exactly people that I would not want to interact with.)

Since I doubt we can request that the review system is changed so you cannot review posts in tags where you have no active participation, can we at least interject (using the "extra script privilege") a banner reminding the reviewing users they should skip reviews outside their area of expertise in case the answer is not clear cut?

Clarification

I am not saying that only experts should review questions or answers on their topic of expertise. That would be stupid and counterproductive altogether.

I am saying that people who are not experts should be aware of their non-expertise, and their inability to judge content which might seem correct to them, but is really just a bunch of nonsense, or something that is not worth preserving on the public face of the site.

By not reviewing and skipping, you make the flag reach the moderators, who then can either consult an actual expert—or more likely see that an expert raised the flag, or that it was raised out of pure spite and ignore it—and act accordingly.

$\endgroup$
6
  • 9
    $\begingroup$ Hear, hear. This problem has come up numerous times, on the other side as well where non-expert reviewers also vote to close when they shouldn't, using some superficial criteria. I think it's the latter which bothers me much more, since it discourages new people from wanting to participate. $\endgroup$
    – Todd Trimble Mod
    Dec 12, 2017 at 16:02
  • 3
    $\begingroup$ @Stefan: If I start voting to close arbitrary number theory questions as off topic, would you find that "keeping the community together"? Or would you find that detrimental, since it serves to bring chaos, misinformation, and mystify the correct usage and maybe even the correct mathematical results that can be posted here? $\endgroup$
    – Asaf Karagila Mod
    Dec 12, 2017 at 23:50
  • 3
    $\begingroup$ @Stefan: But that's the point, that I also accept the fact that for the most part, even if something "looks clear", it might be complete nonsense because I know absolutely nothing about research level number theory. I don't understand why you won't extend the same courtesy to other fields. $\endgroup$
    – Asaf Karagila Mod
    Dec 13, 2017 at 0:01
  • 3
    $\begingroup$ To clarify. I'm not saying that "only experts should review stuff". I'm saying that people who are not expert should remember that they are not experts, and therefore might not be qualified to judge the content of a post, even if it doesn't look like immediate crap. And it's better to leave this post for "the next person", until it reaches the moderators which can call upon an actual expert if need be to be the judge of quality (or see who raised the flag, and judge by that). $\endgroup$
    – Asaf Karagila Mod
    Dec 13, 2017 at 0:02
  • 1
    $\begingroup$ +1 Todd. (am too sleep-deprived to offer my own two cents) $\endgroup$
    – Yemon Choi
    Dec 13, 2017 at 0:37
  • 2
    $\begingroup$ Whoever voted to close as unclear, I'd be very happy to clarify, if I understand what is unclear. I'm asking for a banner to be added to the review system to remind people that skipping a question is not a bad idea if the topic of the post is outside their area of expertise. This, I hope, will help to reduce the terrible reviews and closure votes (from the review system, anyway) that people sometimes do unintentionally due to lack of experience. $\endgroup$
    – Asaf Karagila Mod
    Dec 14, 2017 at 11:33

1 Answer 1

Reset to default
-1
$\begingroup$

I think I understand and appreciate the situation described by "inexpert" handling of a question on MathOverflow, and I can see some rationale in trying to avoid this by influencing the review system so that such type of handling is minimized. Let me point out how this could backfire.

The primary case under concern is that a question is posed that needs particular expertise to interpret, form a response, and appreciate (and maybe review and edit). If such expertise is applied, more often than not any response from such expertise will require almost that same expertise to understand and appreciate.

In short, the specialists end up talking to one another and no one else understands or learns. The (relative) onlookers are given the raw material of the question and answers, and if they are lucky possibly a link to introductory material, but additional insight or opportunities to learn are not provided, UNLESS the respondent (or questioner) has a mindset to frame the response to provide those opportunities. Further, the suggestion provided above encourages outsiders to stay outside, and not offer opportunities to make the postings more accessible.

Although this may not be possible with every post, I would encourage people to review with a goal of accessibility, whether they understand the material or not. The goal shouldn't be questions and answers that solve immediate needs, but to provide a repository of short readings where present day readers and readers far into the future can learn more from a well posed question and answer. Let the specialists provide the knowledge, insight, and intuitions, but also let the outsider (including reviewers, so let us train the reviewers differently from what is suggested by Asaf ) provide contributions by (among other things) suggesting how to make the content more accessible.

Indeed, there will be problems with any implementation of the review process. To me that suggests reexamining the goals before changing the process.

Gerhard "Ask Me About Customer Training" Paseman, 2017.12.12.

$\endgroup$
8
  • 2
    $\begingroup$ C'mon, it's not like specialists are the only people who post reasonable (or even good) questions. That's a big fallacy right there. But I certainly don't expect someone who is not a set theorist to write a good answer on a technical and subtle topic. Do you? And since there are still people who do that, it happens that these posts get flagged from time to time as well. So now someone who is not an expert might say "oh, yeah, that looks fine". And you have a crap answer left visible. I don't see how your "backfire" scenario is even close to reality... $\endgroup$
    – Asaf Karagila Mod
    Dec 12, 2017 at 20:51
  • 1
    $\begingroup$ I guess I don't understand the reality of which you speak. Isn't the problematic scenario like the following? Q posts question or answer P on subject S. P gets reviewed by me (I am ignorant of lots of things), and I make the wrong choice: either P is not nonsense and I reject it because I have some reason to think it bad, or P is nonsense but I find it intriguing and encourage it. Someone who knows subject S (but may not have review privileges) can see my mistake clearly. Gerhard "Or Is It Something Else?" Paseman, 2017.12.12. $\endgroup$
    – Gerhard Paseman
    Dec 12, 2017 at 20:58
  • 3
    $\begingroup$ Well, if you are not sure it's garbage, or see that there is some interaction which hints of this being high quality... SKIP! $\endgroup$
    – Asaf Karagila Mod
    Dec 12, 2017 at 20:59
  • 1
    $\begingroup$ Except that is not what you and your solution (as I understand it) are advocating. You are saying (more correctly, I am reading). "Hey, you're not a set theorist, leave the set theory question for someone else to review.". So you are encouraging me to skip the review because set theory is not my specialty. Gerhard "Saying Doesn't Matter. Reading Does" Paseman, 2017.12.12. $\endgroup$
    – Gerhard Paseman
    Dec 12, 2017 at 21:08
  • 1
    $\begingroup$ My solution advocates that there is a banner reminding people that if they are not 100% sure about something which is outside their area of expertise, they should skip. $\endgroup$
    – Asaf Karagila Mod
    Dec 12, 2017 at 21:16
  • 1
    $\begingroup$ If there were enough expertise in the review system to ensure the post does not get skipped by everybody, your solution might help. I see a different reality unfolding. I suspect your solution influences material away from community improvement (as well as community interference). I think a better solution is to train the community to improve more and interfere less by guided action than by guided inaction. However, I am not a moderator or forum designer. Perhaps the moderators and forum designers will adopt your suggestion. Gerhard "Has Had His (Matterless) Say" Paseman, 2017.12.12. $\endgroup$
    – Gerhard Paseman
    Dec 12, 2017 at 21:30
  • $\begingroup$ I found that the moderators on the site handle about 50% of my LQ flags anyway. $\endgroup$
    – Asaf Karagila Mod
    Dec 12, 2017 at 21:31
  • 1
    $\begingroup$ After Stefan Kohl also raised similar concerns, I posted a clarification. $\endgroup$
    – Asaf Karagila Mod
    Dec 13, 2017 at 0:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .