9
$\begingroup$

Is there data available (besides crawling thousands of questions) on how often we use the downvote button, and how quickly?

The motivation for this question is Carlo Beenakker's recent answer on MOM.

I agree instinctually that downvoting is close to the soul of the 'unwelcomeness' spoken about regarding MO, but I'm curious if these instincts are borne out in the data.

Ideally the data set would list downvotes 'per-capita', meaning that sites with higher traffic wouldn't be exaggerated in standing by the naturally higher number of aggregate downvotes, but aggregate data together with traffic data could obviously be used to reconstruct a per-capita view of things.

$\endgroup$
17
  • 20
    $\begingroup$ I'm a bit skeptical about the premise of this question. Just based on my own observations, I would guess that by far most down votes are cast on questions clearly outside of the scope of MO (e.g. undergraduate homework exercises from textbooks), and these actions have little to do with whether MO is perceived as welcoming to the kind of mathematicians we would want to engage with the site but are currently off-put by its culture. $\endgroup$
    – Sam Hopkins
    Commented Jan 19, 2021 at 16:31
  • 10
    $\begingroup$ I more or less agree. My impression is that MO is perceived as "un-welcoming" mainly by some people whose experience is mostly on social media like Facebook, Reddit or Twitter, where the platform permits the creation of human relationships. I am afraid that MO platform is really too limited for this, and that there is no obvious way to wipe out the cultural divide. $\endgroup$
    – Francesco Polizzi
    Commented Jan 19, 2021 at 18:37
  • 3
    $\begingroup$ @SamHopkins I’m not sure what the ‘premise of my question is’ besides a request for data; if you mean that you disagree with my instincts, that is fine. You seem to work in fields that are widely respected by mathematicians at large, combinatorics/representation theory/algebraic geometry. As someone who posts primarily category theory stuff (lately), I’ve had downvotes almost instantly on questions that ended up more than 5 positive once seen by a larger crowd. I actually inquired about this on meta, and was told by a mod that there are suspected serial downvoters for certain topics. $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 20:27
  • 1
    $\begingroup$ @FrancescoPolizzi My two favorite undergraduate professors, an algebraist and analyst by trade, both expressed a similar sentiment to me that MO was unwelcoming and I should be careful coming here; the algebraist was younger, but the analyst was in his 60's. I hope they don't fit your description of "Facebook, Reddit or Twitter" people that are trying to use MO wrong. $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 20:30
  • 3
    $\begingroup$ @AlecRhea: Sorry, I didn't mean to attribute any viewpoints to you that you did not express. What I meant was I'm not sure numbers concerning downvotes overall tell us a lot, because of how these votes are mostly used to do basic quality control. But I do believe that aggressive downvoting against questions at the e.g. graduate level posed in good faith is potentially an important contributing factor to the perceived unwelcoming nature of MO and that's worth discussing. I now realize that comments under the answers also discussed whether the overall downvote numbers are telling us much. $\endgroup$
    – Sam Hopkins
    Commented Jan 19, 2021 at 20:33
  • 1
    $\begingroup$ @SamHopkins No worries, just a miscommunication. Yes, I agree that finding out which metrics to look at is tricky business and we might not be able to glean any hard answers, but I'm just trying to find some objective metric of this perceived unfriendliness as a starting point for a larger discussion that is hopefully more solid than 'a lot of people have been saying lately'. $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 20:35
  • $\begingroup$ @AlecRhea: I do not know. In my experience, I certainly saw people complain about the "un-friedliness" or the "lack of empathy" on MO. I have the impression that this is not really related (only) to the downvotes, but rather to the structure of the QA-platform, much less suitable than others to generate human warmth. As a matter of fact, I have also in mind people complaining after their question was well-received, because they did not like the tone of the answer (that was polite, but cold, so it could appear as dismissive). $\endgroup$
    – Francesco Polizzi
    Commented Jan 19, 2021 at 21:23
  • $\begingroup$ From one of your comments it seems that you might be also interested in differences in down-voting between various tags. I'd guess that various data related to this could be obtained from SEDE, too. I will at least point out that there is this question on Meta Stack Exchange: Which tag has the highest incidence of downvoting? Glorfindel posted there an answer with a query which looks at various tag and the percentage of questions which have at least one downvote: data.stackexchange.com/mathoverflow/query/828585/… $\endgroup$
    – Martin Sleziak
    Commented Jan 19, 2021 at 21:47
  • 2
    $\begingroup$ " the platform permits the creation of human relationships. I am afraid that MO platform is really too limited for this" <-- lol, what? Also, the comment generally displays a sample bias. $\endgroup$
    – David Roberts Mod
    Commented Jan 19, 2021 at 23:54
  • 3
    $\begingroup$ Here's something I have observed more recently: it used to be that an obviously off-topic, but mathematically-sensible question (say an undergrad question), would get a single downvote. This I believe has the effect of keeping it off the front page, and there seemed to be an unspoken "gentleman's agreement" that it wouldn't be downvoted past a score of -1. These days I am seeing such questions more often get a score of -2 and -3. It's only a small thing, but it shows a shift in user base who (understandably) never got the non-existent memo. $\endgroup$
    – David Roberts Mod
    Commented Jan 20, 2021 at 3:30
  • 2
    $\begingroup$ Alec, maybe the problem with MO is that people are spreading nasty untrue rumours about us and telling undergrads not to come here. I wouldn't put it past some people that they'd do it out of pure malice. ='[ $\endgroup$
    – Harry Gindi
    Commented Jan 20, 2021 at 5:00
  • 3
    $\begingroup$ @DavidRoberts: It seems to me evident that, since all the discussions in MO outside Mathematics are in general considered off-topic, it is very difficult to create human relationships in the usual sense. To have these, one should share some details of his private life, that is common on Facebook and Twitter, but not at all on MO. I do not see any bias in this, it is just a plain observation. With this I am not saying that MO is better or worse than Twitter, I am just saying that it is structurally different. $\endgroup$
    – Francesco Polizzi
    Commented Jan 20, 2021 at 6:20
  • 2
    $\begingroup$ @FrancescoPolizzi I may have a broader definition of what counts as a human relationship, then. I don't disagree that the types of professional relationships one can build on MO are different to ones that revolve around pictures of family and so on. $\endgroup$
    – David Roberts Mod
    Commented Jan 20, 2021 at 6:58
  • 1
    $\begingroup$ @FrancescoPolizzi Of course, there is also chat associated with MO. In chat, many things which would be off-topic on the Q&A site are perfectly fine, so if somebody needs a place to socialize, chat would be a reasonable choice. (If I look at Homotopy Theory - which is currently the most active chatroom associated with MO - I'd guess that at least the most active users probably know each other a bit by now, having posted thousands of messages there. $\endgroup$
    – Martin Sleziak
    Commented Jan 20, 2021 at 7:06
  • 2
    $\begingroup$ @MartinSleziak: About this I agree, but the chat is not the first environment that a newcomer on MO sees. Surely, the very active users can have a more complete experience on the site. $\endgroup$
    – Francesco Polizzi
    Commented Jan 20, 2021 at 7:31

2 Answers 2

Reset to default
28
$\begingroup$

We can use the Data Explorer to see how often we up- and downvote. This query compares the number of posts (including deleted ones) and the number of up- and downvotes for all sites in the network. On MathOverflow, a post receives (on average) 5.29 upvotes and 0.46 downvotes.

enter image description here

Of the 176 sites in the network, we're #23 when it comes to upvotes per post, and #35 for downvotes.

We can't use SEDE to measure how fast we downvote; see Why is vote time missing in the SE data dump and SEDE? for details.

$\endgroup$
13
  • $\begingroup$ Thank you Glorfindel; I agree that anonymity in voting is important, but I don’t think (say) hour-by-hour timestamps on votes would be enough to reconstruct anything personal, and they could still be edifying to see how many posts receive ‘first hour’ downvotes here versus other sites. I think this may be an important metric in understanding our perceived unfriendliness. $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 11:34
  • $\begingroup$ Running a few quick queries, it appears that one downvote every 5-4 posts is fairly standard, while MO is rocking double that at almost one downvote every other post. Is there any way to view all sites in one window? $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 11:41
  • $\begingroup$ Ah, I just saw that you attempted this and it timed out, my bad for not reading and thank you for trying. $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 12:01
  • $\begingroup$ It turns out I made a mistake in the query and now managed to create a correct one, see the updated answer. $\endgroup$
    – Glorfindel
    Commented Jan 19, 2021 at 12:18
  • $\begingroup$ I will add that the table Votes still contains the date, but not the exact time. But since we cannot look at them on a finer scale than days, this is unlikely to give some interesting data, since the most votes come on the first few days: data.stackexchange.com/mathoverflow/query/1236796/… data.stackexchange.com/mathoverflow/query/1236798/… $\endgroup$
    – Martin Sleziak
    Commented Jan 19, 2021 at 13:35
  • $\begingroup$ An unfortunate problem with average (=mean) downvote counts is that it's hard to know what these mean. With about 0.5 downvotes a post, this could mean that roughly 1/2 of all posts have one downvote. I know that I sample questions in a very biased way (I tend only to read question with certain tags which I follow) but this seems way off the mark: it's very rare for me to see any question with a downvote. $\endgroup$
    – Matthew Daws
    Commented Jan 19, 2021 at 15:24
  • 3
    $\begingroup$ An alternative explanantion is of course that a small number of posts have a large number of close votes. We do get spam (blantant advertising or otherwise nonsense) and some "please do my Calculus 101 homework for me" questions, which I guess are downvoted, closed and deleted in short order. Would these affect the average? From the perspective of being "unwelcoming", I am not in the least concerned about this class of question. $\endgroup$
    – Matthew Daws
    Commented Jan 19, 2021 at 15:26
  • $\begingroup$ Finally, I suppose the problem is that we don't have the data analysis tools to really be able to say any more...? $\endgroup$
    – Matthew Daws
    Commented Jan 19, 2021 at 15:27
  • 2
    $\begingroup$ Despite all this questionability, it is very interesting. Seems like in terms of (downvotes per post)/(upvotes per post), the least friendly are sites like Islam, Android, SO.Spanish, Health, SE.Meta, Christianity, Biblical Hermeneutics, ... while the most friendly ones are ... SE.Korean, Mathematica, Latin, Emacs, and TeX $\endgroup$
    – მამუკა ჯიბლაძე
    Commented Jan 19, 2021 at 15:37
  • 1
    $\begingroup$ @MatthewDaws Perhaps queries looking only at posts which were not deleted would - at least to some extent - solve the problem that this includes some posts which definitely deserved to be downvoted (such as spam and off-topic posts). I have added one such query into my answer - probably there are many other meaningful ways how to restrict the questions from which we're taken the average. $\endgroup$
    – Martin Sleziak
    Commented Jan 19, 2021 at 16:24
  • 1
    $\begingroup$ In addition to downvotes per post, downvotes per view could be interesting, and MO have rather high view numbers (at least compared to Cross Violated, which I frequent more ...) $\endgroup$
    – kjetil b halvorsen
    Commented Jan 24, 2021 at 12:42
  • $\begingroup$ I agree that downvotes/post is of little relevance to compare with other sites, and should be divided with the number of views by people who have the right to vote (dividing by the number of views would be a first good approximation). $\endgroup$
    – YCor
    Commented Jan 31, 2021 at 17:33
  • $\begingroup$ The problem is that the number of views is only known for questions and it includes anonymous users and users without voting privileges. $\endgroup$
    – Glorfindel
    Commented Jan 31, 2021 at 17:37
9
$\begingroup$

Here are some additional SEDE queries. (As my knowledge of SQL is rather limited, take all of this with a grain of salt.)

  • Percentage of Upvotes and Percentage of Upvotes (multisite). MathOverflow currently gives 91.94%.
  • We can obtain average number of up/down-votes on post which weren't deleted if we look into table Posts rather than PostsWithDeleted. (But we must restrict the votes to those questions too, which adds a bit more complexity to the query and makes it run slower.) Here is a query which does that: Upvotes/downvotes per posts - only the post which are not deleted. (I will add a link to the same query where Posts is replaced with PostDeleted - if the query works correctly, this should give the same numbers as Glorfindel's query for deleted posts.) If we only take posts which weren't deleted, this gives 0.16 downvote per post - compared to 0.46 if we include the deleted posts.
  • We could also look at average number of downvotes cast by a user. Of course, it probably does not make sense to take all users - since many users have an account here but they have never done anything on the sites. Here is a SEDE query which counts users with reputation at least 125 - it returns 10.75 as the average for MO. (I chose 125, since this is the reputation needed for downvoting. You can change the parameter in the query to any different threshold. For example, if you want to see how generous are the users with upvotes, perhaps it is more sensible to set it to 15, since users can upvote from 15 reputation points. Here MO is among the most upvoting sites, with average 33.64 upvotes per user.)
  • Mostly as curiosity I will include what are the results if we take anonymous feedback1 instead of votes. Glorfindel's query with Upvotes/Downvotes per post gives 0.28 upvotes and 0.12 downvotes as average after this modification. If we look at the percentage of upvotes we get 69.94% from anonymous feedback.

მამუკა ჯიბლაძე suggested in a comment to look specifically at new users. The following queries only look at the posts which weren't deleted, which should exclude stuff such as obvious homework. (You can use similar queries with PostsWithDeleted table instead of Posts, if you want to included the deleted posts, too.)

1These are "votes" from users who actually cannot cast votes - they are recorded as feedback but do not influence score/reputation. Links to more details on this can be found here: https://meta.stackexchange.com/tags/post-feedback/info

$\endgroup$
13
  • $\begingroup$ Thank you, but the ratio of upvotes to total votes might be misleading due to the existence of many posts with extremely inflated amounts of upvotes, because their content is wonderful or written by an authority figure in a field. I have seen many such questions/answers on MO; would it be possible to get a list like this one, but for the metric listed in Glorfindel's post? $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 11:49
  • $\begingroup$ Let me rephrase; I am sure it is possible, but I am too illiterate in the syntax for this search engine to manifest it. Could you please query this oracle my question, or point me in the direction of a page explaining the syntax for how to ask it questions? $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 11:53
  • $\begingroup$ Ah, I just saw that Glorfindel tried to get all of them in one window and it timed out. $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 12:01
  • $\begingroup$ The edited list is awesome, but how do I tell which site corresponds to which entry on the list? $\endgroup$
    – Alec Rhea
    Commented Jan 19, 2021 at 12:13
  • $\begingroup$ @AlecRhea Sorry about that - the updated query also shows names. In any case, Glorfindel mentioned in chat that he now has a query with exact numbers which does not time out. I suppose he'll update the answer soon. $\endgroup$
    – Martin Sleziak
    Commented Jan 19, 2021 at 12:19
  • 1
    $\begingroup$ I don’t think that average number of votes per user is a sensible measure, as this will vary by several orders of magnitude among users. Many users likely vote at a more-or-less constant rate, hence the absolute numbers will be proportional to the age of their account. The number of votes divided by age of the user account probably isn’t a useful thing to average either, as some users participate on the site much more than others, and those will also vote more. I’d suggest to look at the number of votes divided by reputation of the user. $\endgroup$
    – Emil Jeřábek
    Commented Jan 19, 2021 at 16:42
  • $\begingroup$ @EmilJeřábek If I understand correctly, what you suggest is taking this number for each user: downvotes/reputation. And then taking the average? Here is a query which shows these numbers for som users and here is a multi-site query with averages. (Although I am not sure how the numbers otained from this can be interpreted.) $\endgroup$
    – Martin Sleziak
    Commented Jan 19, 2021 at 16:58
  • 2
    $\begingroup$ That’s what I meant, thank you. The idea was to get the average number of downvotes per “unit of participation”, using reputation as a proxy for a measure of active participation on the site. (Perhaps the lower bound should be larger than 125, to avoid users whose main source of reputation was the association bonus.) But looking at the results, I’m not sure whether the numbers are really comparable across the network. $\endgroup$
    – Emil Jeřábek
    Commented Jan 19, 2021 at 17:14
  • $\begingroup$ @EmilJeřábek You probably know this - but anyway, I will mention that you can change the parameter for the bound on reputation in the box above the query or simply by changing the number in the url: data.stackexchange.com/mathoverflow/query/1359913/… (Then you need to click on Run Query - if you're the first to run it with that value, the results aren't cached.) $\endgroup$
    – Martin Sleziak
    Commented Jan 19, 2021 at 17:19
  • 2
    $\begingroup$ So mathoverflow is the first wrt upvotes per rep among $\geqslant1000$ reputation users! And they dare complain we are unfriendly, the BASTARDS :E $\endgroup$
    – მამუკა ჯიბლაძე
    Commented Jan 21, 2021 at 7:23
  • 2
    $\begingroup$ But on the serious side - probably to measure it properly one has to restrict to only votes for newbie questions/answers - say, from the users that have registered shortly before the post and have at least one still open question (to exclude the homework guys). $\endgroup$
    – მამუკა ჯიბლაძე
    Commented Jan 21, 2021 at 7:41
  • $\begingroup$ @მამუკაჯიბლაძე I agree with the joke comment how dare they. $\endgroup$
    – Harry Gindi
    Commented Jan 29, 2021 at 13:33
  • 1
    $\begingroup$ @მამუკაჯიბლაძე I have tried to include at least some data concerning votes on posts by relatively new users. $\endgroup$
    – Martin Sleziak
    Commented Jan 30, 2021 at 20:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .