34
$\begingroup$

I feel like MO is no longer really doing what we intended.

When I look at a new question, rather than trying to help a person, I start thinking: is this a human? is the question really motivated, or are they just laying down standard sequences of buzzwords and name-drops. It can be difficult to tell an AI-generated question from ones generated by a grad student "dazzled" by big machines, as they often act quite alike.

Which gets to the point. Most of my effort is consumed doing exactly what the owner of the AI chatbot wants. i.e. we are performing a commercial service, and we are not being paid, nor are we getting any other sort of benefit out of this arrangement.

What options are there other than "raising the gates" and using in-the-flesh authentication, verification that people are human beings? I suppose a counter-argument is that we could easily verify someone is human and yet all their on-line interactions may still be done by AI chat-bots. But at least there would be some accountability and there would be a bottleneck on the process.

$\endgroup$
15
  • 5
    $\begingroup$ Just one observation: the vast majority of AI-generated posts appears to be by new unregistered accounts which do not give any clue about the identity of the person behind them. $\endgroup$
    – Stefan Kohl Mod
    Commented Sep 25 at 20:20
  • $\begingroup$ @StefanKohl correct, but given the nature of the problem if we were to adapt based on that feature, the AI bots would adapt even more quickly. It seems if we are to adapt to this problem it should be in a direction that AI-bots can not grow towards. $\endgroup$
    – Ryan Budney
    Commented Sep 25 at 21:00
  • 10
    $\begingroup$ Don't have any real suggestions, just wanted to say: as a regular MO user, I 100% agree with the sentiment that my experience on the site has gotten dramatically worse as a result of the prevelence of AI content. $\endgroup$
    – Sam Hopkins
    Commented Sep 25 at 21:10
  • 4
    $\begingroup$ I haven't noticed significant uptick in such questions, but I've also been rather selective about what posts I've been viewing (or maybe the mods/community have been doing a great job weeding them out before I get to them, or maybe I'm too gullible). Do we have some statistics on how common such posts are? $\endgroup$
    – Kimball
    Commented Sep 26 at 1:36
  • 4
    $\begingroup$ @Kimball I created this thread after dealing with a succession of 6 or 8 such chatbot questions/answers that all appeared within a few minutes of each other. I've certainly noticed an uptick over the past few months. Currently they appear to come in waves. $\endgroup$
    – Ryan Budney
    Commented Sep 26 at 1:45
  • 3
    $\begingroup$ @Kimball It could be that a lot of these questions go unnoticed if one does not have sufficiently high reputation to see deleted answers and deleted questions $\endgroup$
    – Yemon Choi
    Commented Sep 26 at 1:48
  • 3
    $\begingroup$ @Kimball it's definitely trending up. $\endgroup$
    – David Roberts Mod
    Commented Sep 26 at 1:49
  • 2
    $\begingroup$ Perhaps it is more accurate to say some users are interfering with MO being used as intended? That there is plenty of abuse of this forum should, I think, meaningfully be distinguished from the forum not serving its purpose at all, which I think isn't the case. I have both received useful answers to my questions and answered (hopefully) human-generated questions, hopefully usefully, even in the Age of AI. $\endgroup$
    – LSpice
    Commented Sep 27 at 12:12
  • 6
    $\begingroup$ As a real human who has often found it very hard to ask a question in earnest, given the sometimes quick-to-be-negative-feedback, I think the opposing sentiment is: it's already hard enough to get new(ish) users to ask questions so most suggestions to deter AI/chatbot questions would likely make the site even more insular than it already can feel. (stackoverflow.blog/2020/01/22/…) $\endgroup$
    – cheyne
    Commented Sep 27 at 17:25
  • 4
    $\begingroup$ @cheyne: that's a good argument, but of course the counter-argument is nobody will want to participate on a site where they are being abused and lied to. i.e. increasingly users see that even if they want to help people, it's getting increasingly more difficult as real humans are becoming increasingly thin on the ground. $\endgroup$
    – Ryan Budney
    Commented Sep 27 at 17:29
  • 2
    $\begingroup$ @RyanBudney I completely agree. I don't see a way forward without forcing the company (stack) that owns these sites to pitch in with their resources; at least maybe so that these posts can exist (for those excited to help out the AI tech companies) but they are all flagged? Like... if the community members of mathoverflow solve what is really a problem for the owner of the site, they would have indirectly done free labor regardless. Maybe that's the lesser of evils but still seems they should help. $\endgroup$
    – cheyne
    Commented Sep 27 at 17:38
  • 3
    $\begingroup$ @cheyne: I think that is beyond their technical capabilities. Even if they wanted to do something like that, they couldn't. $\endgroup$
    – Ryan Budney
    Commented Sep 27 at 17:40
  • 3
    $\begingroup$ Just out of curiosity, what use pattern of MO exposes a user to a palpable amount of AI generated posts/answers? I personally mostly browse the front page under the "Active" tab. The AI stuff that I see doesn't feel like a flood, and what I see has usually already been acted on by other users or mods. $\endgroup$
    – Igor Khavkine
    Commented Oct 2 at 12:07
  • 4
    $\begingroup$ @IgorKhavkine My experience isn't too different from that. I have some items suppressed from my display and a few items highlighted. When I notice a problematic question or answer I often look at other activity by the same user, which leads me to chains of poor questions or answers that I otherwise might have missed. $\endgroup$
    – Ryan Budney
    Commented Oct 2 at 16:13
  • 4
    $\begingroup$ The problem is just getting worse and worse. I counted like half a dozen new unregistered users asking weird questions, who I highly suspect to be AI, in just the past several hours this evening. $\endgroup$
    – Sam Hopkins
    Commented Oct 7 at 0:01

3 Answers 3

Reset to default
35
$\begingroup$

It's become a real nightmare. I know that you have to register to ask questions on math.se. Though I would not have supported this in the past, I would support implementing this here as well. It wouldn't stop the constant flood of bad AI questions altogether, but it might slow it down a little bit.

$\endgroup$
5
  • 8
    $\begingroup$ For the record, here is a post from 2017 about registration for asking on MO: Should MathOverflow require registration to ask a question? (Naturally, the site has changed since then - so the same proposal might be viewed differently now.) $\endgroup$
    – Martin Sleziak
    Commented Sep 25 at 19:33
  • 6
    $\begingroup$ I would support making users register. It's still a step below banning anonymity. And it might help with the AI bot problem. Is there a way we could do a vote on it as a community? $\endgroup$
    – Sam Hopkins
    Commented Sep 25 at 22:19
  • 11
    $\begingroup$ @SamHopkins mods can raise it with the board, for now I suggest making a new meta post specifically about the proposal to require registration, with two CW answers: Yes and No, and let users vote. $\endgroup$
    – David Roberts Mod
    Commented Sep 26 at 0:01
  • 2
    $\begingroup$ @DavidRoberts: done; see meta.mathoverflow.net/questions/6042 $\endgroup$
    – Sam Hopkins
    Commented Sep 26 at 2:40
  • $\begingroup$ @Sam thanks for that $\endgroup$
    – David Roberts Mod
    Commented Sep 26 at 2:45
11
$\begingroup$

I personally think this site would be much better if no postings were anonymous. I believe MO already asks that answers be reasonably kind. And if this caused some now anonymous posters to think a bit harder before asking questions, this might serve them well.

$\endgroup$
6
  • 12
    $\begingroup$ I think that MO has a reputation for unkindness, and that this might make contributors who would be welcome here reluctant to participate under their real names. (I personally do not see it as unkind, except for occasional spats, but, then, I guess a regular participant is unlikely to be someone who sees the site that way, and also unlikely to be familiar with the current new-user experience.) $\endgroup$
    – LSpice
    Commented Oct 2 at 19:07
  • 3
    $\begingroup$ Previous discussions on this topic: Anonymity Policy and On the role of anonymity in a site dedicated to research-level mathematics. $\endgroup$
    – Martin Sleziak
    Commented Oct 3 at 6:00
  • 2
    $\begingroup$ @LSpice Many people conflate being straightforward and forthright as being rude. I have very rarely encountered people on MO who I would classify as "unfriendly" as in being maliciously nasty for no good reason (though a few users fall under this category, unfortunately). On the other hand, I have seen many users get "offended" that their ego ended up being bruised because they asked a question that either doesn't make sense or is a lot more trivial than thy think. $\endgroup$
    – Stanley Yao Xiao
    Commented Oct 4 at 4:13
  • 3
    $\begingroup$ @Stanley you aren't seeing the people who don't ask their question due to MO's reputation. The slightly difficult culture is well-attested; if you aren't phased by it, you are privileged. It's not about people getting annoyed at being told their low-quality question is off-topic, by the way. $\endgroup$
    – David Roberts Mod
    Commented Oct 5 at 9:30
  • 2
    $\begingroup$ @DavidRoberts I have certainly heard some very negative comments regarding MO from people all over the mathematics community, but I still don't see how to quantify this notion of "unfriendliness". If I can summarize the negative opinions I have heard, it's that MO has rules which are too rigid and "gatekeepers" who think too highly of themselves. I am not sure one can infer that there is a problem with the community at large based on these opinions. If you know something I don't, please share $\endgroup$
    – Stanley Yao Xiao
    Commented Oct 5 at 17:47
  • 4
    $\begingroup$ @StanleyYaoXiao there was a working group a few years back called together by Ravi Vakil, including people whose voices aren't heard here on MO. I don't think I can share anything about it. Feel free to email Ravi about it if this is something you need convincing of. $\endgroup$
    – David Roberts Mod
    Commented Oct 5 at 22:10
-18
$\begingroup$

In the long run, the (ironic) solution may be to implement an AI of our own designed to flag other AI generated posts, and to stay ahead in the whole arms race associated with that approach.

But consider this: AI’s aren’t antithetical to MO’s purpose as I understand it, which is

To explore the bleeding edge of mathematics research.

Certainly bad AI questions/answers like the ones we’ve recieved so far aren’t helpful, and I suspect that this is all AI companies will be able to produce for some years to come (unless some of us personally try to speed the process up).

I would say we keep doing what we’ve been doing; having a fascinating conversation about mathematics. If AI reaches a point where it can participate meaningfully cool, but until then I don’t mind if some other humans trying to make a thing (remember: this is all AI is) use my input here to try and improve their thing. That’s kind of how I use the site (but for a different thing), so who am I to judge?

$\endgroup$
21
  • 16
    $\begingroup$ I feel like this answer misses Ryan Budney's point that now instead of helping a person learn math, us "real" MO users are helping some company write software. $\endgroup$
    – Sam Hopkins
    Commented Sep 25 at 21:53
  • 30
    $\begingroup$ @AlecRhea: Math is my job. While out of a sense of civic mindedness I’m happy to help actual people if they have problems I find interesting, I am not willing to work for free for a for-profit company. And I resent them lying to try to trick me into working for free. $\endgroup$
    – Andy Putman
    Commented Sep 25 at 23:24
  • 21
    $\begingroup$ I don't really see modern AI as having anything to do with mathematics. These are basically fraud machines, not thinking machines. So if we got to the point where AI's could really think about mathematics without having mathematicians do all the hard work for them, that would be great. We're just not there, yet. $\endgroup$
    – Ryan Budney
    Commented Sep 26 at 0:31
  • 6
    $\begingroup$ Not sure how this is relevant to the discussion. The nature of AI will change drastically as we go into the future. Current implementations likely aren't all that relevant to future implementations. $\endgroup$
    – Ryan Budney
    Commented Sep 26 at 1:05
  • 28
    $\begingroup$ I just wanted to chime in that I disagree fundamentally with your contention that by responding to people spamming this site with LLM content we are somehow advancing mathematics. If the people feeding content into LLMs do not understand what they are posting here, and do not understand the responses given by people here, then how is this advancing anything rather than generating more bullshit? $\endgroup$
    – Yemon Choi
    Commented Sep 26 at 1:13
  • 18
    $\begingroup$ "You are helping a person learn math when you help some company write software" - I can't express strongly enough how much I disagree with this. $\endgroup$
    – Yemon Choi
    Commented Sep 26 at 1:14
  • 16
    $\begingroup$ You are being kind of silly, Alec. You can conflate bland linear algebra with talking about research in mathematics, but this isn't the nature of our discussion here. You've lost the plot entirely. $\endgroup$
    – Ryan Budney
    Commented Sep 26 at 1:17
  • 16
    $\begingroup$ @AlecRhea: I don't view mathematics as a "money making endeavor". That's a total mischaracterization of what I said. If that was my view, I would be working at a hedge fund or something. I reiterate: I'm happy to contribute to the general public (and the university I work at even views that as part of my job -- I list things like MO on my annual report each year). What I'm not willing to do is be tricked/manipulated by a for-profit company. If a company is making money off my labor, I should be compensated for it (and so should you, and everyone else). $\endgroup$
    – Andy Putman
    Commented Sep 26 at 1:44
  • 11
    $\begingroup$ @Alec there are great turmoils in the internal workings of the Stack Exchange system, one visible sign of which was the big moderator strike over the company outlawing actions over AI spam answers (among other things). The company is really really keen to get into the AI space, and there are lots of behind the scenes discussions. Making keen suggestions like this is, while I assume in good faith, blind to what may be happening to the system in the near-to-medium term. $\endgroup$
    – David Roberts Mod
    Commented Sep 26 at 1:49
  • 6
    $\begingroup$ @Alec, you appear to be confused about the nature of the conversation here. $\endgroup$
    – Ryan Budney
    Commented Sep 26 at 1:49
  • 41
    $\begingroup$ I am quite puzzled by the votes to delete this post. This is supposed to be a discussion, and this answer presents an opinion that is on topic, expressed in a polite way, and does not include offensive or discriminatory language or any such things. Sure, it is an unpopular opinion, but that is not a valid reason to censor it. Vote it down to oblivion, if you will (I did), but why delete it? $\endgroup$
    – Emil Jeřábek
    Commented Sep 26 at 13:31
  • 4
    $\begingroup$ FWIW I second @EmilJeřábek $\endgroup$
    – Yemon Choi
    Commented Sep 26 at 15:47
  • 4
    $\begingroup$ In light of @EmilJeřábek's comment, I retracted my vote to delete $\endgroup$
    – Sam Hopkins
    Commented Sep 26 at 19:17
  • 5
    $\begingroup$ Talking about whether you're happy for AI trainers to "use my input here" to train seems to be either missing the point or conflating two issues. One issue is scraping MO to provide input for training. The other is posting to MO, presumably to validate the output as part of the feedback process of the training. Some people may be happy with the first but not the second. This meta question is primarily about the second, but your answer can be read as responding primarily to the first. $\endgroup$
    – Peter Taylor
    Commented Sep 27 at 12:28
  • 4
    $\begingroup$ Yes, what @PeterTaylor says is an important point. I think there are legitimate questions about whether organizations should be allowed to use MO (or any other Stack Exchange) content to train their AI systems. But having AI post content (questions or answers) to MO to help with training is infinitely more insidious than that, because it represents using the site in a way different from its intended purpose to help people learn research-level math. $\endgroup$
    – Sam Hopkins
    Commented Sep 28 at 17:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .