Timeline for ChatGPT strikes MathOverflow
Current License: CC BY-SA 4.0
20 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 17, 2022 at 20:27 | comment | added | Loreno Heer | nonsensical of course | |
| Dec 17, 2022 at 18:23 | comment | added | Jukka Kohonen | @Loreno, nonsensical or non-nonsensical? | |
| Dec 15, 2022 at 21:37 | comment | added | Loreno Heer | I had quite a lot of conversations with ChatGPT today feeding it some unsolved problems, and some solved ones from here math.utah.edu/~bestvina/eprints/questions-updated.pdf and having a look at the answers. It is quite interesting but mostly non-nonsensical. However interestingly enough, when given an example and then asking ChatGPT to break it down to the definitions and pointing out the errors, it seems to be capable of adapting the answer. (it won't be stored for the next user though). | |
| Dec 14, 2022 at 14:00 | comment | added | Jukka Kohonen | I have failed to get a sensible answer when I ask it to two-color a cycle graph of five vertices. It happily reports the requirements of two-coloring (adjacent vertices have different colors). Then it happily suggests colorings like red,blue,red,blue,red. When I point out that the first and last are now red, it agrees it is incorrect, and offers another wrong one, ad infinitum. All in excellent English. So already the five-vertex cycle graph is something over the horizon of its inference capabilities. | |
| Dec 13, 2022 at 14:02 | comment | added | Pablo H | @darijgrinberg I'm very far from knowing how GPT/ChatGPT work, but I've heard that such AI systems do have finite memory (as in Markov chains). | |
| Dec 12, 2022 at 6:56 | comment | added | Todd Trimble Mod | @TimCampion Sounds like ChatGPT is cheating off of Lean's test. | |
| Dec 11, 2022 at 19:55 | comment | added | Tim Campion Mod | Somebody pointed out to me that ChatGPT seems to know something about what it means to prove things in Lean. I haven't explored this too much, but you can ask it to give a proof of XYZ in Lean, and it will give you actual Lean code (which may or may not actually work). | |
| Dec 11, 2022 at 8:53 | comment | added | User1865345 | As I remarked in CV too, this is Mathgen on steroids. | |
| Dec 10, 2022 at 17:09 | comment | added | darij grinberg | @AlexM.: Not sure what statement of mine you're referring to, as I think we are agreeing. What I mean is that AI for proofs seems both realistic and useful to me, but ChatGPT is definitely not it. The best part about proofs is that you can use automated reinforcement learning, which is likely much faster than anything that relies on human feedback. But that was clearly not the optimization target for ChatGPT. | |
| Dec 10, 2022 at 16:27 | comment | added | Yemon Choi | @darijgrinberg " I have a hunch that it treats lexically similar "words" as semantically similar" - by complete coincidence reading this made me think of marking HW and some referee work I should be doing. | |
| Dec 10, 2022 at 12:47 | comment | added | Alex M. | @darijgrinberg: I disagree with your last statement: natural language is very tolerant to local ambiguity (and even local falseness), whereas the mathematical one is not (and, in fact, the absence of precision at every point of a statement can even lead to catastrophic results). The problem is that the AI methods developed lately are insensitive to details, and tend to capture only the "general shape of a thing", while these subtle and minute details that are discarded by AI tend to form most of the modern mathematical thinking. | |
| Dec 10, 2022 at 6:13 | comment | added | darij grinberg | An alternative conclusion is that rhetoric and poetry are significantly easier than (even quite straightforward) mathematics. Flattering, but not very convincing. I'm skeptical about general AI, but the prospect of having an AI help out with mathematical proofs actually sounds quite plausible to me! | |
| Dec 10, 2022 at 6:10 | comment | added | darij grinberg | I have so far been unable to get ChatGPT to correctly prove $\operatorname{Tr}(AB) = \operatorname{Tr}(BA)$, whether or not I use LaTeX or add suggestive extra questions. It either forgets about non-diagonal entries, or turns an $a_{i,j}$ into $a_{i,i}$ somewhere. When I tell it about its errors, it "recognizes" them, sometimes elaborating on a counterexample I provide, but then it repeats the same errors again. I have a hunch that it treats lexically similar "words" as semantically similar, which dooms it to uselessness in mathematics. | |
| Dec 9, 2022 at 21:14 | history | edited | Martin Brandenburg | CC BY-SA 4.0 |
added 308 characters in body
|
| Dec 9, 2022 at 21:08 | history | edited | Martin Brandenburg | CC BY-SA 4.0 |
added 36 characters in body
|
| Dec 9, 2022 at 21:03 | history | undeleted | Martin Brandenburg | ||
| Dec 9, 2022 at 21:03 | history | edited | Martin Brandenburg | CC BY-SA 4.0 |
added 36 characters in body
|
| Dec 9, 2022 at 20:51 | history | deleted | Martin Brandenburg | via Vote | |
| Dec 9, 2022 at 20:47 | history | edited | Martin Brandenburg | CC BY-SA 4.0 |
added 36 characters in body
|
| Dec 9, 2022 at 20:37 | history | answered | Martin Brandenburg | CC BY-SA 4.0 |