7
$\begingroup$

Suppose I want to download all my questions and answers, e.g., as separate latex files. Is there a way to do this automatically? For example, this can be done on Facebook and on Google. There are many good reasons a user might want to do this, e.g., to create a book of open problems. If this is currently not possible, how difficult would it be to add such a feature?

Of course, the slow way is to open every question/answer, click "edit", and copy and paste. But that will take many hours. Also, I learned that if you deleted a question then you cannot "edit" it, so you have to first undelete, then grab the text, then re-delete. Also, I learned that you can't delete more than 5 things in one day. Thought I'd record that observation in case it helps any future user.

$\endgroup$
2

2 Answers 2

Reset to default
9
$\begingroup$

Ok, actually, this was very easy. Thanks to Martin for the links. I have created my first data explorer query and it does what I wanted. Now I have a CSV file with everything I've ever written on MathOverflow. Very easy to extract from there into a latex file if I wanted to.

Here's the query: https://data.stackexchange.com/mathoverflow/query/1816844/get-all-questions-and-comments-from-a-given-user

In the spirit of "teach a person to fish..." I will try to solve future "Can I get X data" myself before posting on meta.

$\endgroup$
3
  • $\begingroup$ Probably it is worth mentioning that SEDE query returns at most 50000 results. But even most prolific users have much less posts and comments. So this isn't really a problem here on MO. $\endgroup$
    – Martin Sleziak
    Commented Jan 29 at 4:37
  • $\begingroup$ On Stack Overflow there is one user over 50k posts and several users over 50k comments - such users would have to download their contributions from SEDE in several parts. $\endgroup$
    – Martin Sleziak
    Commented Jan 29 at 4:38
  • 1
    $\begingroup$ And since in the question you have mentioned deleted posts too, probably it is worth explicitly pointing out that deleted posts aren't available in SEDE. (There is a table PostsWithDeleted, but it contains only some data for deleted posts, body isn't one of them.) $\endgroup$
    – Martin Sleziak
    Commented Jan 29 at 5:48
4
$\begingroup$

There is already a fine answer - this is just a minor addition. Some people might prefer Markdown for the posts rather than HTML. The other query uses Body from the Posts table - this contains HTML. Markdown can be obtained using the PostHistory table.

I will repeat a few of the usual caveats:

  • SEDE is only updated once a week - so you won't get very recent posts.
  • You don't get the deleted posts in this way. (There is a table PostsWithDeleted which contains some data about deleted posts - but neither body nor the post owner. So it cannot be used for the purposes discussed here.)
  • The results of a query are limited to 50000 rows - but so far even the most prolific posters on MO are below 50k posts (or 50k comments) - so this won't be problem for MO in the near future.

Naturally, it is possible to do much more - if you have a look at rene's answer to "Dump of my own Stack Exchange content", you can see that there is a query which collects contributions over all sites, creates a HTML file consisting of all results. And also an alternative query, which returns Markdown - again for posts from all sites in the network.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .