Very interesting suggestion. While I definitely think that, if doable, this should be a separate activity not related to MO in any way, something related came to my mind.
I've seen several great answers here on MO which contain highly nontrivial proofs/arguments. For me in terms of provided information they are similar to an arXiv preprint: a piece of math to be treated with great care; in particular, if going to use it in, say, a paper, I feel obliged to check it myself first.
So would not it be great if there would be something like such a Review section but with the purpose of submitting MO answers? After all this is a Q&A site, and eventually the answers are supposed to reach finalized state, so why not use a separate section to certify tricky answers once and for all? They could then be tagged appropriately, and if some people would decide to use them in a publication they would know that the information provided has been double-checked by some other members of the community.
I know that implicitly this is being done all the time - as soon as I see an interesting nonobvious answer the first thing I try to do is to check it myself, and I am sure many of us here are like this. But without being responsible I can never be sure I will not overlook something, out of laziness or lack of time or whatever. On the other hand if I decide to review an answer explicitly designated for reviewing and will know that the result will be some sort of certificate from the community, I will be much more attentive and careful.