I can see that the post here display nicely typeset mathematical formulas. How can I get such formulas in my posts? What syntax is used to typeset them?

Note: This is an attempt to make at least brief overview of MathJax syntax as suggested some time ago in this answer: FAQ on typesetting of formulae hard to find. The question and answers are community wiki in order to encourage collaborative editing. We should probably try to keep the tutorial simple and brief - with links to other resources, in case somebody needs more details.