This site uses MathJax - which is a library which for displaying mathematics in web browsers. The syntax is basically LaTeX, so if you are already familiar with TeX or LaTeX you should be fine. (It is worth keeping in mind that only stuff that can be used inside math mode works in MathJax too.1)
It is useful to know that you can learn some parts of the syntax also from posts by other users. If you see a formula, right click gets you to MathJax context menu. By choosing "Show Math As > TeX Commands" you can see the source which you can use to typeset formulas.2
The mathematical formulas are enclosed in dollars. If you want a centered formula, you can use double dollars. For example $a^2-b^2=(a-b)(a+b)$ and $$\frac{a^2-b^2}{a-b}=a+b$$ gives $a^2-b^2=(a-b)(a+b)$ and $$\frac{a^2-b^2}{a-b}=a+b$$
Here is an overview of some basic commands, mainly through examples. If you need more than what is listed in this very basic overview, you can consult some of the resources linked in another answer to this question.
- Superscripts, indices: You can use
^ for superscripts and _ for subscripts. If it is supposed to contain more than one symbol, then you enclose the content between {..}. (The same is true in many other cases, not only here.) Examples: $(x_1+\dots+x_n)^2$ $(x_1+\dots+x_n)^2$, $a^{b+c}=a^b\cdot a^c$ $a^{b+c}=a^b\cdot a^c$ and $F_n=F_{n-1}+F_{n-2}$ $F_n=F_{n-1}+F_{n-2}$. This is how you get double subscripts/superscripts: $a^{b^c}$ $a^{b^c}$ and $x_{n_k}$ $x_{n_k}$.
- Fractions, radicals. Again, if you are using more than one symbol, use
{..} - they are not needed if there is only one symbol. (But you can still use them if you want. For example, $\frac\alpha2$ and $\frac{\alpha}{2}$ give the same result.) Examples: $\frac\alpha2\cdot\frac{2^2}{\alpha^2}$ $\frac\alpha2\cdot\frac{2^2}{\alpha^2}$, $\sqrt{x^2+y^2}$ $\sqrt{x^2+y^2}$, $\sqrt[3]5$ $\sqrt[3]5$, $\sqrt[n]n\ge\sqrt[n+1]{n+1}$ $\sqrt[n]n\ge\sqrt[n+1]{n+1}$.
- Inequalities:
$\ge$ for $\ge$, $\le$ for $\le$ and $\ne$ for $\ne$. For strict inequalities you can simply use $<$ $<$ and $>$ $>$. Examples:
$xy\le\frac{x^p}p+\frac{x^q}q$ $xy\le\frac{x^p}p+\frac{x^q}q$
$f\left(\frac{x+y}2\right)\ge\frac{f(x)+f(y)}2$ $f\left(\frac{x+y}2\right)\ge\frac{f(x)+f(y)}2$,
$AB\ne BA$ $AB\ne BA$.
- Bracket and parenthesis: Since
{..} have special meaning in MathJax/LaTeX, to get curly brackets you can use $\{..\}$. Example: $\{x+y; x\in A, y\in B\}$ $\{x+y; x\in A, y\in B\}$. Other types of brackets:
$(x,y)$ $(x,y)$,
$[x,y]$ $[x,y]$,
$\langle x,y\rangle$ $\langle x,y\rangle$,
$\lfloor x \rfloor \le \lceil x \rceil$ $\lfloor x \rfloor \le \lceil x \rceil$.
- It is sometimes useful to change size of brackets, if the content between them is large:
$\left(1+\frac1x\right)^x$ $\left(1+\frac1x\right)^x$. The operators \left and \right work for other types of parenthesis, too.
- Sums, products:
$\sum_{k=1}^n k = \frac{n(n+1)}2$ $\sum_{k=1}^n k = \frac{n(n+1)}2$; $\prod_{n=1}^\infty (1-x_n)=0$ $\prod_{n=1}^\infty (1-x_n)=0$. Notice that delimiters are displayed differently in centered formulas: $$\sum_{k=1}^n k = \frac{n(n+1)}2$$ $$\sum_{k=1}^n k = \frac{n(n+1)}2$$
(To achieve a similar effect in inline formulas you can use \limits as in $\sum\limits_{k=1}^n k = \frac{n(n+1)}2$ $\sum\limits_{k=1}^n k = \frac{n(n+1)}2$.)
- Sets:
$\in$ $\in$, $\notin$ $\notin$, $\cup$ $\cup$, $\cap$ $\cap$, $\setminus$ $\setminus$. Examples: $A\cap (B\cup C)$ $A\cap (B\cup C)$, $A\cap B=\{x\in A; x\in B\}$ $A\cap B=\{x\in A; x\in B\}$. You can also use $\bigcup$ $\bigcup$ and $\bigcap$ $\bigcap$. Example: $\bigcup_{i\in I} \left(X\setminus A_i\right) = X\setminus \left(\bigcap_{i\in I} A_i\right)$ $\bigcup_{i\in I} \left(X\setminus A_i\right) = X\setminus \left(\bigcap_{i\in I} A_i\right)$. The delimiters work here similarly as for sum and product: $\bigcup\limits_{i=1}^\infty (-n,n)=\mathbb R$ $\bigcup\limits_{i=1}^\infty (-n,n)=\mathbb R$.
- Some special fonts:
$\mathbb N$ $\mathbb N$, $\mathbb R$ $\mathbb R$ for blackboard bold. Some other common fonts: $\mathcal B, \mathrm B, \mathfrak B, \mathbf B, \mathscr B$ $\mathcal B, \mathrm B, \mathfrak B, \mathbf B, \mathscr B$
- Some special symbols
$\pm\infty$ $\pm\infty$. $3\mid6$ but $3\nmid 7$ $3\mid6$ but $3\nmid 7$, $\vec a$ $\vec a$, $\binom nk = \binom{n}{n-k}$ $\binom nk = \binom{n}{n-k}$.
- Calculus:
$\lim_{n\to\infty} \left(1+\frac1n\right)$ $\lim_{n\to\infty} \left(1+\frac1n\right)$, $\int_{-\infty}^\infty e^{-x^2} \,dx = \sqrt{\pi}$ $\int_{-\infty}^\infty e^{-x^2} \,dx = \sqrt{\pi}$, $\frac{\partial f}{\partial x}$ $\frac{\partial f}{\partial x}$.
- Greek letters: Simply use
$\alpha$ $\alpha$, $\beta$ $\beta$, $\gamma$ $\gamma$, etc. Some of them have two variants: $\epsilon$ $\epsilon$ and $\varepsilon$ $\varepsilon$, $\phi$ $\phi$ and $\varphi$ $\varphi$.
- Operators:
$\max(a,b)$ $\max(a,b)$, $\min\{a,b\}$ $\min\{a,b\}$, $\sin^2x+\cos^2x=1$ $\sin^2x+\cos^2x=1$, $\ln(1+x)\le x$ $\ln(1+x)\le x$.
- Some examples using matrices:
$$\begin{pmatrix}
1 & 2 & 3 \\
0 & 1 & 4 \\
4 & 2 & -1
\end{pmatrix}$$
$$\begin{pmatrix}
1 & 2 & 3 \\
0 & 1 & 4 \\
4 & 2 & -1
\end{pmatrix}$$
$$\left(\begin{array}{cccc|c}
1 & 1 & 1 & 1 & 0 \\
0 & 1 & 3 & 1 & 2 \\
1 & 1 & 3 & 1 & 4 \\
1 & 1 & 5 & 4 & 2
\end{array}\right)$$
$$\left(\begin{array}{cccc|c}
1 & 1 & 1 & 1 & 0 \\
0 & 1 & 3 & 1 & 2 \\
1 & 1 & 3 & 1 & 4 \\
1 & 1 & 5 & 4 & 2
\end{array}\right)$$
1Many further details on the differences between the two could be added, but this is probably a reasonable rule of thumb. For more detailed information see for example: What is the difference between LaTeX and MathJax? at TeX Stack Exchange or What is the relation between Latex and MathJax? at Mathematics Meta.
2See also How to view LaTeX source of equations? (Mathematics Meta)
