This question relates directly to issues that are often discussed in the philosophy of mathematics.
According to one of the standard accounts of structuralism, what mathematical objects are at bottom are the structural roles that they play within a mathematical system. One might define the equivalence relation whereby element $a$ in structure $A$ is equivalent to element $b$ in structure $B$ exactly when there is an isomorphism of $A$ to $B$ taking $a$ to $b$. This results in what I have called the isomorphism orbit of the object $a$ in $A$.
According to some accounts of the philosophy of abstract structuralism, what a mathematical object is, is the structural role that it plays in abstract mathematical structures.
The problem for this view arises with nonrigid mathematical structures such as the complex field $\newcommand\C{\mathbb{C}}\C$.
Here is how I describe the situaiton is my book, Lectures on the Philosopy of Mathematics (MIT Press)
Although one conventionally describes $i$ as "the square root of negative one," nevertheless one might reply to this, ``Which one?'' in light of the fact that $\newcommand\unaryminus{-}\unaryminus i$ also is such a root:
$$(\unaryminus i)^2=(\unaryminus 1\cdot i)^2=(\unaryminus 1)^2i^2=i^2=\unaryminus 1.$$
Indeed, the complex numbers admit an automorphism, an isomorphism of themselves with themselves, induced by swapping $i$ with $\unaryminus i$---namely, complex conjugation:
$$z=a+bi\qquad\mapsto\qquad\bar z= a-bi.$$
The conjugation map preserves the field structure, since $\overline{y+z}=\bar y+\bar z$ and $\overline{y\cdot z}=\bar y\cdot\bar z$, and therefore the complex field is not a rigid mathematical structure. Since conjugation swaps $i$ and $\unaryminus i$, it follows that $i$ can have no structural property in the complex numbers that $\unaryminus i$ does not also have. So there can be no principled, structuralist reason to pick one of them over the other. Is this a problem for structuralism? It does seem to be a problem for singular terms, since how do we know that the $i$ appearing in my calculations this week is the same number as what will appear in your calculations next week? Perhaps my $i$ is your $\unaryminus i$, and we do not even realize it.
If one wants to understand mathematical objects as abstract positions within a structure, as in abstract structuralism, then one must grapple with the fact that in light of the conjugation automorphism, the numbers $i$ and $\unaryminus i$ play exactly the same roles in this structure (see Shapiro, 2012). The numbers $i$ and $\unaryminus i$ have the same isomorphism orbit with respect to the complex field, and so in this sense, although distinct, they each play exactly the same structural role in $\C$. This would seem to undermine the idea that mathematical objects are abstract positions in a structure, since we want to regard these as distinct complex numbers.
The point is that one cannot understand the mathematical objects as identical to the structural roles that they play within a system, since $i$ and $\unaryminus i$ play exactly the same role in $\C$ as each other, yet are to be regarded as distinct complex numbers.
In fact, there is nothing special about the numbers $i$ and $\unaryminus i$ in this argument. For example, the numbers $\sqrt{2}$ and $\unaryminus\!\sqrt{2}$ also happen to play the same structural role in the complex field $\C$, because there is an automorphism of $\C$ that swaps them (although one uses the axiom of choice to prove this). Contrast this with the real field $\newcommand\R{\mathbb{R}}\R$, where $\sqrt{2}$ and $\unaryminus\!\sqrt{2}$ are of course discernible, since one is positive and the other is negative, and the order is definable from the field operations in $\R$ via $x\leq y\iff\exists u\ x+u^2=y$. It follows that the real number field is not definable in the complex field by any assertion in the language of fields. In fact, there is an enormous diversity of automorphisms of the complex field; one may move $\sqrt[3]{2}$, for example, to one of the nonreal cube roots of $2$, such as $\sqrt[3]{2}(\sqrt{3}i-1)/2$. Therefore, the numbers $\sqrt[3]{2}$ and $\sqrt[3]{2}(\sqrt{3}i-1)/2$ are indiscernible in the complex field---there is no property expressible in the language of fields that will distinguish them. Indeed, except for the rational numbers, every single complex number is part of a nontrivial orbit of automorphic copies, from which it cannot be distinguished in the field structure. So the same issue as with $i$ and $\unaryminus i$ occurs with every irrational complex number. For this reason, it is problematic to try to identify complex numbers with the abstract positions or roles that the numbers play in the complex field.
Meanwhile, one may recovers the uniqueness of the structural roles simply by augmenting the complex numbers with additional natural structure. Specifically, once we augment the complex field $\C$ with the standard operators for the real and imaginary parts:
$$\text{Re}(a+bi)=a\qquad\qquad\text{Im}(a+bi)=b,$$
then the expanded structure $\langle\C,+,\cdot,\text{Re},\text{Im}\rangle$ becomes rigid, meaning that it has no nontrivial automorphisms. Thus, every complex number plays a unique structural role in this new structure, which is Leibnizian. This additional structure is implicit in the complex plane conception of the complex numbers, which is part of why the number $i$ appears fine as a singular term---it refers to the point $(0,1)$ in the complex plane---whereas $\unaryminus i$ refers to $(0,-1)$. The complex plane is not merely a field, for it carries along its coordinate information by means of the real-part and imaginary-part operators, making it rigid. In the complex plane, every complex number plays a different role.
