Mathematical Reflections 2016, Issue 1 - Problem J363

Problem:
Solve in integers the system of equations $$\begin{array}{rcl} x^2+y^2-z(x+y)&=&10 \\ y^2+z^2-x(y+z)&=&6 \\ z^2+x^2-y(z+x)&=&-2. \end{array}$$

Proposed by Titu Andreescu, University of Texas at Dallas, USA

Solution:

Observe that if $(x_0,y_0,z_0)$ is a solution to the given system, then also $(-x_0,-y_0,-z_0)$ is a solution. So, if there exists a solution, we can assume without loss of generality that $x_0+y_0+z_0 \geq 0$.
Adding the first equation to the third equation and subtracting the second, we get $$x^2-yz=1.$$ Adding the first equation to the second equation and subtracting the third, we get $$y^2-zx=9.$$ Adding the second equation to the third equation and subtracting the first, we get $$z^2-xy=-3.$$ So, the given system becomes:
$$\begin{array}{rcl} x^2-yz&=&1 \\ y^2-zx&=&9 \\ z^2-xy&=&-3. \end{array}$$
Subctracting the first two equations, then the last two equations and then the first equation and the third equation, we obtain the system of equations
$$\begin{array}{rcl} (x-y)(x+y+z)&=&-8 \\ (y-z)(x+y+z)&=&12 \\ (z-x)(x+y+z)&=&-4. \end{array}$$
Since by our assumption we have $x+y+z \geq 0$, we obtain that $x+y+z \in \{1,2,4\}$. We have three cases.

(i) $x+y+z=1$. Hence, we have $x-y=-8$ and $z-x=-4$, which gives $y=x+8$ and $z=x-4$. Therefore, $x+(x+8)+(x-4)=1$, which gives $x=-1,y=7,z=-5$.
(ii) $x+y+z=2$. Hence, we have $x-y=-4$ and $z-x=-2$, which gives $y=x+4$ and $z=x-2$. Therefore, $x+(x+4)+(x-2)=2$, which gives $x=0,y=4,z=-2$.
(iii) $x+y+z=4$. Hence, we have $x-y=-2$ and $z-x=-1$, which gives $y=x+2$ and $z=x-1$. Therefore, $x+(x+2)+(x-1)=4$, which gives $x=1,y=3,z=0$.

An easy check shows that the only solutions are $$(x,y,z) \in \{(1,3,0),(-1,-3,0)\}.$$