[math]a^2 + 2ab - 3b^2 = 61 [/math]
[math]\implies (a + 3b) \cdot (a - b) = 61 \tag {E01} \\[/math]
[math]\text{a,b are natural numbers, RHS is prime and RHS > 0: } [/math]
[math]\qquad \implies a > b; \quad a \neq b ; \quad a,b > 0 \\[/math]
[math]\text{Considering the factors of 61 : } [/math]
[math]\text{Case - 1 : When } \: \begin{cases} (a+3b) = 61 \\ (a-b) = 1 \end{cases} \: : [/math]
[math]\qquad \boxed{\boxed{\: a = 16; \quad b = 15 \:}} \\[/math]
[math]\text{Case - 2 : When } \: \begin{cases} (a+3b) = 1 \\ (a-b) = 61 \end{cases} \: : [/math]
[math]\qquad b = -15 \: \ngtr 0 \: \times \\[/math]
Read more »