Via Lagrange multipliers:
[tex]L(x,y,\lambd)=x^2y+x+y+\lambda(xy-5)[/tex]
[tex]L_x=2xy+1+\lambda y=0[/tex]
[tex]L_y=x^2+1+\lambda x=0[/tex]
[tex]L_\lambda=xy-5=0[/tex]
[tex]\underbrace{10}_{2xy}+1+\lambda y=0\implies \lambda=-\dfrac{11}y[/tex]
[tex]xy=5\implies y=\dfrac5x\implies\lambda=-\dfrac{11}5x[/tex]
[tex]\impliesx^2+1+\left(-\dfrac{11}5x\right)x=0\implies x^2=\dfrac56\implies x=\pm\sqrt{\dfrac56}[/tex]
[tex]xy=5\implies y=\pm\sqrt{30}[/tex]
At these points, we get local minima of [tex]f\left(\pm\sqrt{\dfrac56},\pm\sqrt{30}\right)=\pm2\sqrt{30}[/tex].
- - -
Another way to do this is to make [tex]f(x,y)[/tex] a function independent of [tex]y[/tex], which is made possible by the constraint.
[tex]xy=5\implies y=\dfrac5x[/tex]
[tex]\implies f(x,y)=f\left(x,\dfrac5x\right)=F(x)=6x+\dfrac5x[/tex]
[tex]\implies F'(x)=6-\dfrac5{x^2}=0\implies x=\pm\sqrt{\dfrac56}[/tex]
and so on.
