a) To find the critical points, we need to set x' = 0 and y' = 0
In the first equation, that is only satisfied when x = -1 or y = n$\pi$, where n is an integer.
However, when we carry those constraints to the second equation, x = -1 is no longer valid as $\cos(y)$ is bound by -1 and 1.
So the critical points are (0, 2n$\pi$), where n is an integer and (2, n$\pi$), where n is $\pm1, \pm3, \pm5, ...$ or 0
b) To get the corresponding linear system we take the Jacobian matrix and substitute the critical points in:
For (0, 2n$\pi$), where n is an integer:
\begin{pmatrix} 0 & 1\\ -1 & 0 \end{pmatrix}
For (2, n$\pi$), where n is $\pm1, \pm3, \pm5, ...$:
\begin{pmatrix} 0 & 3\\ -1 & 0 \end{pmatrix}