I am not sure what can be described as unusual. Unlike most other saddle points that I have seen, which have two intersecting lines, these have three. I have also observed no saddle point in system $(b)$ is different from any of the other saddle points. All surrounding trajectories behave in the same way. The saddle points are not identical to the one in $(a)$, but are rotated by $\pi/6$ radians.
In Calculus II only non degenerate stationary points were considered: Hessian (if $n=2$ it is $\begin{pmatrix} H_{xx} &H_{xy} \\ H_{yx} & H_{yy}\end{pmatrix}$) should have it determinant different from $0$ and if it is positive we get extremum—minimum as $H_{xx}>0$ and maximum as $H_{xx}<0$; $H_{yy}$ automatically has the same sign—and if determinant is negative we get a saddle).
Here critical point $(0,0)$ is degenerate (and even all second order derivatives vanish here) and it is not the standard saddle. In fact it is called
Monkey saddle (three valleys and three mountains joint here while in the usual saddle two valleys and two mountains join). While ordinary saddle is represented by $\renewcommand{\Re}{\operatorname{Re}}$ $\Re (x+iy)^2=x^2-y^2$ the monkey saddle by $\Re (x+iy)^3=x^3-3xy^2$. The multiplicity of the types and complexity of degenerate critical points even as $n=2$ is a reason why you have not looked at them in Calculus II.
In fact system (b) is also integrable for any $\alpha\ne 0$. May be centers are difficult to find directly but there are simple geometric observations.
The beauty of $\alpha=\sqrt{3}=\tan (\pi/3)$ is that triangles are regular (and near critical points it `almost' coincides with (a)). The sides of triangles are $\pi/\sqrt{3}$ so the whole plane is tiled by regular triangles and their vertices are monkey saddles and the whole picture is not only periodic but has a rotational (by $\pi/3$) symmetry and 6 mirror symmetries. Then the rest of stationary points (centers) must be centers of these triangles (and in the regular triangle there is a single center). Here I ignore directions, just purely geometric picture.
Obviously for $\alpha \ne \sqrt{3}$ we need just scale with respect to $x$, so we get equilateral triangles with the base $\pi/\alpha$ and a height $\pi/2$ (the same as before) and the rest of stationary points (centers) must be on the distance $\pi/6$ from the base and they are centroids (points of intersection of medians) of those triangles. Periodic structure remains but only 2 mirror symmetries and rotational by $\pi$ symmetry.
PS Up to a factor
\begin{equation}
H(x,y)=\sin(4y)-\sin( 2y+2\alpha x) -\sin(2y-2\alpha x),
\end{equation}
Then as $\alpha=\sqrt{3}$
\begin{equation}
H(x,y)=\sin(4\mathbf{x}\cdot \mathbf{e}_1 )+\sin(4\mathbf{x}\cdot \mathbf{e}_2 ) +\sin(4\mathbf{x}\cdot \mathbf{e}_3)
\end{equation}
with $\mathbf{x}=\begin{pmatrix}x\\y\end{pmatrix}$, $\mathbf{e}_j=\begin{pmatrix}\sin (2\pi (j-1)/3)\\ \cos (2\pi (j-1)/3)\end{pmatrix}$; those are three unit vectors with angles $2\pi/3$ between. Then rotation by $2\pi/3$ preserves $H(x,y)$ and by $\pi/3$ replaces $H(x,y)$ by $-H(x,y)$ which also preserves geometry.
Show Posts