$\newcommand{\Res}{\operatorname{Res}}$
Here is my solution：
$I=\int_0^{2\pi} \frac{cos2\theta dz}{1-2acos\theta+a^2}~dx$
By substitution: let $z=e^{i\theta}$, then $dz=ie^{i\theta}, d\theta= \frac{dz}{ie^{i\theta}}=\frac{dz}{iz}$
$cos\theta=\frac{e^{i\theta}+e^{-i\theta}}{2}=\frac{1}{2} (z+z^{-1})$
Similarly, $\cos2\theta=\frac{1}{2} (z^2+z^{-2})$
\begin{align*}
I=&\int_{|z|=1}\frac{\frac{1}{2} (z^2+z^{-2})}{1-a(z+z^{-1})+a^2}\frac{dz}{iz}\\
=&\frac{1}{2i}\int_{|z|=1}\frac{z^2+\frac{1}{z^2}}{z-az^2-a+a^2z}dz\\
=&\frac{1}{2i}\int_{|z|=1}\frac{z^4+1}{z^2(z-az-a+a^2z)}dz\\
=&\frac{1}{2i}\int_{|z|=1}\frac{z^4+1}{z^2(z-a)(1-az)}dz
\end{align*}
Let $f(z)=\frac{z^4+1}{z^2(z-a)(1-az)}$
$f(z)$ has simple pole at $z=a, z=\frac{1}{a}$, and pole of order 2 at $z=0$, but since $-1<a<1$, we only consider $z=a$ and $z=1$.
z=0\begin{align*}
&\Res(f(z);a)=\Res(\frac{\frac{z^4+1}{z^2(1-az)}}{z-a};a)=\frac{a^4+1}{a^2(1-a^2)},\\
&\Res(f(z);0)=\Res(\frac{\frac{z^4+1}{(z-a)(1-az)}}{(z-0)^2};0)=\frac{d}{dz}\frac{z^4+1}{(z-a)(1-az)}
\end{align*}
at $z=a$, $=-\frac{1+a^2}{a^2}$
By Residue Theorem, $I=2\pi i\frac{1}{2i}(\frac{a^4+1}{a^2(1-a^2)}-\frac{1+a^2}{a^2}=\frac{2\pi a^2}{1-a^2}$
