a) $r^2+2r+17=0$
$(r+1)^2=-16$
$r+1=\pm 4 i$
$r_{1}=-1 -\ 4 i$
$r_{2}=-1+\ 4 i$
$y(x)=c_{1} e^{-x} cos{(4x)}+c_{2} e^{-x} sin{(4x)}$
$y^{\prime \prime}+2 y^{\prime}+17 y=40 e^{x}$
$\therefore y_{p}(x)=A e^{x}$
$y_{p}^{\prime}(x)=A e^{x}$
$y_{p}^{\prime \prime}(x)=A e^{x}$
Plug in:
$A e^{x}+2 A e^{x}+17 A e^{x}=40 e^{x}$
$20 A e^{x}=40 e^{x}$
$A=2$
$y_{p}(x)=2 e^{x}$
$y^{\prime \prime}+2 y+17 y=130 \sin (4 x)$
$y_{c}(x)=B \cos 4 x+C \sin 4 x$
$y_{c}^{\prime}(x)=-4 B \sin 4 x+4C \cos 4 x$
$y_{c}^{\prime \prime}(x)=-16 B \cos (4 x)-16 C\sin(4 x)$
Plug in:
$(-16B\cos(4 x)-16C\sin (4 x)+2(-4 B \sin 4 x+4C\cos 4 x)+17(B \cos (4 x)+C\sin (4 x))=130 \sin 4 x$
$\therefore(16B+8C+17B) \cos (4 x)+(-16C-8B+17C) \sin (4 x) =130 \sin 4 x$
$\left\{\begin{array}{l}{B+8 C=0} \\ {C-8 B=130}\end{array}\right.$
$\left\{\begin{array}{l}{B=-16} \\ {C=2}\end{array}\right.$
$\therefore y_{c}(x)=-16 \cos (4 x)+2 \sin (4 x)$
$\therefore y(x)=c_{1} e^{-x} cos{(4x)}+c_{2} e^{-x}sin{(4x)}-16 \cos (4 x)+2 \sin (4 x)+2 e^{x}$
b) $\because~~y(0)=0$ , $y'(0)=0$
$y(x)=c_{1} e^{-x} cos{(4x)}+c_{2} e^{-x}sin{(4x)}-16 \cos (4 x)+2 \sin (4 x)+2 e^{x}$
$y'(x)=-c_{1} e^{-x} cos{(4x)} -{4} c_{1} e^{-x} sin{(4x)}+{4} c_{2} e^{-x} cos{(4x)}-c_2 e^{-x} sin{(4x)}+8 \cos (4 x)+64 \sin (4 x)+2 e^{x}$
Plug in $y(0)=0$:
$0=c_1-16+2$
$\therefore~~ c_{1}=14$
Plug in $y'(0)=0$ and $c_{1}=14$:
$0=-14+4c_2+8+2$
$\therefore~~\left\{\begin{array}{l}{c_{1}=14} \\ {c_{2}=1}\end{array}\right.$
$\therefore~~ y(x)=14 e^{-x} cos{(4x)}+e^{-x}sin{(4x)}-16 \cos (4 x)+2 \sin (4 x)+2 e^{x}$