Funny story: after I posted this I also checked my work against wolfram alpha, which said I got it wrong for the homogeneous part. I spent some time trying to spot any error, but there isn't any. I am smarter than the machine!
(You can press "show step by step solution" and wolfram alpha will come up with what I came up with, so that's why I'm so confident. I filed a bug report, maybe someday this will be fixed...)
$\renewcommand{Re}{\operatorname{Re}}$
Here we use the method of undetermined coefficients. First we find the characteristic equation and its derivative (which will definitely be useful):
$$Q(r) = r^2 + 4r + 5$$
$$Q^\prime(r) = 2r + 4$$
Now we solve for $te^{-2t}$, for which our exponent is $r_1 = -2$.
$$Q(r_1) = 4 - 8 + 5 = 1$$
$$Q^\prime(r_1) = -4 + 4 = 0$$
So we want to use $t e^{-2t}$. If we plug that in we get $L[t e^{-2t}] = e^{-2t}(t Q(-2) + Q^\prime(-2)) = t e^{-2t}$. This is what we want, so we're done the first part.
Now we solve for $e^{-2t} cos(t)$, for which the exponent is $r_2 = -2 + i$.
$$Q(r_2) = (-2+i)^2 + 4(-2 + i) +5 = 4 - 4i - 1 - 8 + 4i + 5 = 0$$
$$Q^\prime(r_2) = 2(-2+i) + 4 = -4 + 2i + 4 = 2i$$
So we want to use $t e^{(-2 + i)t}$. If we plug that in, we get $L[t e^{(-2 + i)t}] = e^{(-2 + i)t}(t Q(-2 + i) + Q^\prime(-2 + i)) = (2i)e^{(-2 + i)t}$.
We don't want that 2i, so let's divide it out in the input. It turns out that $1/2i = i/-2 = -i/2$. So $L[\frac{-i}{2} t e^{(-2 + i)t}] = e^{(-2 + i)t}$, which is nearly right.
Note:
$$\Re e^{(-2 + i)t}) = \Re (e^{-2t}e^{it}) = \Re (e^{-2t}(\cos t + i \sin t)) = e^{-2t} \cos t$$.
Since $L$ is linear, if we take only the real part of the input, we'll get only the real part of the output. So we need to compute the real part of $\frac{-i}{2} t e^{(-2 + i)t}$.
To do that let's multiply it out:
$$\frac{-i}{2} t e^{(-2 + i)t}$$
$$\frac{-i}{2} t e^{-2t}(\cos t + i \sin t)$$
$$t e^{-2t}(\frac{-i}{2}\cos t + \frac{-i}{2}i \sin t)$$
$$t e^{-2t}(\frac{-i}{2}\cos t + \frac{1}{2}\sin t)$$
The real part of that is $\frac{1}{2}t e^{-2t}\sin t$, so that $L[\frac{1}{2}t e^{-2t}\sin t] = e^{-2t} \cos t$.
So then we put our two subproblem solutions together and we get a particular solution, thanks to the linearity of $L$. Our particular solution is:
$$Y(t) = t e^{-2t} + \frac{1}{2}t e^{-2t}\sin t$$
What remains is to compute a general solution for the homogenous part, and we can combine that to find the general solution for the non-homogenous ODE.
During the above work we found that $-2 + i$ was a root of the characteristic equation. So the conjugate is also a root, $-2 - i$, and we can take the real and imaginary parts of $e^{-2 \pm i}$ to find the solution, which is:
$$y_{gen[homogeneous]}(t) = c_1 e^{-2t}\cos(t) + c_2 e^{-2t}\sin (t)$$
So the general solution for the nonhomogeneous ODE is:
$$y(t) = c_1 e^{-2t}\cos(t) + c_2 e^{-2t}\sin (t) + t e^{-2t} + \frac{1}{2}t e^{-2t}\sin (t)$$
