It is annoying, but i've come to just expecet it now, so after that first hit, I just start turning around
Seems to happen the most with cutlass and doll. The two you have to be the closest for. Maybe getting that close to them scares them and they jump back, lol.