Wow! How interesting. You know people assume that being a sex writer is very interesting but the truth is that most questions are repeated with relatively high frequency. Yours, though, is a first.
Sex isn't a race
The answer may lie in your wording of the question. You say you fantasise about your ex 'in order to reach the goal' and that might be the issue. Do you think of sex as some rushed process in which you and your partner race to the finish line?
Many of us feel awkward during sex, especially with new partners. Here we are naked, in a weird position where everything is hanging out. Meanwhile, we are sweating and our bodies might make weird noises.
While sex feels good, that lull between the beginning of intercourse and orgasm can feel endless if you aren’t comfortable with your partner. That discomfort can make you want to get it over and done within the quickest way you know how – such as through fantasy or changing position. Sex makes many of us feel vulnerable and we aren’t okay with that.
'What you need to learn is how to give and receive sexual pleasure while remaining fully present in the moment.'
Give and receive pleasure
Perhaps what you need to learn is how to give and receive sexual pleasure while remaining fully present in the moment. Look at your wife and focus on what you are both doing. What kind of touch does she enjoy? What kind of touch do you enjoy? Are there particular things that your ex did or said that really turned you on? Perhaps a way she kissed you? How about asking your wife to do them? (But do not mention your ex at all while you are making this request, or you might end up dead.)
Most people fantasise during sex at one point or another, and they tend to think about previous sexual encounters so, in that, you are not alone. However, you seem to need this fantasy regularly and you are not okay with that. Looking into your wife’s face or eyes will also help you to stay focused on her.