Focussing on sexuality alone and discussing any sexual activity in terms of 'energy' alone is too narrow a view, I feel. It's not just arousal and release; there is also a longing for intimacy, for the feel of another body, caresses, even smells, and discussing that yearning is perhaps more embarassing for most people than scientifically dissecting human sexuality as if the whole subject had nothing to do with them personally. It's a whole complex of psychological and emotional issues, and that's why sexuality cannot be simply sublimated or 'transmuted' into spiritual power, in my mind. People talk about manipulating sexual energy as if it was simply a matter of pressing some switches and routing it through a special tantric electrical transformer. It's not that easy, I'd say. The suppression of sexuality can have undesirable side effects like aggression, for example, and even if you are able to suppess that as well, there are still those secondary desires I mentioned, and they won't just simply disappear after you've finally had an orgasm.
In my mind, the best way to handle the problem is honest introspection, examine what's really there, without judging, without shame, as calmly as you are able to. There might be no resolution for some of those issues but will at least be aware of them and include them when you perform a sincere analysis of your emotions.
Sincerity is the key, I feel, thoroughly knowing yourself and admitting to yourself what you are, warts and all. If you construct the self-image of a true spiritual seeker while constantly ignoring your own feelings - ANY feelings -, you're setting yourself up for failure because they will always bubble up to the surface. Like I said, initially it may be enough to be aware of them, there is no immediate solution for every problem, and anyway, some problems just cannot be solved, only managed.
All in all, I think that repressed or suppressed sexuality might affect your state of mind and thus your magic - note that I wrote 'affect' and not 'weaken' or 'impair'. It may impinge on the emotional aspect of rituals (but so would a fulfilled sex life, only in a different way). I don't believe that chastity automatically increases your magical powers, it's more complicated than that.