I can understand "I told you so!" as an infuriating thing. Tell me, is it because you were given good advice and didn't take it, so therefore you feel, er, short-sighted? Or is it something else, like a perception that your advice-giver feels superior? Would you say the aggravation is more from you feeling inferior or a perception that they feel superior?

Marriage is the triumph of imagination over intelligence. Second marriage is the triumph of hope over experience.
(Oscar Wilde)