Society, civilized society at least, is never very ready to believe anything to the detriment of those who are both rich and fascinating. It feels instinctively that manners are of more importance than morals, and, in its opinion, the highest respectability is of much less value than the possession of a good chef.
You seem so clear about who you are. So certain that you are just right as you are, that for all your intelligence and maybe in spite of it, you never seem to need a second opinion.