As long as you don't fly openly in the face of society, society doesn't ask any inconvenient questions; and it makes precious short work of the cads who do. There are no secrets better kept than the secrets everybody guesses.
I see is that there are many people who destroy their marriages because of one-night stands with someone else. And as the French say, "C'est ne pas grave." It's not something easy to swallow, but at the same time, it does not justify you to end a long-lasting relationship because something happened.
I do not see why I should e’er turn back, Or those should not set forth upon my track To overtake me, who should miss me here And long to know if still I held them dear. They would not find me changed from him they knew — Only more sure of all I thought was true.
Nothing that rest on some contradictory basis shall succeed or last in the long run ("ne saurait réussir ou durer, à la longue", Fr.); all that involve (or imply...) a contradiction is fatally destined, early or late, to disintegrate and disappear.