You can hold yourself back from the sufferings of the world, that is something you are free to do and it accords with your nature, but perhaps this very holding back is the one suffering you could avoid.
I don't like standing near the edge of a platform when an express train is passing through. I like to stand right back and if possible get a pillar between me and the train. I don't like to stand by the side of a ship and look down into the water. A second's action would end everything. A few drops of desperation.