It isn't the desire to abide by the law that makes everyone behave as society requires, but the fear of punishment. Each one of us carries a gallows inside us.
Criminals, people who commit crimes, usually society rejects these people. They are also part of our society. Give them some form of punishment to say they were wrong, but show them they are part of society and can change. Show them compassion.