Chapter 19 and 20:
The story of the suicide of Lucretia is well known (and if you don't know it, Collin Garbarino has a great summary here). Augustine uses this popular story to set up a problem for the reader: either Lucretia was innocent in the sexual assault and so murdered an innocent; or she was guilty of adultery and deserved to die, and therefore doesn't merit the hero-status the Romans have attributed her.
Obviously, Augustine says, we don't think that she was complicit in her own rape, and consequently we do believe that she was innocent of adultery but guilty of murder. We can understand that she felt shame because of what happened to her, but as Christians we also hold that the problem of shame cannot be solved by sinful means. We see this in the example of the Christian women who have been raped: "They declined to avenge upon themselves the guilt of others, and so add crimes of their own to those crimes in which they had no share. For this they would have done had their shame driven them to homicide, as the lust of their enemies had driven them to adultery. Within their own souls, in the witness of their own conscience, they enjoy the glory of chastity."
The problem is, the world says that sin must have some recompense. The woman who has been raped (or the man in the same circumstance, I suppose) feels soiled, though we all see that she is not. And of course a great evil has been committed, so there is an actual sin out there that in Augustine's day was certainly going to go unpunished by the Roman Courts (it's not as if they could drag the barbarians in before them!). So much we can all agree upon, Christian and non-Christian alike. And I think that we can go one more step and say that both Christian and non-Christian will agree that in these cases two wrongs do not make a right. Adding yet another sin to the mix will not solve either the soiled feelings or the reality of the wrong that has been done. And while we might disagree over whether or not suicide is a sin (as Augustine argues so forcefully in Chapter 20, the Christian is against killing any rational being), we can all admit that 1) something should be done; 2) that something should not be another evil.
So where does that leave us? With the split between the Christian and the non-Christian. The non-Christian is going to search for justice in the city of man. The courts, the laws, the methods and processes of the time will be turned to in an attempt to remove the soiled feeling from the victim and to punish the guilty party. The Christian, on the other hand, while fully supporting the attempts of the city of man to pursue justice and healing, realizes that such things will never fully be found in this world and instead turns his eyes to Christ, in whom alone justice and mercy can be found that are completely sufficient.