I always preferred forgetting to forgiving, and then only after retribution (or outright revenge) is obtained. Balance the books, settle the debt, and move on. As for "the forgiven secure in knowing harmful things are excusable", I would add the forgiven is also secure in knowing harmful things will not be forthcoming, and his actions will carry no painful consequences.
While some people forgive, I doubt that they ever truly forget. There may be lapses of memory as time progresses due to perhaps the one forgiven is not near. However, one's thoughts do come back to haunt one from time to time. So to speak. When we demand a debt be paid to society it is not revenge, but rather getting even. Again, is the guilty exonerated. Depends on one's state of mind.