Yeah, basically, just encrypt and leave it at that. Just make sure it's a good digital lock and NOBODY has your key. (There ARE ways for somebody to get you to give them your key.)

The problem, I feel, is that, even though sensitive data is encrypted, to NOT get rid of it still leaves it THERE. A worry. Unwarranted, true. But a thorn in my side.