One particular thing to be careful of are core dumps.
What I did at a previous shop was remove the passwords as part of a smart gdb script that runs when the core is dumped, before it gets written to a readable location.
Writing the script also helped to demonstrate how to extract the passwords in the first place.
Stack traces, too. I did some work with a heavy Java shop and pretty much everything sensitive ended up in a stack trace at some point.
Java is just too verbose in every possible way.