I've only ever blown speakers when I've run less power than I should have. I'm much more comfortable running gobs more power than I need, and listening carefully for any hint of distortion resultant from the speakers reaching the end of their travel. Really, if you are reasonable about your volume and aren't trying to fill the room from your bass amp, then the extra power will get you lots of clean for your sound that isn't available from a smaller amp. A bigger amp won't make your cabinet louder, but it will make it cleaner, in my experience.
John