Most speakers are 'ruined' by underpowering them rather than overpowering.

What's the scientific basis for that claim? Clipping can damage speakers, but simply giving them less power than they may require for full performance isn't going to hurt anything.

I'd say if your current amp isn't straining and levels you typically listen to and there's no evidence of clipping then no reason to spend more money.