Most TV shows use the surround channels for a laugh track and/or music. Most of the Dialog is in the center only.
Before I build a center channel speaker, I listened in Phantom mode and got the stereo effect in the front. After I put the center in, I was also a little dissappointed to hear how little sound goes to the other speakers. I sometimes go back to phantom, but not too often now.
For music, I use 3-stereo because the center has a deeper bottom ~50hz than my mains, which don't go deep at all. I don't have a sub, yet.
By all means use your extra speakers as surrounds, then play a DVD in true DD or DTS 5.1. You won't go back.