When you say you are utilizeing the un-used channels, I assume that you are splitting the audio signal to feed to the seperate amp channels.
How are you splitting the signal??
You may need to use a buffering device.
Niles Audio, AVDA-3
Audio-Plex, Y-F2,Y-M2