You will not harm the amp. The only time the warning applies is if you are driving both sets at the same time. Turning the volume down or off while cycling through both is good but for the short duration you experience and at normal volumes it shouldn't be a problem.
_________________________
Music system
Model 990/7500/Magnepan 1.6 QRs/Technics SL1200 MK2/Aperion S-12 Subwoofer/OWA3/Sony NS75H DVD
APC H15 Power Conditioner

TV System
Large Advent Loudspeakers/ Polk center/Monoprice surrounds/Panasonic Viera 42 inch/Onkyo HT-RC260/Sony BDP S590/Directv


Home Theater System
Onkyo PR-SC886/Outlaw 7125 Klipsch RF-82 L/R,RC-62 center, RB-35 SR/SL, BENQ HT1075, Outlaw LFM1-EX/OPPO BDP-83/Directv
Harmony ONE
Blue Jeans and Monoprice interconnects
APC H15 Power Conditioner