First, the problem with video when switching sounds like HDCP handshaking, and since the 971H doesn't include HDCP it's most likely the display that's having the problem with the 990's switching. You might try turning the TV off and back on when the problem arises.
Second, you will lose nothing by using an HDMI-to-DVI cable and an optical or coaxial digital audio cable to connect the DVD recorder to the 990. The audio output will be Dolby Digital, DTS, or PCM stereo, all three of which are transmitted in exactly the same way whether via HDMI or via optical/coaxial. Also, the Toshiba manual's comment about "digital RGB" would relate to a case of connecting to a DVI display - the 990's DVI switching is transparent when connecting an HDMI source to an HDMI display, so as far as the DVD recorder and TV are concerned it's an HDMI signal path all the way.