Update: I unattached and reattached the hdmi-dvi cables from the tv hdmi1 to the 990 dvi out and the other hdmi-dvi cable from the ps3 to the 990 dvi2.

Settings for video 2 in 990 are video dvi2 and audio optical 3. I did the test display in the ps3 again and got 720p and 1080i fine but 1080p displayed a screen full of gray pixels like when the cable goes out then alternates blinking screens with a screen with pink vertical lines and a black screen. Last time I only saw a black screen when it tested for 1080p.

Anyone have any recommendations? Im thinking of just returning the two cables and getting an hdmi-hdmi cable since I only have one hdmi device.