You might want to give it a try, but don't expect it to work. I bought a manual, press-button type switcher from Monoprice, for just this purpose, and it wouldn't work correctly. It ended up changing some of the colors, and the electronics just wouldn't play nice.
I love the idea of HDMI, but there's way too much background stuff going on i the handshake process to make even the most simple-seeming ideas work.
Scott