It could be done, but not easily.

All the data in HDMI (audio and video) is digitally encoded. The audio data is interleaved with the video and control data (packets) and sent to the receiving device. The clock signal sent over HDMI allows the receiver to sync up with the data bits BUT this clock CAN'T be used to play the audio. The HDMI receiver uses a process called Audio Clock Regeneration to retrive the audio clock from the HDMI clock.

There's chips availalbe to do all this.