Pi2 + HIFIBerry = HDMI and Audio output

Hey Guys,

Need some help to achive my network. So I connected my Pi2 (OSMC) to my TV via HDMI, and my TV to my amplifier. If I dont want to turn my TV on whenever I want to listen music can I attach a HIFIBerry, and than connect to the 2nd channel on my amplifier? Will it work that way if I watch TV i use pi–>HDMI –>TV–>Amplifier, if I listen music Pi–>Hifiberry–>RCA–>Amplifier? To control I use e.g. Kodi remote controller to get my stuff from cloud.

Thank you guys.