The main way that people pick what to watch on TV is through an on-screen interface of some kind–whether it’s through a cable set-top box’s UI, an app on a device like the Roku, or in a web browser, some kind of UI that is on the screen that is also used for watching the video is the dominant method.
However it is interesting that in recent years a proliferation of new ways to pick what to watch have appeared:
You stream the video from one device to another–from a phone or a PC to a TV. You might be streaming video that is locally stored on device A to device B, or you might be pulling the video down from the internet to device A, then sending it over to device B. In any case device A, your phone or whatever, stays actively involved. If you turn off device A the video stops on device B.
This might involve sending just the raw video stream from your phone to the TV or it might involve capturing the screen of your PC, encoding it, and sending that over as video.
On one device, you pick out a video or stream to watch, and it starts on the other device. But the video is actually directly playing on the other device–for example, your phone just passes a URL over to a Chromecast, and the Chromecast starts playing the video directly from the internet. If you turn off your phone the Chromecast keeps playing.
Now, if the video you are casting is actually stored on your phone, then obviously it’s working more similarly to Airplay. Turning your phone off in that circumstance would be like turning off Netflix’s servers.
Also your phone might be used as a remote control for the Chromecast.
Apps like Spotify and Plex support this. Under this model you initiate playback on one device and it happens on another device, but it’s not just the other device pulling in raw media from the internet somewhere. Rather, this model assumes you have the same app installed on both devices. So, for instance, you control Spotify on device A but playback happens on device B, in the Spotify app for device B.
You are either using voice to control a set-top box directly (for example, with the Apple TV’s Siri remote) or speaking into the air to a device like Amazon Echo or Google Home, which then begins playback on a separate Fire TV or Chromecast-like device.
What is ideal?
The Chromecast isn’t for me. I understand simplicity and cheapness but I don’t like to use my phone to control the TV, and since I often watch TV with my family, a shared, on-screen user interface is best. At the same time, though, I can see the appeal of having cast functionality as an option in a device. Pretty much the same applies to the handoff and Airplay approaches.
The way I think it should work: You begin to Airplay from a phone to a TV. If possible, it switches over to handoff, and if handoff is not possible, to casting. If none of that is possible then it stays in Airplay. This way, to the extent possible your phone is used only as a remote, and you get a native app UI on the playback device.
As for voice control, it’s fine for the 5% of the time I have a clear idea of what I want to watch ahead of time. But I usually don’t and need to browse around. (And the idea of having a conversation with a bot about what to watch sounds very annoying.)
On the one hand it seems like the device that I am talking to should be the same one that is controlling video playback–it is odd to talk to one computer just to control another computer. On the other hand, the voice-listening computer might be best in different place than the TV, you might want to use it while the TV is on, it probably has specialized listening hardware you don’t want on a set-top box, and you might not want your voice computer to have to deal with the vagaries of HDMI-CEC to turn on your TV or stereo receiver.
So, the ideal device would be a traditional set-top box or streaming device that you primarily interact with via its on-screen UI, that may or may not have always-on listening capability but can be somehow controlled by voice, and that can handle casting/airplay/handoff in a fluid and intelligent way. Not very exciting but there you go.