Apple Music’s “Library” concept is still confusing

Apple Music is much improved, but it is still way too complicated.

Fundamentally it is complicated because it wants to be your cloud library for “your music,” that is, music that you have ripped from CDs or bought from iTunes or whatever, while also being a modern streaming service. That decision has led to the centrality of the “library” concept.

For a pure streaming service, the concept of a library is simply redundant. There is the streaming catalog, and then your playlists. A “library” need be nothing more than a really big playlist (“My 10,000 favorite songs”) or a meta-playlist that just shows you everything in all your other playlists. There’s no reason to introduce another level of hierarchy, and especially not to hang functionality off of it.

For a service that also wants to be a repository of “your” music, maybe you also need a library. But the execution can be improved.

Apple has improved on the library concept a number of ways since Apple Music launched. You can now set it so that adding tracks to a playlist from the Apple Music streaming catalog no longer automatically adds them to your library. Previously, your playlists could only have tracks that are in your library, which is bonkers for a cloud jukebox service. Another improvement is that your own music files now either upload to Apple Music, or match to DRM-free cloud files that you can download free of any restriction on any other computer. Previously, they only matched to entries in the Apple Music streaming catalog, and downloading them was the same as downloading any other Apple Music track–that is, they were DRM’d tracks tied to your subscription. There are real improvements!

Nevertheless the centrality of the library concept still holds Apple Music back. For example, some kinds of metadata only sync between devices for tracks that are in your library. If you create a playlist of tracks, half of which are in your library and half of which are not, and then listen to it 10 times on your iPhone, when you pull up that playlist in iTunes on a Mac, only the tracks in your Apple Music library will have playcounts. That is just silly. Whether or not you have listened to a track is a fact that has nothing to do with whether it is in your library.

Another consequence: in the iOS app, there is now a “downloaded music” section. This is helpful! But if you browse in this section by artist or album you will see, not the artists and albums currently downloaded to your phone, but only the artists and albums downloaded to your phone that are also in your library. So, if you make a playlist of 10 new albums to check out (and why would you want them in your library?) and save it offline, none of them will show up under the artist or albums area of your downloaded music. Those tracks will only be visible if you look under that specific playlist. Again, this is silly: An album is either downloaded to your phone or not. Why are albums that are downloaded to your phone not visible in the “albums” view of the downloaded music section unless they are in your library?

My solution would be this: Your iCloud Music Library should contain only matched and uploaded songs. It should not be possible to add Apple Music tracks to your library. Your library is your music that you brought to Apple Music, and it should not intermingle with Apple Music tracks. When I check out a book from the library, I don’t make room for it on my shelves.

This means that when you search for an album in Apple Music, you shouldn’t see the “Add” button, or the cloud button that you see when Apple Music thinks that album is already in your library. It’s worth pointing out how confusing those buttons are in practice, by the way. Here’s what happens when I look up the album “The Discovery of a World Inside the Moone” by the Apples in Stereo in Apple Music.

I see the “add” button, even though the complete album is already in my library. Why? Because it thinks I’m missing one track, “I Can’t Believe.” Why? When I look at that album in my library, I see that every track but that one matched, and that one track uploaded. Why? I don’t know. I just know that this is an ugly mess. I’d rather not see any ugly “add” or cloud-looking buttons at all or have what I am looking at in the Apple Music catalog change based on what’s going on with my matched and uploaded music. (Of course, behind the scenes Apple should correlate when I listen to matched music to entries in its catalog for the purposes of recommendations. But all that should be invisible to me.)

Finally, when it comes to browsing-type features–where currently I can browse my library by artist or album or song–make it so that I can just browse “My Library” (uploaded and matched tracks) or “My Library and Playlists” (which gives people that want it the ability to build a “collection” of songs they don’t own). This would seem to provide the best of both worlds, and would be conceptually simpler.

main screen turn on

The main way that people pick what to watch on TV is through an on-screen interface of some kind–whether it’s through a cable set-top box’s UI, an app on a device like the Roku, or in a web browser, some kind of UI that is on the screen that is also used for watching the video is the dominant method.

However it is interesting that in recent years a proliferation of new ways to pick what to watch have appeared:


You stream the video from one device to another–from a phone or a PC to a TV. You might be streaming video that is locally stored on device A to device B, or you might be pulling the video down from the internet to device A, then sending it over to device B. In any case device A, your phone or whatever, stays actively involved. If you turn off device A the video stops on device B.

This might involve sending just the raw video stream from your phone to the TV or it might involve capturing the screen of your PC, encoding it, and sending that over as video.


On one device, you pick out a video or stream to watch, and it starts on the other device. But the video is actually directly playing on the other device–for example, your phone just passes a URL over to a Chromecast, and the Chromecast starts playing the video directly from the internet. If you turn off your phone the Chromecast keeps playing.

Now, if the video you are casting is actually stored on your phone, then obviously it’s working more similarly to Airplay. Turning your phone off in that circumstance would be like turning off Netflix’s servers.

Also your phone might be used as a remote control for the Chromecast.


Apps like Spotify and Plex support this. Under this model you initiate playback on one device and it happens on another device, but it’s not just the other device pulling in raw media from the internet somewhere. Rather, this model assumes you have the same app installed on both devices. So, for instance, you control Spotify on device A but playback happens on device B, in the Spotify app for device B.

Voice control:

You are either using voice to control a set-top box directly (for example, with the Apple TV’s Siri remote) or speaking into the air to a device like Amazon Echo or Google Home, which then begins playback on a separate Fire TV or Chromecast-like device.

What is ideal?

The Chromecast isn’t for me. I understand simplicity and cheapness but I don’t like to use my phone to control the TV, and since I often watch TV with my family, a shared, on-screen user interface is best. At the same time, though, I can see the appeal of having cast functionality as an option in a device. Pretty much the same applies to the handoff and Airplay approaches.

The way I think it should work: You begin to Airplay from a phone to a TV. If possible, it switches over to handoff, and if handoff is not possible, to casting. If none of that is possible then it stays in Airplay. This way, to the extent possible your phone is used only as a remote, and you get a native app UI on the playback device.

As for voice control, it’s fine for the 5% of the time I have a clear idea of what I want to watch ahead of time. But I usually don’t and need to browse around. (And the idea of having a conversation with a bot about what to watch sounds very annoying.)

On the one hand it seems like the device that I am talking to should be the same one that is controlling video playback–it is odd to talk to one computer just to control another computer. On the other hand, the voice-listening computer might be best in different place than the TV, you might want to use it while the TV is on, it probably has specialized listening hardware you don’t want on a set-top box, and you might not want your voice computer to have to deal with the vagaries of HDMI-CEC to turn on your TV or stereo receiver.

So, the ideal device would be a traditional set-top box or streaming device that you primarily interact with via its on-screen UI, that may or may not have always-on listening capability but can be somehow controlled by voice, and that can handle casting/airplay/handoff in a fluid and intelligent way. Not very exciting but there you go.