Category: video

Stream of thought on improving streaming

Rakesh Agrawal

The streaming revolution has entirely changed the way we consume video content. At our fingertips, in most parts of the world, we have access to large libraries of content.

Way more content than we could ever watch from MVPDs, much less OTA broadcasts. We can watch on-demand content from around the world. By aggregating niche interests and diasporas, it makes it more economically viable to create content. In our house, much of the content we watch is in Hindi and is original content created by Netflix. (Ironically, the Netflix hit Indian Matchmaking wasn’t available in Hindi last I checked. I expect that’s because there is a high degree of English fluency among Indians who watch that show.)

Despite the pandemic-era jokes about having watched all of Netflix (which according to estimates would take more than four years of nonstop viewing), you could never do it because new content is added all of the time.

You can watch something that will be interesting any time of day. You’re not stuck with infomercials if you’re up at 3 a.m.

The big tradeoff has been degradation of the user experience. It used to be simple: want to go forward? Press the forward button. Rewind? Press rewind. Closed captioning? There was a button for that. It didn’t matter what channel you were watching or what program. It all worked the same.

Now each button – if there is even a button – works differently from device to device and from service to service. In some services, pressing skip goes forward 10 seconds; others 30 seconds. Want to turn on subtitles or a secondary audio track? Go digging through layers of menus, which again vary be service. And they might change when the service decides to redesign their app.

I’m an edge case for sure. On the device front, I have Apple TV, Chromecast, Roku, TiVo, Amazon Fire TV, Apple Vision Pro and the streaming services integrated into my Samsung TV. (Probably more in the junk drawer.) For services, I have Netflix, Apple TV+, Hulu, Disney+, Peacock, Max and YouTube. I’m also the only known subscriber to Paramount+.

Multiple services and devices is fairly common. The average American household has four SVOD (subscription videos on demand) services. That’s a lot of complexity and user frustration.

The best user experiences feel invisible to the user. That’s definitely not the case in the streaming world.

There are a lot of ways to improve the streaming interface. (Personalization is a much more complex issue that I’ll visit in a future post.)

Here a few ideas:

Revisit the status bar

It drives me nuts that most video players, including YouTube, show how long a show is.

If something takes up space, it should provide meaningful information. The length of the video doesn’t change no matter how many times I look at it. What does change is how much of the video is left. It’s much more helpful to know that I have 0:35 remaining than that the video is 1:56 long.

Pick QWERTY or ABC

This is one more in the “drives me nuts” category. Pick QWERTY or ABC for searches. It shouldn’t vary when I search on different services. Again, there’s no competitive differentiation here.

My preference is QWERTY because the breaks are the same and ingrained from all of the typing we do. If you use ABC, I have to spend time looking around when you use 3 columns, 4 columns, etc.

I would expect that this has been A/B tested to death, but there is little reason to expect that the audiences would be so different across services.

Theoretically, the faster option would be speech. Unfortunately, speech recognition technology still works poorly in a namespace this broad. (Disclosure: I worked with Alexa speech recognition technology when I was at Amazon.)

Makes fast forward / rewind / skip / pause all work the same

The standard transport controls should perform the same. A skip is a skip. Pause is a pause.

There are two notable exceptions:

  • With AVOD (Ad-supported video on demand), the license might require you to disable skip during commercials.
  • Amazon Prime’s X-ray feature which provides information about the actors in the current scene. (Disclosure: I worked with X-Ray team when I was at Amazon.)

Retain state on closed captioning

When I switch from one service to another, the captioning should stay on (or off.) I shouldn’t have to find the controls (which are different) to turn it on or off. Ideally there would be a physical button.

This is another feature we’ve lost in the transition from a TV-driven interface to app-specific interfaces.

Store preferences for language

A lot of content comes in multiple languages. Some have multiple subtitles available, some have multiple audio tracks, others have both. I should be able to set a preferred language, instead of again having to search through menus. Regardless of what content I choose, the service should pick the language(s) I understand.

Like button

One of the hallmarks of our social world is the Like. Whether it’s a thumbs up, a like or a favorite, it’s an expression that we value something.

Platforms should have a physical Like button on their remote controls. TiVo started with thumbs up / thumbs down buttons on their iconic peanut remote. (Unfortunately the company is a shadow of its former self and has deprecated that functionality.)

Beyond providing data for personalization engines, the Like button could be turned into a social experience. In social co-watching environments, pressing the button could have an icon jump on the screens of the people I’m watching with.

Search by video length

This is especially important for the in-flight use case. I have 45 minutes left in my flight. What can I finish watching before touchdown?

To date, this has mostly applied to IFE systems. But with airlines like Delta and United adding bandwidth that supports streaming, this will become more important to online streaming services.

Even at home, I might want a 30-minute watch before I pick up the kids.

Have searches default to free

If I’m doing a search, I want free. Just because you can sell me a piece of content doesn’t mean I want to buy.

If it’s $5 on VOD, but available on a Subscription Video On Demand (SVOD) I have, I want the free one. You don’t even have to ask.

There are often business reasons/pressures to optimize for the one that generates revenue, but this would be very high up on the list of dark-pattern design. Charging people for things they should get for free has brand and regulatory risk.

Easy temporary connection

Another travel-related use case. More and more hotels are incorporating streaming services into their in-room TVs. Some even have dedicated Netflix buttons on the remote.

The difficulty comes in connecting your account, assuming you even have an account. You should be able to buy a day pass. (The service might even credit the cost of the day pass toward a subscription.)

Better profile handling

Platforms have profiles. Services have profiles. It’s a confusing mess. I can be logged into my Chromecast profile and my spouse’s Netflix profile. Ideally, if I log into my Chromecast profile, it logs into my Netflix profile.

Unlike the other improvements I’ve talked about, this has a lot of hair on it. There might not be a one-to-one matching of services and profiles. Some accounts aren’t shared, others are. Cost and service limitations might restrict some combinations.

All of this leads to poor personalization. My spouse and I share an Audible account. My spouse listens almost exclusively to romance titles; I mostly listen to books on product design, tech and finance. You can imagine what our personalization looks like!

There’s a reason I saved this one for last. Solving this is a lot harder than solving the rest.

Will these things ever happen? I hope so. At the same time, I’m not hopeful. Improving user experience is a cost, with the benefit harder to measure. It’s even more complicated when there are dozens of companies involved. Consumer Electronics Control (CEC) was introduced nearly 20 years ago and barely works today.

In the meantime, I’ll give you the same advice that I give to people who ask me how they should design their video apps: copy Netflix. It may not be the best in every area. But it’s the one that the most users have become most accustomed to.

Update: I asked ChatGPT to write a PRD based on this post. Here’s how it went.

Apple Vision Pro is an amazing device at an amazing(ly high) price

Rakesh Agrawal

Spend even 30 minutes with the Apple Vision Pro and you’ll be impressed. I’ve largely given up trying new hardware at launch because it is hard to set up, buggy and overall so frustrating that it’s not worth the hassle.

But AVP is different. The screens are gorgeous. The scrolling and overall user interface is fluid. (There are definitely things that I would do differently.) It did crash a few times while I was using it.

It’s the best V1 of a new product that I’ve seen in a long, long time. The included demos are amazing. You can feel yourself walking alongside dinosaurs or walking on a rope suspended above water. You can watch a Disney movie inside the Disney theater in Hollywood; the movie is presented on a giant screen around a model of the theater.

(Incidentally, I was part of market research study sponsored by Apple a few years ago. Based on the tests they did and the scans they took of my face, it was most likely for development of the AVP.)

The current headset is roughly 1 1/2 pounds. It’s a bit uncomfortable to wear for more than 30 minutes. Some people have complained that they feel nauseated from the experience, but I didn’t have that issue. (If you get dizzy watching IMAX movies or at Disneyland video rides like Avatar, you might want to sit this one out.)

Apple, true to form, nickel and dimes you. A case is $200. A belt clip is $50. Both are necessary. Without a case, you could easily end up with a $2,500 repair bill. This is not a device you just want to throw in your backpack. You’ll need the belt clip if you want to stand up and walk around.

The biggest challenge, besides the price, is the lack of software. Once I finished with the awe-inspiring demos, I was left with the question “what next?” What can I do that is significantly better than on other devices that don’t cost $3,500? The answer today is not much. Certainly not enough for me to keep the device. (I’ll be returning it to Apple.) But I envision a lot of use cases as the price comes down, the device gets lighter and it becomes more comfortable.

Video

With the gorgeous screens, it’s a great device for entertainment. But only if you like Apple TV+, Disney or Max. Want Netflix, YouTube or Peacock? You’re out of luck. If I’m Netflix, I hold back launching on AVP. Apple needs them. It’s a great leverage point for negotiations to reduce the vig Apple takes on in-app subscriptions. Right now, it’s not like Netflix is going to get additional subscribers by launching on AVP. If you can throw down $3,500 for an AVP, there is a very strong chance you already have Netflix.

Usage of in-home video consumption has changed dramatically over the last decade. It’s no longer immersive. People are frequently multitasking, using second and third screens to check email, engage with social media and play games. Video consumption is also frequently done with other people – each person in a family using a $3,500 headset seems unlikely. It’s not as immersive, but I can buy a 65″ OLED TV, premium Sonos sound system and Chromecast for less than that, and everyone in the room can watch.

I also had trouble drinking while wearing the headset because the can bumped up against it.

Games

Any device this powerful will inevitably be used for games, the screens alone will create an immersive experience.

The big challenge here is that it’s competing with Oculus. There are hundreds of games already available. At 1/10 the price of an AVP. The price dramatically reduces the TAM for game developers. Unless you can port a game from Oculus for next to free, it’s not worth doing it yet.

Another challenge is the lack of controllers. Apple’s pinch interface isn’t suitable for a lot of games. I expect that eventually Apple will support other gestures and controllers.

Training

This is where I see the biggest bang for the buck at today’s price. To the extent that it can reduce the costs or improve the quality of training, it can pay for itself.

Imagine applications like training surgeons or auto mechanics. They could practice in a realistic space without damaging real equipment or needing a lot of cadavers.

Travel

I love to travel. I’ve visited hundreds of cities around the world, awed at the spray and noise of Iguazu Falls in Argentina and hiked to Machu Picchu. Unfortunately, not everyone has the time, resources or physical ability to do that.

Apple has created a maps product that in most ways exceeds Google Maps in the United States. I’m surprised that Apple didn’t create at least a beta version of the 3D views it has captured for Apple Maps. Not only would it show off the AVP, it would show off how much further ahead Apple Maps has gotten.

Work

I’m not a believer in the widely discussed sit-around-in-a-virtual-conference-room use case for AR/VR that is often touted. It isn’t additive. People already tire of Zoom meetings where the only impression on them is a camera. Add in a 1 1/2 pound headset? No way.

But there are plenty of other applications that would actually use the capabilities. Imagine a surgeon seeing an overlay of an MRI on her patient in the OR. An air traffic controller guiding planes in real time with a 3D view of aircraft and terrain. (I 😂 as I write this given how ancient the technology FAA uses is, but a guy can dream.)

More immediate use cases:

  • Architects placing renderings of a building in a downtown to show to zoning boards.
  • Urban planners showing residents what a new park would look like.
  • Utility workers identifying buried power lines.
AR view of a flight tracker app on Apple Vision Pro.

Porn

(Feel free to skip this section; I include it for completeness.)

Let’s face it – porn will be a common use for the AVP. Porn has driven the adoption of many emerging technologies, including the VCR, DVD players and even the Internet itself. I expect point-of-view porn to be especially compelling.

Because it is a closed ecosystem, you’re unlikely to get native porn. Just like with iPhone and iPad, Apple won’t allow developers to create porn apps. For now, the Web browser or Photos app will have to suffice.

Although most Web pages are 2D, there are emerging toolkits that will allow 3D. The Photos app in AVP displays 3D spatial photos and videos. If I were shooting porn today, I’d use an iPhone 15 Pro to capture spatial video so that I have a library ready to go as adoption of the AVP increases.

I can envision so many other possibilities for the Apple Vision Pro. If we get even half of them, Apple will has pushed the world forward.

No, Netflix shouldn’t launch its own streaming box

Rakesh Agrawal

It’s a little early for that and it might turn out to be unnecessary. There’s too much content that isn’t available streaming. As much as I would like to stream everything, I find that a lot of what I want to watch isn’t available. 

Netflix has made acquiring a box so easy that you can do it almost by accident. Buy a DVD player, Xbox, Playstation, Wii… chances are you already have Netflix streaming. It also saves the hassle of setting up another box and having another remote control — which are significant challenges when it comes to the general population.

Creating a proprietary box would create competition with companies that have distribution already. Why would Roku continue to distribute Netflix if they were competing with Netflix on hardware?

Netflix is also trying to use pricing to encourage consumer adoption. Plans with DVDs are more expensive than plans without. Netflix is also using recommendations and UI to shape demand for content. (I’d love to know Netflix A/B testing results on whether to show “DVD Only” when people search for content on streaming devices.)

If I were are at Netflix and I wanted to experiment with sending out boxes, here’s what I’d do:

  1. Identify which users do not have streaming devices. (No need to subsidize those who don’t need it.)
  2. Among those users, identify users who are primarily ordering DVDs whose content is also available on Netflix streaming. (They have a substitute available.)
  3. Among those users, identify those who are frequently ordering DVDs. (Cost savings of not mailing to them is material.)
  4. Send them email offers with varying discounts for a Roku box, fulfilled either by Roku or by Amazon.