Tag: apple-vision-pro

A PM’s view of Apple Vision Pro

Rakesh Agrawal

See also: my impressions of Apple Vision Pro and VR/AR in general.

The Apple Vision Pro is the best V1 of a product I’ve seen in a long time. But whenever you’re creating a completely new visual and input interface, you’re going to have some polishing to do.

There are some issues that are just the state of new technology and will get better over the course of time: weight, comfort, cost and stability being at the top of the list.

Eye tracking and gesture tracking is as good as I’ve seen, but it still needs improvement. This problem is exacerbated by the very subtle distinction among selected items. More on that below.

Some have complained that the AVP can cause nausea. I haven’t had that issue. The issue I have run into is the repeated use of the pinch gesture. It can cause my hand to get tired and I fear getting RSI. I had to take a break, not because of the weight of the headset but because my hands got tired. (This may also be an artifact of testing; if I were just watching a movie, this wouldn’t be an issue.)

Putting on my product manager and usability hat, here are the top new features and fixes I would make:

New features

Multiuser support

A $3,500 device should provide multiuser support. I can’t easily share it with my family. At a minimum, I should be able to switch among Apple Family Sharing users. AVP does have a guest mode, but it requires reconfiguration each time.

Importing iPad apps

As with all new hardware, initial software is going to be limited. Apple has some great demos and a few apps built by third-party developers that take full advantage of the AVP interface and features. But you’ll burn through them quickly.

AVP also supports iPad apps. They don’t provide the same rich experience as native apps, but they do provide valuable features. Notably, most of the frequently used Apple apps haven’t been re-written for AVP.

AVP allows you to go through the App Store and manually select iPad apps. It would be better if it provided a list of your most frequently used apps to add them with a few clicks.

Automatic free trials

Out of the box, users should have 7- to 30-days of free access to all of the AVP optimized apps. This would give them the ability to really experience the power of the platform.

I had the same issue with Oculus. In order to fully understand the device, I had to buy a lot of games. I wasn’t going to do that.

Demo/training mode

Unlike most products, it’s hard to teach someone to navigate the interface. AVP already has a screen mirroring feature that allows someone else to see what the AVP pro is seeing.

When I’ve shown friends how to use the AVP, I’ve had to say “look all the way to the right, look down, see that?”

Add a “laser pointer” to that mode. Instead of having to talk the directions, I could have a pointer appear on the screen to guide them. You could all add a reverse pointer, where the trainer could see where the eyes are looking.

Fixes and tweaks

Setting up AVP

For initial setup, the AVP requires that you hold your iPhone near it to download account credentials. It’s unclear how far you need to hold it. I found myself repeatedly moving my arm forward and backward while trying to pair it.

Sample content

When you try AVP in store, there are some gorgeous pictures that are shot for purpose. They show off spatial video and photos, panoramas and other features of the device. At the risk of pulling a U2, those sample images and videos should be included in the Photos app. (Possibly in a folder labeled Sample Content, in much the same way that Windows included sample pictures.)

My initial reaction when I saw those pictures: I need to buy an iPhone 15 Pro so that I can capture spatial photos.

Too little differentiation between items when selecting

This is one of the biggest usability issues. The difference between selected and non-selected items is very slight. For a device that requires looking at an item to pick it and where you don’t have precision control like with a mouse, this is a big problem. It’s especially a pain when using the on-screen keyboard. (See more below.)

I thought this would be changeable in accessibility settings, but I couldn’t find it. Regardless, the default differentiation needs to be greater and some users would benefit from being able to set it even higher.

Keyboard

Hate the on-screen keyboards where you have to navigate with a remote control to enter data? The AVP’s virtual keyboard is at least 10 times more difficult than those. It took me 5 minutes just to enter my Disney+ login information. There were too many misread keystrokes.

For AVP to be a content production device, the keyboard needs to be much better. Yes, you can pair a physical bluetooth keyboard, but that’s yet another accessory to carry with you.

Top left menu item is really difficult to access

AVP has a side panel that is used to navigate among key controls. Selecting the top item was very difficult. Given that the item is “Applications,” this really needs to be improved.

I initially thought it was an eye tracking problem for me, but I had a friend try it and he had the same issue.

Control Center too hard to get to

Similar to the above issue. As with iPhone, the Control Center comes down from the top. The way it is supposed to work is that you look up and can select it. Frequently, I put my head back as far as my neck would allow and I still couldn’t get the Control Center tab to come up. Again, my friend had the same issue.

At other times, it’s a gnat that I can’t swat away.

Lack of progress indicator when launching apps

Sometimes there is a delay when trying to launch an app. Because the background is a passthrough of what your eyes are looking at, it feels like something crashed or that the app didn’t launch. A spinner or loading indicator would make this a more comfortable experience.

Universal transport controls

This isn’t AVP specific, but it has long been on my video wishlist, and with a change to a completely new environment, maybe I can finally get it.

I want rewind, fast forward, play, pause, skip, go back and other transport controls to work the same regardless of which video app I’m in.

I want the numbers on the right side to reflect how much time is left in the video I’m watching. (Showing the total duration isn’t very helpful. It’s also not a good use of space: the number doesn’t change.) When I press skip, I don’t want it to go forward 5 seconds in one player, 15 seconds in another and 30 seconds in another.

My cable TV and DVR remotes didn’t work differently depending on what channel I was watching. This is the same. Transport controls aren’t a competitive differentiator. Unless there is an app-specific feature (like Prime Video’s X-ray), they should work the same.

The only exception is the skip feature during ads. That control can be disable during an ad and not shown on ad-free services.

Apple Vision Pro is an amazing device at an amazing(ly high) price

Rakesh Agrawal

Spend even 30 minutes with the Apple Vision Pro and you’ll be impressed. I’ve largely given up trying new hardware at launch because it is hard to set up, buggy and overall so frustrating that it’s not worth the hassle.

But AVP is different. The screens are gorgeous. The scrolling and overall user interface is fluid. (There are definitely things that I would do differently.) It did crash a few times while I was using it.

It’s the best V1 of a new product that I’ve seen in a long, long time. The included demos are amazing. You can feel yourself walking alongside dinosaurs or walking on a rope suspended above water. You can watch a Disney movie inside the Disney theater in Hollywood; the movie is presented on a giant screen around a model of the theater.

(Incidentally, I was part of market research study sponsored by Apple a few years ago. Based on the tests they did and the scans they took of my face, it was most likely for development of the AVP.)

The current headset is roughly 1 1/2 pounds. It’s a bit uncomfortable to wear for more than 30 minutes. Some people have complained that they feel nauseated from the experience, but I didn’t have that issue. (If you get dizzy watching IMAX movies or at Disneyland video rides like Avatar, you might want to sit this one out.)

Apple, true to form, nickel and dimes you. A case is $200. A belt clip is $50. Both are necessary. Without a case, you could easily end up with a $2,500 repair bill. This is not a device you just want to throw in your backpack. You’ll need the belt clip if you want to stand up and walk around.

The biggest challenge, besides the price, is the lack of software. Once I finished with the awe-inspiring demos, I was left with the question “what next?” What can I do that is significantly better than on other devices that don’t cost $3,500? The answer today is not much. Certainly not enough for me to keep the device. (I’ll be returning it to Apple.) But I envision a lot of use cases as the price comes down, the device gets lighter and it becomes more comfortable.

Video

With the gorgeous screens, it’s a great device for entertainment. But only if you like Apple TV+, Disney or Max. Want Netflix, YouTube or Peacock? You’re out of luck. If I’m Netflix, I hold back launching on AVP. Apple needs them. It’s a great leverage point for negotiations to reduce the vig Apple takes on in-app subscriptions. Right now, it’s not like Netflix is going to get additional subscribers by launching on AVP. If you can throw down $3,500 for an AVP, there is a very strong chance you already have Netflix.

Usage of in-home video consumption has changed dramatically over the last decade. It’s no longer immersive. People are frequently multitasking, using second and third screens to check email, engage with social media and play games. Video consumption is also frequently done with other people – each person in a family using a $3,500 headset seems unlikely. It’s not as immersive, but I can buy a 65″ OLED TV, premium Sonos sound system and Chromecast for less than that, and everyone in the room can watch.

I also had trouble drinking while wearing the headset because the can bumped up against it.

Games

Any device this powerful will inevitably be used for games, the screens alone will create an immersive experience.

The big challenge here is that it’s competing with Oculus. There are hundreds of games already available. At 1/10 the price of an AVP. The price dramatically reduces the TAM for game developers. Unless you can port a game from Oculus for next to free, it’s not worth doing it yet.

Another challenge is the lack of controllers. Apple’s pinch interface isn’t suitable for a lot of games. I expect that eventually Apple will support other gestures and controllers.

Training

This is where I see the biggest bang for the buck at today’s price. To the extent that it can reduce the costs or improve the quality of training, it can pay for itself.

Imagine applications like training surgeons or auto mechanics. They could practice in a realistic space without damaging real equipment or needing a lot of cadavers.

Travel

I love to travel. I’ve visited hundreds of cities around the world, awed at the spray and noise of Iguazu Falls in Argentina and hiked to Machu Picchu. Unfortunately, not everyone has the time, resources or physical ability to do that.

Apple has created a maps product that in most ways exceeds Google Maps in the United States. I’m surprised that Apple didn’t create at least a beta version of the 3D views it has captured for Apple Maps. Not only would it show off the AVP, it would show off how much further ahead Apple Maps has gotten.

Work

I’m not a believer in the widely discussed sit-around-in-a-virtual-conference-room use case for AR/VR that is often touted. It isn’t additive. People already tire of Zoom meetings where the only impression on them is a camera. Add in a 1 1/2 pound headset? No way.

But there are plenty of other applications that would actually use the capabilities. Imagine a surgeon seeing an overlay of an MRI on her patient in the OR. An air traffic controller guiding planes in real time with a 3D view of aircraft and terrain. (I 😂 as I write this given how ancient the technology FAA uses is, but a guy can dream.)

More immediate use cases:

  • Architects placing renderings of a building in a downtown to show to zoning boards.
  • Urban planners showing residents what a new park would look like.
  • Utility workers identifying buried power lines.
AR view of a flight tracker app on Apple Vision Pro.

Porn

(Feel free to skip this section; I include it for completeness.)

Let’s face it – porn will be a common use for the AVP. Porn has driven the adoption of many emerging technologies, including the VCR, DVD players and even the Internet itself. I expect point-of-view porn to be especially compelling.

Because it is a closed ecosystem, you’re unlikely to get native porn. Just like with iPhone and iPad, Apple won’t allow developers to create porn apps. For now, the Web browser or Photos app will have to suffice.

Although most Web pages are 2D, there are emerging toolkits that will allow 3D. The Photos app in AVP displays 3D spatial photos and videos. If I were shooting porn today, I’d use an iPhone 15 Pro to capture spatial video so that I have a library ready to go as adoption of the AVP increases.

I can envision so many other possibilities for the Apple Vision Pro. If we get even half of them, Apple will has pushed the world forward.