Productivity is correlated with display size, and VR should be the next step in productivity, because it allows your whole space to become a display and an interface. When paired with powerful hardware and innovative software it should be the next step in computing.
Therefore, VR should be a replacement for large displays in productive work, not a device that everyone is wearing outside or casually watching videos with. The use cases are professional; such as programming, CAD, design etc. I believe that properly implemented VR would allow utilizing full visual-spatial capabilities of our brains, totally eliminating the process of hunting for files and switching windows.
It would be possible to dive into code and see it all at once like an intricate factory. Our brains are capable of navigating very complex environments, but this capability is lost when its viewed through a small window.
I wish they'd explore this area, rather than go for a mass market appeal. What kind of software is possible, and how much productivity can be increased with maxed out hardware with high resolution VR? I'm not sure if 4K resolution with a M2 chip is powerful enough for a truly innovative product. Maybe it is, but there's still room for a "pro" product.
I don't think there's a use case for glasses-type product, with a watch-like functionality and environmental AR which would show information pop-ups everywhere. We already have information overload, and we don't need any more of that.
It seems to me if there was even a correlation, which I'm not sure, it would have more to do with resolution, since same resolution but bigger display doesn't actually show more information.
you just have to ignore that at the extremes it's probably negatively correlated, and there's a significant non-zero possibility that at least in the short term, that possibility of negative correlation applies to the area VR is interested in as well.
That's a dubious statement. Filling the binocular field of view is fine; it's about 120° horizontally, give or take. But past that it starts to be a problem. Generally, if you look at the screen and any edge is blocked by your nose without turning your head, it's likely too large to work comfortably on. (but still fine for immersive content like games or movies)
You can turn also turn your head and move around in your room, i.e. your whole room could be full of separate displays, or just pieces of code floating in space. It's not just how much you see at a time, but how easily you can switch context.
Typical large (32") monitor with normal spacing is about 30-35deg vertical and 60-70deg horizontal and 27" is usually less. If your eyes are 20" from the screen, it's probably not healthy. Binocular field of view (without head motion) of VR systems is usually 100deg in both dimensions (20deg overlap in the middle). I'd expect Apple VR to be "retina".
I'm bullish on VR but I think the "many floating monitors around a desk" is a bad use case. I mean, you can just alt-tab through windows or desktops as it is.
In my mind the real power comes from spatial hand tracking as a new UX paradigm(seeing an interacting with objects in 3d space) and possibly the addition of large displays anywhere a monitor couldn't have feasibly been, (eg, floating in the server room in a cramped space when needed).
I don't think working on code will be all that different. There isn't any physical spatial interaction with text on paper that the current editors are missing.
I only see this as a supplemental feature for certain use cases (coarse-grained 3D operations). In general nothing beats the efficiency of working with a keyboard for navigating application functions, or (at least with current technology) the mouse for precision 2D navigation/selection. Their advantage being precise and definite/discrete input.
For actual work, even with a VR headset I see myself still mostly using keyboard and mouse.
I agree that "floating virtual monitors" isn't much of an improvement over real monitors - but it's an transitional step that you take to experiment with building a UI that's not possible with traditional physical interfaces.
Perhaps, in a lab setting, doing straightforward, focused tasks like the ones they use in these productivity studies, I could sustain a productivity boost long enough for the researcher to get a two-tailed p<0.05.
Make me do it in real life, though, 40 hours a week, week after week after week, and I expect I would burn out very quickly. I like my work well enough, but I don't think I want to actually stick my whole head inside of it.
Maybe not all the time, but it would be great for me to be able to travel and not have to worry about taking a huge productivity hit without my desk and monitors.
It definitely does not scale linearly. A second monitor is a big advantage. A third less so. Infinite visual space would be barely more valuable than that. And really just having all that screen real estate isn't useful unless it's matched by extremely high resolution, frame rate and field of vision. I can flick my eyes between two monitors but any headset on the market is going to make me turn my head while squeezing my temples.
I can get behind this. I always though VR was a gimmick but I got an oculus ~2016 and was honestly blown away. However, the #1 thing that turns me off to VR is that it's uncomfortable. That's ok though as long as the experience is worth it.
I truly believe that VR is perfect for training/sim type games. Actual games.. Could really only count on one hand the games I feel are worth diving into VR for.
Andrews, C., Endert, A. and North, C., 2010, April. Space to think: large high-resolution displays for sensemaking. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 55-64):
There’s much more. Search “sensemaking” or “immersive analytics” on Scholar. Depth and width of field, achieved via stereoscopy and tracking, is well documented as a performance enhancer when it comes to generalizeable task completion in VR.
Same. I tried the “dual monitor” setup for a couple years and just went back to one. The second was just causing distractions and context switching. It’s easier to focus on what’s right in front of you than look around for stuff.
And my work involves working with 2-4 ‘documents’ at once, analyzing, comparing, and summarizing them. My focus space is a little larger than one window.
Both Meta and Microsoft are exploring AR/VR in the workplace (Microsoft pretty much entirely). I’m not sure why you’re suggesting they’re not exploring this area.
Don't forget even the home use case where you're much less likely to have a pair of 30" monitors or even the space to put them... nor the IP concerns of the company, if you did have them in your living room. I agree, for apple this is most likely to be a creative/productivity play at high dollar. The challenge will be pricing and multi-hour usability (weight/watts/weariness).
You get retina resolution, high refresh rate, HDR, and 2Factor authentication (Biometric+Goggles)/information security in one package. Of course you'll still likely want to have a laptop to drive it (untethered) except for the simplest mobile games.
I agree that VR/AR is the future of productivity computing, but I think we are a bit far off still, half a decade at least. I have played around with remote desktop apps on the Meta Quest 2 and these are the main challenges I see:
- 4k is not nearly enough. I think we will likely need 8k screens.
- The headsets are not light enough. You need something with all day comfort for work.
- There is still too much eye-strain from the quality of the lenses and the fixed focus.
- Input is still not natural and precise enough. We would need to be able to see our own hands and type with zero latency. Hand-tracked pointing needs to be able to pick up on small subtle finger motion like a mouse or trackpad can
Just common sense. Watch -> Phone -> Tablet -> Laptop -> Desktop. You'd be least productive on a watch and most productive on a desktop, and adding extra displays would make you even more productive. Each of them has an interface that's designed to max out productivity on that particular display size. We don't yet have an interface that maxes out productivity on a VR device.
It would be possible to dive into code and see it all at once like an intricate factory. Our brains are capable of navigating very complex environments, but this capability is lost when its viewed through a small window.
People can learn 3D environments by viewing them in a 2D screen very happily, and millions of gamers do exactly that every day. I can still remember my way around the map in GTA 3 about 20 years after I first played it...
Gaming environments are more like 2.5D. There aren't many games which are fully open in 3D, because the world isn't, and humans can't fly.
The "flight" in GTA is more like a moving viewpoint than a space full of 3D objects and affordances.
Full immersive 3D navigation through code and file systems would be an interesting experiment. I suspect we'd get used to it very quickly, and we'd then wonder why we spent so long tied down by virtual paper metaphors. (I could be wrong, of course.)
> It would be possible to dive into code and see it all at once like an intricate factory. Our brains are capable of navigating very complex environments, but this capability is lost when its viewed through a small window.
This is such an Engelbart-esque statement. Bravo! I'd love for AR to go there too :)
In case VR you are right, but AR, with current state of software might be good for navigation only. Later on, every physical thing can have its own augmentation, GUI, either by something like QR code or Bluetooth. In a bus stop you could buy a ticket not by using dedicated point of sale device but by using augmentation software in glasses. Posters could change into trailers etc.
> Therefore, VR should be a replacement for large displays in productive work, not a device that everyone is wearing outside or casually watching videos with. The use cases are professional
I hope you’re wrong about this.
I hope there will be a great personal (non-work) use case for the next generation of VR/AR. Something like gaming, watching sports (always have awesome “seats”), movies with friends, maybe map/gps/etc. for when I’m running.
I know there are options for this already. But honestly, I haven’t seen anyone who’s not a dork using VR/AR. I want the version the average person is excited about. I’m ready to plunk down any amount of money Apple wants from me.
If it’s VR/AR for work, count me out. I don’t have any interest in increasing my work productivity lol.
Social experiences are the use case you're looking for. Multiplayer gaming is a lot more fun in VR, watching video together with people is great, even just hanging out and socializing is great and it beats video or voice hands down. You're right that it's currently in the dork stage of development, the internet went through that period as well. I'd enjoy it while it lasts though, I dread the day when it's fully monetized and mainstream.
I had a Quest 2 and honestly thought it was at it best when I was lying down and watching videos in bed on a huge screen. All the other use cases I just didn’t have a good space for
Have to say I agree with Jony Ive on all of these points.
The idea of an AR wearable that is tethered to a fixed base station is so very lame. Why even bother with AR when you can only augment the reality that is in your office or bedroom?
Also the eye-view outer display is essential for situations where one person is using the goggles but others aren’t. How unsettling would it be for someone to look at you, be able to see you, but for their upper face to completely obscured by goggles? How could you have an in person face-to-face conversation with someone like that?
The fixed base station is a fundamentally different tradeoff for AR vs VR.
Base station for AR would be detrimental, as you inherently want to explore and interact with the world around you.
For most VR use cases that is not the case, and I can see the argument for the benefits of a base station (higher performance) outweighing the costs (limited movement).
One solution to this for AR is using "fog computing", where your AR device may offload its computation to a node on the local network that's more powerful but also not as far away (in terms of latency) as a cloud server. Perhaps with the continued adoption of UWB 5G, those AR basestations could exist at cell towers or in local establishments - think about a clothing store running fog nodes to allow AR users to visualize wearing different clothes as they see them in the store.
I haven't read more than the abstract, but it seems there is some good research going into this. It must be years off however, as we barely have LTE networks in some locations, let-alone UWB 5G to even begin supporting this type of architecture.
Apple won’t release this product unless it is mind blowing. If Bloomberg’s reporting is true, then we should all be looking forward to a very cool demo at WWDC and an incredible product some time in 2023.
Very few people predicted the iPhone accurately before its launch. And it’s reasonable to say the iPhone blew minds because Apple figured out multitouch AND reduced the latency of the interface to the extend required so that it felt natural to use. If they mail AR in the same way, they will have the next iPhone.
I feel the same way but it’s also worth remembering that while the first iPhone was a hit amongst early adopters, it didn’t really become ubiquitous until the 4 or 4S, which were also vastly superior devices which had a few years of iterations based on experience of early adopters. The same is true of the Apple Watch, which also blew away the competition on first release, and was still called a failure but was a sleeper hit, by itself a larger business than most of Apple’s competitors.
I expect Apple’s AR product to be functional, and dubbed a niche failure by the tech commentariat, but by 2026 really start to eat into the smartphone market.
Meta and Google are just less capable of making humane software, and while they may release slightly better devices from a technical perspective they’ll miss out on the light touch that such an immersive experience will require for normal people to feel comfortable with it.
I think the iPhone isn't the only possible comparison here. Sure, the iPhone blew minds, but what about the iPad? Apple Watch? Airpods? Apple TV? None of these are bad devices (in fact, they're all pretty solid) - they're just niche devices and limited in the way the iPhone wasn't/isn't. I'd expect the headset to be similar.
I'm not so sure. I think they will deliver something but I have a feeling that Meta is going to beat them to the punch as far as a productivity focused device is concerned. Meta might wind up with most of the market share while Apple trails them although I'm sure that most people who own a MacBook Pro will buy an Apple headset as well. I think this is more of a PC vs Mac situation than it is Android vs iPhone.
I would be shocked if Meta can convince the masses to buy their headset over Apple's. If Meta tries to beat Apple on price, their headset won't be very capable and won't be much different than what is already out there (which the masses aren't interested in). If they try to compete with Apple by releasing a similarly priced and similarly capable headset, Apple wins every time. I think you're underestimating how much people dislike Meta/Facebook/Zuck. And I'm not just referring to techies either. "Normal" people hate them too.
meta has the head start, but never bet against apple for revenue. VR is still new for casuals. if apple releases new hardware people will buy it and you'll see an explosion of devs creating apps for it.
It’s so telling how much WWDC has been dedicated to ARKit over the past many years yet not a single viable use has been shipped.
AR through a phone or iPad is lame and close to useless. Only looks good in video demos till you use it yourself and see how clumsy it is interacting through a tiny window.
Very clear they have a headset in mind for all this but are struggling to ship.
I'm excited overall about these 'leaks' and details surfacing from Apple and Meta. I'm optimistic the AR side will win (versus VR only), but overall, by being shepherded by these big orgs, I expect hardware costs to reduce dramatically for end users. Lower costs will pull in more users and allow us to really design experiences for a wider AR audience.
Designers, like Ive, should not be left unchecked. As part of an overall team they have value, but they should never be given carte-blanche as Ive had.
I was surprised to find myself agreeing with the point of view ascribed to Ive.
My takeaway though is that the device itself and how it should look/work is not obvious, and that is a bad thing.
For a device like the iPhone, iPod a few people kicking around ideas over sushi could very quickly (and with wild enthusiasm, no doubt) come to agree on the basic form, basic functionality.
If our VR-future has to be shoe-horned into existence, maybe it was not to be.
If anything, I want to see "the cyber" go away, not become more pervasive.
On the other hand, no one has released a VR system with mass market appeal. I have and enjoy a Valve Index, but it’s stayed in the box for more than a year because I can’t figure out a socially viable spot for it in my new house. Maybe Ive’s influence on this product will make a big difference.
The Oculus quest 2 definitely has fit that bill. It has game console level sales. How truly useful and confirtable it is may be questionable, but it certainly supports a market.
VR seems to have an insurmountable motion sickness problem. I tried the Quest with a dozen friends and half of them felt sick enough within minutes to never try it again.
Indeed, it's clear that Ive hasn't been a positive force on Apple's products for a while. His pathological fetish for thinness at all costs resulted in objectively inferior products.
Apple can and does make mistakes, for example the touch bar, and the MacBook keyboard.
If Ive is involved I predict that this product will experience a brief but intense surge in popularity based primarily on the novelty factor and social media influencers followed by a rapid decline into irrelevance.
Sounds like a good solution for a low probability situation. Eye visibility for normal humans is much more practical than issuing every human a VR system and requiring they wear it at all times to maintain the illusion.
The cost of the external eye displays seem absolutely crazy.
Two extra external displays in raw hardware. But worse, they are on the front in exactly the place you want to place the passthrough cameras, so now you have to do extra image processing to virtually reposition the input as if it were on the eyes. So it makes passthrough dev more difficult.
It must be extremely compelling to see those eye screens in action. It must be selling onlookers on the device. That's the only that could justify that much effort?
I had assumed a single oval-shaped display and headset, along with notches for cameras. I also don’t see slightly displaced cameras as causing a significant problem. Brains are pretty adaptable to vision changes.
Readit News
Therefore, VR should be a replacement for large displays in productive work, not a device that everyone is wearing outside or casually watching videos with. The use cases are professional; such as programming, CAD, design etc. I believe that properly implemented VR would allow utilizing full visual-spatial capabilities of our brains, totally eliminating the process of hunting for files and switching windows.
It would be possible to dive into code and see it all at once like an intricate factory. Our brains are capable of navigating very complex environments, but this capability is lost when its viewed through a small window.
I wish they'd explore this area, rather than go for a mass market appeal. What kind of software is possible, and how much productivity can be increased with maxed out hardware with high resolution VR? I'm not sure if 4K resolution with a M2 chip is powerful enough for a truly innovative product. Maybe it is, but there's still room for a "pro" product.
I don't think there's a use case for glasses-type product, with a watch-like functionality and environmental AR which would show information pop-ups everywhere. We already have information overload, and we don't need any more of that.
Where do you get this from?
It seems to me if there was even a correlation, which I'm not sure, it would have more to do with resolution, since same resolution but bigger display doesn't actually show more information.
Its true, up to a point.
you just have to ignore that at the extremes it's probably negatively correlated, and there's a significant non-zero possibility that at least in the short term, that possibility of negative correlation applies to the area VR is interested in as well.
That's a dubious statement. Filling the binocular field of view is fine; it's about 120° horizontally, give or take. But past that it starts to be a problem. Generally, if you look at the screen and any edge is blocked by your nose without turning your head, it's likely too large to work comfortably on. (but still fine for immersive content like games or movies)
In my mind the real power comes from spatial hand tracking as a new UX paradigm(seeing an interacting with objects in 3d space) and possibly the addition of large displays anywhere a monitor couldn't have feasibly been, (eg, floating in the server room in a cramped space when needed).
I don't think working on code will be all that different. There isn't any physical spatial interaction with text on paper that the current editors are missing.
I only see this as a supplemental feature for certain use cases (coarse-grained 3D operations). In general nothing beats the efficiency of working with a keyboard for navigating application functions, or (at least with current technology) the mouse for precision 2D navigation/selection. Their advantage being precise and definite/discrete input.
For actual work, even with a VR headset I see myself still mostly using keyboard and mouse.
Perhaps, in a lab setting, doing straightforward, focused tasks like the ones they use in these productivity studies, I could sustain a productivity boost long enough for the researcher to get a two-tailed p<0.05.
Make me do it in real life, though, 40 hours a week, week after week after week, and I expect I would burn out very quickly. I like my work well enough, but I don't think I want to actually stick my whole head inside of it.
I truly believe that VR is perfect for training/sim type games. Actual games.. Could really only count on one hand the games I feel are worth diving into VR for.
- OLED panels could provide much darker screens and cause less eye strain.
- Eye tracking with higher resolution where they only have to render properly where you are looking at
- I think the headset would be much lighter and comfortable.
Do you have a source?
Anecdotal, I work without external displays and keep one window in focus, this helps my productivity.
https://dl.acm.org/doi/pdf/10.1145/1753326.1753336?casa_toke...
There’s much more. Search “sensemaking” or “immersive analytics” on Scholar. Depth and width of field, achieved via stereoscopy and tracking, is well documented as a performance enhancer when it comes to generalizeable task completion in VR.
And displaying more things mean you need to context switch less.
You get retina resolution, high refresh rate, HDR, and 2Factor authentication (Biometric+Goggles)/information security in one package. Of course you'll still likely want to have a laptop to drive it (untethered) except for the simplest mobile games.
- 4k is not nearly enough. I think we will likely need 8k screens.
- The headsets are not light enough. You need something with all day comfort for work.
- There is still too much eye-strain from the quality of the lenses and the fixed focus.
- Input is still not natural and precise enough. We would need to be able to see our own hands and type with zero latency. Hand-tracked pointing needs to be able to pick up on small subtle finger motion like a mouse or trackpad can
Deleted Comment
People can learn 3D environments by viewing them in a 2D screen very happily, and millions of gamers do exactly that every day. I can still remember my way around the map in GTA 3 about 20 years after I first played it...
The "flight" in GTA is more like a moving viewpoint than a space full of 3D objects and affordances.
Full immersive 3D navigation through code and file systems would be an interesting experiment. I suspect we'd get used to it very quickly, and we'd then wonder why we spent so long tied down by virtual paper metaphors. (I could be wrong, of course.)
This is such an Engelbart-esque statement. Bravo! I'd love for AR to go there too :)
I don't think VR headsets will not take off as display alternatives.
People don't even care about highdpi.
I hope you’re wrong about this.
I hope there will be a great personal (non-work) use case for the next generation of VR/AR. Something like gaming, watching sports (always have awesome “seats”), movies with friends, maybe map/gps/etc. for when I’m running.
I know there are options for this already. But honestly, I haven’t seen anyone who’s not a dork using VR/AR. I want the version the average person is excited about. I’m ready to plunk down any amount of money Apple wants from me.
If it’s VR/AR for work, count me out. I don’t have any interest in increasing my work productivity lol.
Deleted Comment
The idea of an AR wearable that is tethered to a fixed base station is so very lame. Why even bother with AR when you can only augment the reality that is in your office or bedroom?
Also the eye-view outer display is essential for situations where one person is using the goggles but others aren’t. How unsettling would it be for someone to look at you, be able to see you, but for their upper face to completely obscured by goggles? How could you have an in person face-to-face conversation with someone like that?
Base station for AR would be detrimental, as you inherently want to explore and interact with the world around you.
For most VR use cases that is not the case, and I can see the argument for the benefits of a base station (higher performance) outweighing the costs (limited movement).
I haven't read more than the abstract, but it seems there is some good research going into this. It must be years off however, as we barely have LTE networks in some locations, let-alone UWB 5G to even begin supporting this type of architecture.
https://ieeexplore.ieee.org/document/9103475
A 4K display per eye, a powerful and efficient SoC on the latest node, and some really fine industrial design will surely be within Apple’s reach.
Everyone will have to build apps for this thing. If it’s real, it will be every bit as successful as iPhone.
Deleted Comment
Very few people predicted the iPhone accurately before its launch. And it’s reasonable to say the iPhone blew minds because Apple figured out multitouch AND reduced the latency of the interface to the extend required so that it felt natural to use. If they mail AR in the same way, they will have the next iPhone.
I am bullish on the prospect of this happening.
I expect Apple’s AR product to be functional, and dubbed a niche failure by the tech commentariat, but by 2026 really start to eat into the smartphone market.
Meta and Google are just less capable of making humane software, and while they may release slightly better devices from a technical perspective they’ll miss out on the light touch that such an immersive experience will require for normal people to feel comfortable with it.
Apple buyers are not necessarily tech savvy (from a percentage perspective).
They use their high end phones for normal consumer things.
I have a hard time imagining what a normal person would want to do with VR/are glasses.
Showing your face in a Videochat? Not being able to show your face?
Playing games?
I would assume that there is still a lot of apple people not playing anything than casual games.
Deleted Comment
AR through a phone or iPad is lame and close to useless. Only looks good in video demos till you use it yourself and see how clumsy it is interacting through a tiny window.
Very clear they have a headset in mind for all this but are struggling to ship.
My takeaway though is that the device itself and how it should look/work is not obvious, and that is a bad thing.
For a device like the iPhone, iPod a few people kicking around ideas over sushi could very quickly (and with wild enthusiasm, no doubt) come to agree on the basic form, basic functionality.
If our VR-future has to be shoe-horned into existence, maybe it was not to be.
If anything, I want to see "the cyber" go away, not become more pervasive.
Quest 2 would like a word with you?
Apple can and does make mistakes, for example the touch bar, and the MacBook keyboard.
If Ive is involved I predict that this product will experience a brief but intense surge in popularity based primarily on the novelty factor and social media influencers followed by a rapid decline into irrelevance.
That way, as long as everyone is wearing a helmet, it looks like no one is wearing a helmet.
Sounds like a good solution for a low probability situation. Eye visibility for normal humans is much more practical than issuing every human a VR system and requiring they wear it at all times to maintain the illusion.
It's gimmicky?
I'm really curios of Fashion conces people really like avatars. I don't think so and I also don't think that the apple brand is a good fit for it.
Real life Avatars with light enhancements will also be weird I think
Let's see.
Two extra external displays in raw hardware. But worse, they are on the front in exactly the place you want to place the passthrough cameras, so now you have to do extra image processing to virtually reposition the input as if it were on the eyes. So it makes passthrough dev more difficult.
It must be extremely compelling to see those eye screens in action. It must be selling onlookers on the device. That's the only that could justify that much effort?