Really cool to see advances/research in foveated display techniques.

Simula's Portable Linux VR headset (www.simulavr.com) is planning on having a rudimentary eye tracking/foveated rendering system.

One of the most important long-term barriers to widespread VR adoption for office work/programming is matching & surpassing the quality of what can be displayed on traditional (e.g.) 4K monitors. It's hard to justify spending 14 hours a day working in a VR headset when the text quality is visibly less sharp than comparable traditional screens. Advances need to be made.

Why would anyone want the hassle of wearing a VR headset for routine office work? It does nothing to improve productivity for most tasks. The value of VR is in other use cases.

What if the "hassle" was solved though? If we get comfortable, light weight, easy-on/off HMDs with good enough resolution I think it will be a game changer.

I work from home with two monitors. If I could pop on a pair of goggles and get infinite screen space I think that could definitely improve my productivity.

It'd give me whiteboard sessions back with my coworkers. The non-productivity boosting feeling of hanging out in the same room. The productivity boosting of having as many displays wherever I want them.

Open office plans and hotdesking are two miserable and widespread trends that VR might help with...

Personally, solving the motion sickness for long term VR usage is the first thing to consider VR for productivity improvements in general office work.

But I'm at the extreme end, The demo from simulavr made me nauseous. I presume even if for someone not being affected by motion sicknesses as bad as me would find it difficult to use it continuously for hours together right?

I disagree. I think the minimal viable HUD is quite minimal, because most of the value is in the "Heads Up" rather than the "Display."

Consider a clip compilation of people walking into poles and tripping into fountains while looking down at their phone. What were they doing, and could your MVP AR glasses have supported it? If the answer is "yes," you have a product.

> I think the minimal viable HUD is quite minimal, because most of the value is in the "Heads Up" rather than the "Display."

You're thinking of AR, while the parent is talking about VR. They're pretty different use cases, and some of those VR use cases need significantly high resolutions to unlock.

Yep, strange, could have sworn it said AR, and that there was a sibling comment discussing exactly this. Ah well.

One day there will be an AR heads-up display with an AI that would've alerted you to your misunderstanding! But alas, not yet.

It's exciting to see that there's companies other than Facebook out there doing this kind of experimentation and research. "Realistic" VR, if it can be pulled off, will be the next frontier of computing. I'm more sceptical of AR because it is much harder to achieve and not as useful in my opinion.

Why not useful? It seems to me that AR is indeed much harder to achieve (well), but incredibly more useful than VR.

By definition the useful parts of AR are virtual. There is a large overlap in AR and VR ux. AR let's you compute on the go but how useful is that really? If virtual interactions are productive wouldn't you setup a space a la desktop computing?

That said AR includes other tech like environment understanding that you might consider much more useful than immersive visuals and spatial interactions alone.

I can see both sides and I expect things to move hand in hand.

> not as useful in my opinion

As someone who spends less than 51% of my waking hours staring at a computer, I strongly disagree with this assertion. Vr has the potential to change the way I use my computer, but AR has the potential to change the way I perceive and interact with the rest of the world.

AR with the eye tracking necessary for foveating has the potential to completely replace smartphones, just control a virtual display with your eyes. And that's one of the less interesting applications. I'm willing to bet AR is the future (but I'm not at all willing to bet when that future will be)

Always love a new Karl Guttag article, and I'm pleased to see people doing all this fundamental engineering and science work.

The more of them I read the more I tend to think that in ten years the best solution is going to be VR with video passthrough. That seems (as a layperson) to have a very clear (at least comparatively) engineering path to delivering a high quality experience.

So many (seemingly unsolveable) issues with AR displays.

This doesn't even touch the software side which seems to need AGI to actually deliver the AR that people actually expect in the end. The trivial things for AR are better served by smartphones and watches for the medium and longterm still.

There's probably also a market for smart eyeglasses which don't deliver a real AR experience but just use a tiny laser to project a few data fields on the retina. There have been a few attempts already that were a bit too early like North Focals that didn't take off but I think the technology will be suitable for mass market adoption soon.

The low rez displays have been around in different forms for years.

Whenever I see them I have trouble coming up with actual applications I would want to use them for that aren't better supplied by say a smartwatch?

Alternative to a smartwatch for people who don't like wearing watches, or can't wear them in some situations. Alternative to Bluetooth headsets. Heads up display for workers and athletes who don't want to keep glancing down at a watch.

Would it be possible to have a watch display (for instance) that appears larger than the actual screen of the watch by using lasers (and eye tracking) to paint the display directly at the retina?

I was going to say impossible, reasoning from how I understand your eye's lens to focus. But that doesn't rule out the possibility of it working if out-of-focus. If you look at a small point light source and let your eyes go out of focus, the light smears to a larger range. The shape and texturing of that range depend upon the light source: for example search for bokeh shots during a partial solar eclipse. This should be emulatable with a small projector.

However the size up of the image should be no bigger than what you see when you let your eyes go out of focus, which isn't really that much. And you'd have to find a way to make your eyes want to focus like that, which could be tricky.

It's got eye tracking. Once you look at the watch and the image is created, whatever part you choose to look at can be in focus.

Babak Parviz had a team working on this at UW back in 2010 (https://spectrum.ieee.org/augmented-reality-in-a-contact-len...). They wanted to use either an LED array with microlenses, or (as you allude) a kind of laser DLP. Both approaches are feasible for projecting a virtual image onto the retina, but integrating them in a working product is very difficult, as illustrated by Parviz's trajectory at Google.

One time when I was a teen I woke up and my brain wasn’t doing saccadic masking for a few seconds, I would move my head left and right and things would motion blur. Eventually it turned back on though. Any idea what that was?

Saccadic masking isn't about head movement. That would be a problem with your vestibulo-ocular reflex. I'd say a likely cause would be an issue with your inner ear, perhaps temporary benign paroxysmal positional vertigo caused by a bit of debris coming loose inside.

> temporary benign paroxysmal positional vertigo

Me: ...

checks profile

> modeless: AR / VR at Oculus

Ah, I see

Haha, actually I had an episode of BPPV myself, not related to my VR work. Triggered by a day of snorkeling, of all things.

I feel that AR/VR are a very interesting intersection of medicine, biology, mathematics and physics. Given you actually work in the industry, what is your experience in this regard? Do people have to have some knowledge in all these fields, or do you rather have strongly focused specialists in each who then collaborate?

E.g. you clearly need some biology/medical knowhow to be able to diagnose various problems that arise in the human part of the system, no?

Just off the cuff, I'd say specialists are important in some areas. But you won't get very far without generalists who can see the whole picture, and study to become specialists where necessary. In a new field, the right specialists may not exist.

I wouldn't say you need a medical degree to diagnose VR-specific biological issues. A lot of this stuff hasn't been particularly well studied so you're going to be discovering new things rather than applying existing knowledge. On the other hand optics specialists are very important; optics is very deep and existing knowledge is very relevant.

Is this more or less what Magic Leap attempted and ultimately failed to deliver?

No. Magic Leap succeeded in delivering a technically impressive AR experience with multiple (2) focal planes. Their failure was in not realizing that the overall experience wouldn't be useful enough to build a business with enough revenue to support the development cost. That's due to a combination of factors like low brightness, small FOV, weight, cost, and difficulty of developing world/object tracking for compelling AR apps.

Also Microsoft totally stole their thunder with Hololens coming out first.

Magic Leap is still going, they’re trying to sell to businesses now. I’ve no idea how they’re doing, though I expect they’ll need to make a lot of sales to recoup all the investors money.

In the future, imagine most people in the developed world wearing frames till contacts with a display come along.

Smart contact lenses already exist in prototype form.

https://www.mojo.vision/mojo-lens

They already do, at least in commercial settings.

But this article is about foveated rendering that is able to track eye movement to keep the hi-res area under focus. The eye-tracking tech is the least interesting part of it!