John Carmack's Take on Mixed Reality
John Carmack's Take on Mixed Reality
69 comments
-
-
Mixed reality is pointless at the home right now. Industrially I want it for warehouse workers yesterday. Make it highlight shit they need and it’ll reduce mental load and cycle time
-
I haven't personally used it, but it is very much a thing in manufacturing, defense contractors at least.
It isn't cheap, but it let's them move employees between lines with little downtime and little training (the required skills were pretty set in stone though). plus it gives them the added bonus of tracking productivity on an unprecedented scale.
-
-
I disagree. Companies have demonstrated how well AR can work.
If my mom had an AR headset, I could more easily help her troubleshoot why her printer won’t print or why her PC has no internet.
AR can also use enhanced data sources to overlay information you cannot normally get on your own. (example: sporting events. See player stats when looking at each player. See game related info. etc.)
The issue is that we don’t have quality, lightweight AR headsets yet. Ideally something with both cellular and wifi.
Apple’s Vision Pro may change things.
Also I think Pokemon Go gave us a sneak peak into what AR gaming could be.
-
Let's be realistic: if your mom had an AR headset, you'd be troubleshooting that and the printer.
-
Normally I would agree with you, but she actually took classes and is quite good with tech thanks to that.
Your comment absolutely made me chuckle, however.
-
-
I think you make good points - social and collaborative activities are where AR will do best. Integrating real world and virtual content could also be huge, though not necessarily the way you described. Overlaying sports data in a sporting event setting is a very late case and unlikely to be developed until the technology is incredibly mature. But overlaying GPS directions or creating beacons and other constructs in real space could be huge if the tech gets just a little more practical.
Also Pokémon Go plays just as well if you turn off AR. It’s mostly a gimmick in my opinion. But other AR gaming examples do exist.
-
-
Well, if we are talking about a headset, I totally agree. I'm not strapping an alien horror movie prop to my face just to see fake waterfalls behind my PC or to allow Adobe to spam my peripheral vision with ads.
But if we are talking about regular glasses, assuming the technology keeps shrinking, then hell yeah all I want is mixed reality and rendering on top of real images.
-
This, I already spend the entire day with my glasses, I would love it if they were smart too. Still think that google glass failed because they made a product for people that don't wear glasses.
-
-
The potential for board and card games is huuuuuuuuuuge though.
-
And professional uses, though the headset form factor is kind of shit for it.
-
i mean i do rather feel that just a projector in the ceiling would be better, though
Certainly less faff than having everyone wear a headset.
-
-
The author is missing the other use cases for high quality passthrough:
- "Pausing" your VR experience temporarily to interact with the real world without taking off the headset. Checking a phone, grabbing a drink, etc.
- Selective passthrough that allows seeing portions of the real world. Being able to see your own hands/arms/body is a huge improvement in VR. As is seeing your desk, chair, etc.
-
He's not writing about the pros and cons of passthrough. He's writing about Mixed Reality gaming's potential to move headsets off shelves. e.g. Will people be hyped enough about XR to buy a headset? He thinks not.
-
The cool stuff is augmented reality. Basically low tech projected holograms. You have many informations in a heads-up display, kinda like they have in fighter jets already, or in video games, but you can have many things. Highlighting direction (when driving or walking), highlight things you're looking for adding informations to objects you're seeing... But we're far away from this still.
-
A few more ideas for MR:
- Instead of watching a movie in VR by myself, give me a Tom and Jerry projection that I can follow around the house. Something that allows me to associate happiness with my home. Maybe just random Smurfs living around the house.
- Embrace the clutter in a haunted house overlay. The sock under the couch will transform into a shadow hand reaching toward you. Faces will try to press through your walls. This is a hell no for me btw.
- A storyteller avatar who sits with you and talks while you are relaxing. Something to combat the loneliness of isolationism.
-
The Canadian House Hippo agrees.
-
I get what he’s saying, but I just think that adding screens wherever I want them e.g. above me while lying down while still feeling in my own environment sounds way better than entering some other VR space entirely
-
I agree with him. I find it amusing that in all the videos of Apple’s headset, the homes portrayed are nothing like a normal home. It’s always this stylish, spacious minimalistic home that none of us normally has. Projecting a virtual screen in my actual, real home would be a very different experience that what they show in their videos. I would probably have more interest in getting away from it in a virtual world than embellishing it with virtual elements.
So yeah, Carmack has a point here, in my opinion.
-
The Apple device does support virtual screens, but it's not what they're focusing on
-
I think I disagree, I live in an apt that’s by no means luxurious. I’ve spent some time in VR and there’s just so long I’d be willing to spend in a fully virtual space. If you have dogs or people around you, you fully isolate yourself from them. If you want a cup of coffee or a glass of water the context switch could become a small inconvenience, etc for other small everyday stuff that VR hinders.
But I totally get what you’re saying and I’d love to have the option of both, but that kind of necessitates that AR tech advances as well as VR.
-
-
-
-
That depends on why you're using it, for displaying close range real world into that's exactly what you want but for prepackaged multimedia experiences (games and other stuff like remote interaction) it doesn't fit as cleanly, and he's probably only thinking about the latter.
Mixed reality is excellent for tracking inventory (including at home, finding where you left your stuff), checking the status of and controlling networked devices, navigation, physical coordination, visual guides (especially if they can map onto the objects in front of you), etc...
-
-
-
Imagine playing D&D in a park, with the monsters and spells blending seamlessly with the outside world
Or AR projected onto a real tabletop for TTRPGs and board games
-
If AR could take the form of glasses about as lightweight as regular glasses, I could see it being more of a thing. Until then, I doubt AR plays a huge role. At least with pure VR, your real environment doesn't play as much of a role so you don't need the portability as much.
-
He's completely forgotten how extremely super useful it would be for industries.
Instantly identify machines and components, quick and simple navigation, simple overview even over very large machine parks and similar environments along with simplified coordination, etc. Along with quick access to detailed documentation and status information and guides like visual repair instructions, etc.
-
Could not agree more. Industry is where this tech will shine. It has applications for home use as well.
I was late to the game with learning house repair. What I wouldn't give for a tool that could assist in identifying problems, identifying parts, or laying out the instructions for accomplishing a goal adapted to my situation.
With most text and video
blog spam"tutorials" enshitified to push more ads, I would jump all over something like that.
-
-
I would like to have a HUD in real life. Shows me a path to where I am going on the ground. Pops up reviews on items I am looking at in the store.
-
Show me people's names and pronouns, any private tags I've pinned them with, reminders about them, etc. Maybe my (self hosted, FOSS) AI assistant has some updates for me from their socials.
If a VR headset could be comfortable and unobtrusive and basically be an outboard engine for social cues, I'd be thrilled.
-
-
That is horrible! I want something unobtrusive. And no ads. First ad I see that HUD device goes right in the trash.
-
-
Names above people I met so I don’t keep forgetting it. Maybe a short summary of earlier interactions.
-
-
While I appreciate and partially agree with his view, I'm personally glad they did tackle MR as soon as it was viable.
I very much have a use case for it and I have been using the pro for it and being disappointed with the results. Our family hangs out together for hours every day. And while I can watch pretty much anything on TV and still enjoy both it and our conversations, lately I have taken to using mixed reality to continue participating in the social aspects of hanging out, while enjoying my own content. Usually I'm playing a game instead of watching TV.
With mixed reality on the Quest pro, I can still see everyone's faces and they can see enough of mine to get my facial expressions. But it's tough for me to always make out what is happening on their TV. So I mostly have to go by audio cues to follow along with their content and any conversations they have relating to it.
With the Quest 3, I'll be able to see their screen too.
And that's not even mentioning just how nice it will be to look at my phone front and center rather than peaking through a hole, or just in the lower quarter of my vision on a Quest pro without the light blockers on. Or pinning another floating screen to my phone, like a text document stickynote type thing. Augmenting my phone could get interesting.
Can't wait til VR is more compatible with outdoors, I know they work fine outdoors, and being careful/vigilant with it is enough to make it already completely viable. But companies aren't gonna jump on it or make software for that use case until it's less likely to lead to an increase in headset failures overall. But I would love for headsets to have GPS and cell capabilities. Augmented outdoor exercise games are the main thing I'm waiting for. Though of course there are also risks to exercise gamification to sort through. Same sort of thing, people being encouraged to do it without knowing the risks or participating in mitigating them for themselves. Tough to get stuff like that realised.
-
I would be curious to hear him list some of the low hanging fruits.
-
When your orchard is just getting off the ground all the fruits should be low 🤷
-
-
I remember when people like him said it was pointless to put a camera in a cell phone because the quality was so bad at first. Apple has historically done well creating markets that didn't exist, or were niche, until they made a product that caught on with the masses (iPod anyone?). And I say this as no fan (and currently don't own any products) of Apple
-
Mixed reality has great practical applications, such as projecting studs and inside wall components on a surface so you know where to not put a hole.
These applications could help justify buying a headset in general, pushing folks that want one for gaming additional reasons to justify it.
So... there's a roundabout way it could help gaming.
-
choose life. be present. live amongst those around you. be aware of what you are going through. the apps can join your world. you don't have to lose yours.
-
Hinestly, I feel the same way about VR as a whole. Would be different if motion sickness wasn't a thing.
-
And he says VR exists to give you a way to experience a different, better environment. To "replace" reality that being the keyword and the way he says it is loaded. Says a lot.
I do think the VR and XR bubble has popped though.
-
AR is the future when shopping/going to restaurants/etc, however
69 comments
I just want a minimap and waypoints. Maybe a highlight function so that it's easier to find the thing I need in a grocery store too but that seems a tad unrealistic
Yeah! This would help so much in navigating the fridge!
Dammit, the ice maker is broken again.
Grocery stores wouldn't want that. They want you to spend as much time as possible in their store so you'll likely put more products in your basket.
In reality AR glasses would likely depend on external data/services provided by some large corporation which would probably make money by nudging the wearer of such a device towards spending more money.
There might be a premium subscription model though...
They'd want it for instacart, however.
How about a highlight function to pick my kid out of the crowd he dove into?
How about a “yellow brick road” to follow when I want to get to the Emerald Castle?
How about highlighting and magnifying those street signs that never seem visible enough, or house numbers as I approach my destination?
How about highlighting dangers, warnings, because I’m paying too much attention to my phone and roll my ankle on that damn protruding gas valve every time around the block?
Or, most importantly for those of us who can never seem to focus on the social aspects, how about a pop up bio for the person I’m talking to so I can remember their name and where I last talked to them?
How about absolutely not? I don't need every random jackass I interact with having information about me available at a glance. This would be abused in so many ways
This was actually one of the first uses for a Google glass like product proposed back in the 80s/90s.