Kyle Goodrich’s Post

Here's how AI on XR glasses can change your reality. Generative AI on XR headsets and future AR/MR glasses could let users change their reality by using this kind of GenAI rendering in real-time. In this concept by Kyle (Design Engineer at Snap), he takes further Apple's built-in experience for its Vision Pro headset (which already lets you to apply basic filters on top of your reality). I talked about it when I tried Apple Vision Pro back in February. I guess anyone who tried AVP, probably noticed that you can even turn a gray rainy day into a sunny day with happy colors (kinda) and get some summer vibes while wearing the Vision Pro. The same principle applies in this concept - We could take this further, only if XR and Spatial Computing Developers, Designers and Creators had a Camera Access on the Apple Vision Pro and Meta Quest 3, we could create wonders with real-time GenAI capabilities. This video is just another example in adition to my "AI and XR are made for each other" article series. This is how creativity and innovation should not be limited by Meta and Apple, privacy is important, but camera access for creators will unleash way more innovation, in regard to AI with XR! Since the release of Apple Vision Pro, most creators are focused on AR/MR and its potential Computing, often with AI (just like I did in my recent concepts) and it's really great to see that other creators, from known tech companies, take it further. I hope that Apple and Meta will find an ethical and secure ways to give developers and creators access to cameras - because currently Apple only allows it in enterprise organizations, and Meta does not allow access to cameras at all. What's your opinion? Should Apple and Meta give creators access to XR Cameras and unleash creativity and innovation? #AI #GenAI #AR #MR #XR #VR #AppleVisionPro #MetaQuest3

This would have been the greatest pack-in title for a Quest 3. Out of many... but yeah. There's just so much we're missing in the whole Meta/Apple/Spatial Computing Space in general. It's almost as if those in charge 𝘷𝘢𝘨𝘶𝘦𝘭𝘺 grasp the idea but then the execution leaves a lot to be desired. Missing blatantly obvious low hanging fruit for execution and immersion. A demonstration such as this is a breath of fresh air. Assuming it can allow for user generated world skins to be created using some sort of prompt that can then be shared by all other users.

🌌 As we prepare for interplanetary journeys, the ability to transform any environment into a familiar, comforting space becomes increasingly vital. Projects like World Skins, powered by generative AI, offer the promise of dynamically altering environments in real-time. Unlike traditional AR, which relies on static elements, this innovation allows for a fluid and responsive experience. Overcoming challenges such as limited camera access and real-time inference speeds will open up new possibilities, enabling us to recreate the essence of nature, home, and Earth wherever we go. The future of AR will help us feel at home, even among the stars.

As I wrote all the way back in 2016 — we are moving towards a future of digital integration within the physical world, which will make digital experiences increasingly more capable of achieving visceral impacts, for better, and worse. Important thing to consider: what happens when we decide to do alternative “skinning” of people using genAI? There was a whole Black Mirror episode on it & it seems we keep using dystopia as product roadmap without even being fully aware of it or putting much consideration in necessary guardrails.

"Introducing SIR: a cutting-edge method for decomposing differentiable shadows in indoor scenes, enhancing material estimation and lighting realism. By utilizing multi-view data and an SDF-based neural radiance field, SIR outperforms existing methods in both synthetic and real-world environments. Its advanced capabilities enable free-view relighting and material replacement, revolutionizing the world of inverse rendering. #AI #MachineLearning #ComputerVision #Research #Innovation"

😹 GARField, developed by University of California, Berkeley and Luma AI, introduces an innovative approach to group elements within 3D scenes. It employs a scale-conditioned affinity field, optimized from multi-level masks provided by the Segment Anything Model (SAM). This allows GARField to handle the inherent ambiguities in grouping, resolving conflicts and overlaps in 2D masks across different views by conditioning on the Euclidean scene scale. The model excels in selecting various groupings based on their 3D size and can be queried interactively or generate consistent clusterings automatically at multiple scales. GARField's ability to extract and manipulate assets in 3D scenes offers significant potential for applications in virtual reality, gaming, and more. 🔗 Discover more on their Project Page: https://lnkd.in/eXcBy6wz 📚 Dive into their research on arXiv: https://lnkd.in/ev3B8jeb 💻 Explore the code on GitHub: https://lnkd.in/emp9wWHE For the latest in 3D scene analysis and machine learning ⤵ 👉 Follow Orbis Tabula #3DSceneAnalysis #RadianceFields

👇 Follow me for more interesting contents. 👇 GitHub: 👉 https://lnkd.in/e7hZvG5Q LinkedIn:👉 https://lnkd.in/ej-jHj9s Google Scholar: 👉 https://lnkd.in/eQM_qKw7 #computervision #technologie #development #university #share #robotics #ai #google #datascience #data #datascientist #AI #ML #ArtificialIntelligence #MachineLearning #modeldeployment #reinforcementlearning #objectdetection #bourse #recherche #workshop #reinforcementlearning #SQL #nosql #sqlite #deeplearning

This is so powerful, and would amazing to integrate with BIM and Digital Twins.

*new* Important step forward for building mixed reality experiences on the Meta Quest: AI-based semantic understanding of the surroundings, answering the question: what's in front of me? https://lnkd.in/ejdgSRKp