Adam Sun’s Post

🚨 Paper Alert🚨 ➡️Paper Title:Nuvo: Neural UV Mapping for Unruly 3D Representations 🌟Few pointers from the paper 🎯 In this paper authors have presented a UV mapping method (Nuvo) designed to operate on geometry produced by 3D reconstruction and generation techniques. 🎯Their method simply requires a representation of scene geometry that allows for sampling visible 3D points, and then they optimize Nuvo from scratch for each scene by minimizing a set of losses that encourage Nuvo to represent a well-behaved mapping for observed points. 🎯 Authors have also showcased that their model is robust to the challenges posed by ill-behaved geometry, and that it produces editable UV mappings that can represent detailed appearance. 🔥Paper Authors: Pratul Srinivasan , Stephan Garbin, Dor Verbin , Jon Barron , Ben Mildenhall 1️⃣Read the Full Paper here: https://lnkd.in/dki6e-8Z 2️⃣Project Page:https://lnkd.in/dhT6dX-v Music by Aleksey Chistilin from Pixabay Find this Valuable 💎 ? Repost it to spread this excellent work in your Network ✨ #3d #nerf #artificialintelligence

What do foundation models know about 3D? Quite a lot. Are they multi-view consistent? No. In Human 3-Diffusion we jointly train a multiview diffusion model with a generative 3d reconstruction model. Output is 3D Gaussian splats and coherent. Results for single image 3d reconstruction are great, even in the occluded areas! https://lnkd.in/evyg6WZ7

🤔Have you ever imagined being able to animate realistic #characters in real time? Score Human Mesh Recovery (HMR) the #method developed by Rutgers University , makes it possible. How does it work? 1️⃣It uses a pre-trained diffusion model to refine the SMPL parameters of human body poses and shapes given an input image 2️⃣ It explores the latent space of the model using DDIM inversion and guided sampling to find the 3D configuration that best fits the image and additional observations 3️⃣ It uses a scoring function to guide the sampling process toward realistic solutions based on the observed data Score HMR is the method for #3D reconstruction of #human pose and form from #images or videos that outperforms traditional optimization-based approaches by offering an efficient, robust and versatile solution. 👆Follow us if you want to stay updated on other AI-topic curiosities. Anastasis Stathopoulos Ligong Han Dimitris Metaxas Paper👉 https://lnkd.in/dRrbnpYm Project 👉 https://lnkd.in/d5mEa9kE

One of the other cool thing about model based procedural grammars is that one can generate millions of auto-labeled geometries and hierarchical structures in #3D space fitting in the range of acceptable variable parameters (really many) over billions of input geometries for a given typology. Here a small capture batching over 2 decades of facades. #trainingdata #generativeai #ml

Say hi to the easiest way to create top notch 3D #digitalassets 🤗 All you need is a single image and Gaussian Splatting.. HOW COOL IS THAT 🔥 #gaussiansplatting #3d #reconstruction

The paper titled "Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstruction" by Olivier Moliner and Sangxia Huang of Lund Laboratory, Technology Development Labs, Sony Corporation, and Kalle Åström of Lund University, has been accepted at the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2024). The conference is held on May 28-30, where author Olivier Moliner will give an oral presentation on May 29 at 10:30 (UTC+3). Here we address the challenges in estimating 3D human poses from multiple views under occlusion and with limited overlapping views. We approach multi-view, single-person 3D human pose reconstruction as a regression problem and propose a novel encoder-decoder Transformer architecture to estimate 3D poses from multi-view 2D pose sequences. Our results demonstrate the efficacy of our approach, particularly in occluded scenes and when few views are available, which are traditionally challenging scenarios for triangulation-based methods. Full paper: https://lnkd.in/g_CQEpW4 FG 2024: https://lnkd.in/dbXpBn8j #FG2024 #ComputerVision #Sony #Research

Man this one is impressive. For novel view synthesis, usually many images along with poses are required to get a clean result BUT using diffusion priors, a few (like 3) images is all you need! This is "ReconFusion: 3D Reconstruction with Diffusion Priors" leveraging a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets, which regularizes a NeRF-based 3D reconstruction pipeline at novel camera poses beyond those captured by the set of input images. This method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions. Check it out! (As usual, relevant links in the comments) #novelviewsynthesis #nerf #diffusionmodels #machinelearning #deeplearning #futurism #3Dreconstruction

Get 3D scene from monocular images by distilling the knowledge in stable diffusion with GeoWizard GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image arXiv paper abstract https://lnkd.in/eNcyrS5g arXiv PDF paper https://lnkd.in/eHGNN57W Project page https://lnkd.in/exgM5W7J ... introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes, e.g., depth and normals, from single images. ... demonstrate that generative models, as opposed to traditional discriminative models (e.g., CNNs and Transformers), can effectively address the inherently ill-posed problem. ... further show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. ... extend the original stable diffusion model to jointly predict depth and normal, allowing mutual information exchange and high consistency between the two representations. ... propose ... to segregate the complex data distribution of various scenes into distinct sub-distributions ... enables ... model to recognize different scene layouts, capturing 3D geometry with remarkable fidelity. GeoWizard sets new benchmarks for zero-shot depth and normal prediction ... enhancing ... downstream applications such as 3D reconstruction, 2D content creation, and novel viewpoint synthesis. Please like and share this post if you enjoyed it using the buttons at the bottom! Stay up to date. Subscribe to my posts https://lnkd.in/emCkRuA  Web site with my other posts by category https://lnkd.in/enY7VpM LinkedIn https://lnkd.in/ehrfPYQ6 #ComputerVision #3D #AINewsClips #AI #ML #ArtificialIntelligence #MachineLearning

Introducing LaRa: The Future of 360° Radiance Field Reconstruction! 🌟 LaRa, a groundbreaking feed-forward model, significantly advances 3D reconstruction by unifying local and global reasoning in transformer layers. This innovative approach leverages Gaussian Volumes, an image encoder, and Group Attention Layers to achieve high-quality and rapid 360° radiance field reconstructions. Unlike traditional methods constrained by small baselines, LaRa excels in handling large baselines, offering high fidelity and robustness even in zero-shot and out-of-domain testing scenarios. LaRa addresses the core challenge of reconstructing objects' shape and appearance from multi-view images, a critical task in computer vision and graphics. By overcoming limitations associated with dense image captures and small camera baselines, LaRa introduces a novel volume transformer. This method progressively and implicitly performs feature matching, efficiently aggregating local and global features to produce photorealistic renderings from just four input images. LaRa demonstrates exceptional performance, achieving high-resolution 360° novel view synthesis with minimal training resources—only four GPUs over two days. The model's efficiency is enhanced by a coarse-to-fine decoding process and efficient rasterization, enabling high-quality mesh reconstruction with off-the-shelf algorithms. With its robust zero-shot generalization and ability to handle diverse viewpoints, LaRa sets a new standard in 3D reconstruction, pushing the boundaries of what's possible in visual effects, e-commerce, virtual and augmented reality, and robotics. "LaRa: Efficient Large-Baseline Radiance Fields" Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, Andreas Geiger Github Repo: https://lnkd.in/gJ78NjqB Arxiv Preprint: https://lnkd.in/g5nrJsuK #Transformers #FewShot #3DReconstruction #ComputerVision #AiResearch

As promised, we have put in ArXiV our two #CVPR2024 papers, and we put the code in Github. 1) G3DR: Generative 3D Reconstruction in ImageNet, https://lnkd.in/ega_MhnE. Project page: https://lnkd.in/e2FVbZRB 2) VkD: Improving Knowledge Distillation using Orthogonal Projections, https://lnkd.in/ekruDJjY If you find single-view 3D reconstruction and knowledge distillation interesting, you will like these papers. :D