An Object is Worth 64x64 Pixels: Generating 3D Objects via Image Diffusion.
Basically, think of the "omages" approach as akin to #MaterialID. Indeed, that's what this looks like when the algorithm breaks down the 3D into discrete geometry.
Most #TextTo3D suffers from making any resulting #3Dmodel as a single mesh. Depending on what that model is intended for, this may or may not be useful. For example, the project page refers to books on a shelf that are, ultimately, a clump of books as a single 3D object, not individual books.
The Omage technique can take complex shapes and break them down into a single 64x64 pixel image, making the 3D shapes into a 2D format (that alone is difficult to grasp)! By converting these Object + Images into "Omages", the result is information that is BOTH a #pbr material and mesh geometry!
Is it perfect? No. It's a first-gen approach to a challenge, but check out the takeaways below for more detail. This is a fascinating way of encoding complex and irregular 3D information. I'm curious as to whether #animation might effectively be encoded as a string of sequential 64x64-bit PNGs, which could, conceivably, be compressed ever further? Fascinatingly, to increase the object resolution, one need only rescale the omage.
Curioser and curioser.
Project: https://omages.github.io/
Paper: https://lnkd.in/gQ3w7PNG
Code: https://lnkd.in/gt8UrW7g
TAKEAWAYS:
1. Omage encodes geometry as a (R,R,3+1) image, which is essentially a 16-bit RGBA PNG!
2. PBR material is encoded in another (R,R,8) image. So three PNGs will give you a realistic 3D object.
3. You can use image generation models to generate 3D objects, one image for one object!
4. Discrete patch structures emerge out of continuous noise during the denoising process.
5. Patch segmentation comes naturally from 2D disconnected components, no instance labels needed!
6. Irregular connectivity & complex topology? No worries, all encoded in a regular image.
7. Change your object resolution by just rescaling the omage!
8. Generated shapes come with UV maps—no unwrapping required.