@sea3dformat @threejs Meshlets are basically chunks of meshes, that get rendered with a mix of rasterization and compute.
So despite tons of stuff being on screen, actual rendering is more like "the pixels you see"
@sea3dformat @threejs Everything is pretty custom at the moment – a custom shader that is able to render the meshlet instances with their original data / texture / UVs etc – but the overall direction feels good
@sea3dformat @threejs Meshlets are generated via @zeuxcg MeshOptimizer, not sure if I'm doing everything right there but I get small compressed meshlets and can render them.
Hacked browser-based meshlet generation into meshopt, too, so that you can drop your own GLBs to pulverize them!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Apple's 3D viewer is based on @threejs, uses @glTF3D and gets more impressive every year!
CAD meshes, environment blends, reflection probes, occlusion: let's take a look at what's special this time –
(long thread ahead)
First, some eyecandy. The viewer looks CLEAN. Why?
- the viewer snaps to canonical angles (side, front, etc.)
- there's likely some blending of environments, with specifically tuned environments per canonical view
There's also a clean transition when selecting the "Color" options, where the phone fills more of the screen.
Only that first part is 3D – all the other menu entries are videos that fade to/from black.
The new 3D models at Apple's website are using @glTF3D, @threejs r165 and gltf-transform 4.0.8 by @donrmccurdy on the iPhone 16 page 🔥
The files aren't really valid though, and won't load in many viewers – but we will fix that.
Let's take a look under the hood –
When downloading the files, you get a bunch of .gltf and .ktx2 files. Curiously, the glTF files are invalid – there are hundreds of errors, for example
– not declaring extensions (KHR_texture_basisu)
- wrong image MIME types (some JPEGs are named PNG!)
- accessor and normals errors
So when loading the files, they'll have no textures... Fortunately we can fix that quite easily.
First, we open the file in a text editor (VS Code here)
Then, format (ctrl + p, > format)
Now that's easier to work with. The structure of glTF is basically JSON.
The file is hidden behind the main logo (on iOS, it launches in AR). Since it's just a link we can take a peek at the USDZ file from a browser's inspector.
Also, it looks like they limit the AR experience to iPhones for some reason.
After unpacking the USDZ (yes, USDZ is literally USD-zipped):
Like some previous event files, this is actually a frame-by-frame animation! Each texture here is a ~50-100kB, 1024x1024 JPEG. There's 150 of them.