,
15 tweets,
6 min read
Read on Twitter
Some #webgl / #threejs insights about the initial implementation of the doggo scene on dogstudio.co (thread)
It first started as a weekend experiment based on @thespite 's article on Matcaps / Litsphere / Spherical environment map clicktorelease.com/blog/creating-… with some additional bells and whistles.
Matcaps are very interesting because this technique allows us to efficiently simulate complex lighting conditions, and they can be drawn by hand (as they were on the website) or made by taking pictures of actual/physical spherical objects (as a red marble, see picture). 
Matcaps have some issues though. It is not possible to make it looks like the camera is moving around the models (it always looks like the model is rotating on itself) and that a strong normal map can produce some very bad aliasing when viewed from a distance.
In that implementation, I alleviated the later issue by reducing the normal map influence the farther the model is from the camera (actually done per vertex in the shader). Another possible (?) solution in some cases would be to have custom mipmaps for the normal map...
This way when the dog is close to the camera the normal map is used to full effect making the model look very detailed and when the dog is further it looks smoother without much aliasing. It took some time to fine tune it empirically to avoid making the transition too obvious.
An interesting effect to implement was the transition between Matcaps. It was based on the distance of the model to some position hardcoded in the shader. Some arbitrary components of the normal map were used to add details to the transition and highlight the "fur" of the model.
(those are some of my marble matcap btw 😅)
Some side projects which were produced while working on this : 1) the whole PRWM file format thing ( github.com/kchapelier/PRWM ) was in part developed for this experience, though in the end it wasn't used because the goal became to have an animated model (which is not supported).
2) The custom blur kernel tool ( kchapelier.com/blur-kernels/ ) which was implemented to try and find an efficient / original blurring method, since at one time the use of a strong vignette blur effect with chromatic aberration was on the table.
3) An in-browser Matcap tool to slightly tweak Matcaps, adding a fresnel rim and some fake (fresnel-based) iridescence. It was naively done by applying the Matcap to a sphere with some additional effect applied and saving the result as a png. It "works" ™️
(well if you apply it iteratively a few dozen of time it will probably introduce a noticeable deformation...). I don't have a public url for this tool as it was made at work during work time 😅
Nowadays, Matcaps are directly supported in #threejs ( threejs.org/examples/?q=ma… ) but adding transitions, normal smoothing, etc. would still require you to get your hand into it. So the reading the article linked at the start of this thread is definitively recommended.
The WebGL work on this project was later continued by @Samsyyyy who notably proved it was possible to use a Matcap with an animated model by doing some normal reconstruction in the fragment shader without any noticeable artifact / deformation 👌
(final doggos)
