Thread by @christogrozev

Got a few questions about how we did the (partial) face comparison in our latest investigation. This is a valid question given that the standing man in the Omsk hospital wears a mask. As this issue is bound to pop-up in the foreseeable future in our Covid times, I will explain.

Obviously, the publicly available professional-grade face-verification services like Azure are not geared up to compare partial faces to other faces. In fact, they don't recognize a mask-covered face as a face at all.

So we need to cheat the system and use a proxy for the missing half-face of our unknown suspect. The idea is to force the software to give us a face match based on the upper face section (eyes, skull shape, ears = signal), while ignoring the lower part (nose, mouth, chin = noise)

To achieve this, we need to substitute the masked area with *another person's* lower-face section, thus adding "noise". At the same time we hope the upper-face features of the two photos are identical enough to cause the AI to give us a high-enough score - ideally >50.

So, we took a random person's lower-part face section and supplanted it on to "unknown suspect" masked area: in this case we borrowed, just for fun, Graham Phillip's face (the man in the middle disrupting our press conference in London).

Here is the somewhat Frankesteinian proxy we are going to use for the partial facial comparison:

And bingo: Azure gave us a 0.58 score; in fact attributing such as high identity score the upper-face segment that it overrode the obviously *different* lower-face parts.

No, if we had done the same trick with the lower-face part of the *known* suspect - FSB's Mikhail Evdokimov - the score obviously would be much higher... near perfect.

Obviously, this hack can be made much more reliable by running the same experiment over hundreds of other people's (or AI-generated) lower-face segment.. But as this was not a critical part of our investigation, we ran it on 5 random faces and always got >50% score (except 1)

(it worked even with @aplusk's face)