Thread by @Pinboard

The spyware scandal in the news today is a chance to reiterate that human beings are incapable of producing defect-free software at any scale. In particular, there is no such thing as a secure online system or a secure mobile platform. This foundational issue won't go away.

Our main line of defense against malicious software is that human ingenuity is also limited, so we only find a fraction of our errors. And the malefactors go on to make more mistakes coding the malware. Incompetence is the great defensive wall securing most of our infrastructure.

The phone situation in particular is dire, and I hope we see a future where these all-in-one devices are supplemented by simpler machines that do just one thing (make phone calls, send text messages) and can't be turned into a 24/7 surveillance beacon by hacking an emoji renderer

There may be defect-free software somewhere (a very simple avionics servo or space probe) but the level of effort required to create it is so many orders of magnitude beyond what goes into commercial software that any programmer's remaining hair stands on end thinking about it

To give you an idea of how bad the situation is: one of the greatest textbooks in our field had an error in a tiny CS101 algorithm (binary search) for 20 years. This was then fixed, the fix proven to be correct, and then they found a bug in the fix as well ai.googleblog.com/2006/06/extra-…

Or there was that time three years ago when every computer more complex than a graphing calculator (and probably that too) was shown to be completely broken. en.wikipedia.org/wiki/Spectre_(…

There's a theorem in computer science called CAP which basically says, if you could have network where nothing ever went offline, you could do cool stuff. A lot of people get excited and try to do the cool stuff anyway, then rapidly discover that such a network does not exist.

We're doing the same thing in online systems. We behave as if creating secure software is possible and wire everything together so we can do cool stuff. There's no mechanism to bring the consequences of this decision back to bite the people who make it. They get externalized.

The one thing the software industry excels at is convincing people that computer bugs are a fact of life, and that the solution for them is even more complex software.

Anyway use Pinboard