Bugs are Law
Code isn't Law. Bugs are.
“Code is law,” Lawrence Lessig said, and legions of cryptocurrency developers repeated the mantra.
Technology disrupts and redistributes power—economic power, social power, political power, military power.
For those dissatisfied with the preexisting power balance between the people and the state, “Code is law” offered the promise that by coding computer instructions we could rewrite the state and build a new constitutional order—one laid down on the ethical principles of Stanford undergrads playing beer pong and their billionaire parents’ oh-so-proletarian political leanings.
Alas, it was not to be. But not for the reasons many think.
A ruling system has power, sure, and code may well indeed wield that level of de facto governance. We could make a fair argument that Silicon Valley companies like Facebook and Google and Microsoft are private sector rulers of at least part of our lives. But the power to subvert a system hands that power to the subverter.
If you can hack the system, then you run the system.
And boy can you hack the system! Code is law can only work when code is perfect, but humans aren't perfect, everything we make is imperfect, the code we write is imperfect.
Given that we are all flawed mortals doomed to die, how on earth could we ever have thought that we were capable of writing perfect code?
That's some Icarus-level hubris right there, people.
Any system based on the assumption that the system itself has been perfectly conceived, designed, and executed will fail.
I work in cybersecurity. People ask me what I do exactly all day long? It means I manage human imperfection for a living.
Imperfection means both incompetence and malevolence. Humans are capable of, and routinely engage in, both. (Incompetence is easier to forgive, as it is not willful, but malice aforethought, less so.)
Incompetence results in bugs. All code has bugs, and some bugs are exploitable security vulnerabilities.
This is a natural law of software, and now that the human race has migrated onto the internet, it is now a natural law of human life. A law that mediates and governs increasingly all aspects of our existence.
Because malice exploits the fruit of incompetence.
Malice exploits those vulnerabilities for profit, for power. For power most of all.
But worse than this, malevolence also inserts bugs on purpose. Why? For all the motives that humans do anything: greed, revenge, power. Maybe an engineer is being blackmailed. Or worse, maybe the software company has received a secret court order from the government compelling them to insert bugs, or fail to fix bugs.
You know, like the stuff we learned about ten years ago in the Snowden documents.
This is the world in which we now live. The human race has migrated onto the internet. We don't live in the real world and "go online". I'm sorry, are you still an AOL customer??
We live online, and power relations between states, between people and the state, even between individuals is now mediated by software flaws.
Let's just say I have trust issues.
How can I trust—how can we trust—anything that happens knowing the above facts to be true? And not only true, fundamental laws of the fifth domain?
Code isn’t law. Or maybe it is. But it doesn’t matter. That's like saying the king’s word is law, but a snake courtier is telling him what to say.
Bug are law. Act accordingly.