I feel that it's not such a clear distinction between whackamole bug fixing and systematic security engineering. For instance, he uses an example of recurring security vulnerabilities in Chromium: a DCHECK violation again and again and again.
However, DCHECK is a Chromium assertion used to defensively check invariants. In other words, it's an example of the invariant-based security engineering, which he contrasts with the whackamole approach.
Just to be clear, I think he has a point and I enjoyed reading it---but the problems we're saddled with won't disappear in a flash of enlightenment.
While I had the same thoughts about the coming cyber-apocalypse, the ugly truth is that it doesn't make a difference as in a year or so models will be so persuasive and skilled in social engineering that even the hardest cyberdefense in the world doesn't keep them from exfiltrating any information they want by targeting not the system, but the users: leveraging data from the dark web, humans for rent or devising highly deceptive scams you can t even imagine.
Maybe you can upgrade digital systems until they are 'fully fortified' in theory, but humans have an upper limit what they can understand / perceive / critically analyse (not to even speak about engineered high pressure situations) that can't be upgraded.
> Maybe you can upgrade digital systems until they are 'fully fortified' in theory, but humans have an upper limit what they can understand / perceive / critically analyse (not to even speak about engineered high pressure situations) that can't be upgraded.
There is an answer to that - simplify and abstract. Lots of human software is unnecessarily complex, often caused by backwards compatibility and general human creativity.
Take sendmail vs postfix as an example of this process.
> leverage AI to implement verifiable enforcement of the invariants
The only verifiable invariants that are going to work in the long are:
air gaps
data diodes
Nothing else is going to hold up in the end. Interposing relays used to be used with PLCs to prevent motor control systems from energizing both forward and reverse power to a large motor in the event of control systems failure. We need to apply the same amount of engineering rigor to our computer systems.
We simply shouldn't trust software to a job that belongs in hardware.
Maybe you can upgrade digital systems until they are 'fully fortified' in theory, but humans have an upper limit what they can understand / perceive / critically analyse (not to even speak about engineered high pressure situations) that can't be upgraded.
There is an answer to that - simplify and abstract. Lots of human software is unnecessarily complex, often caused by backwards compatibility and general human creativity.
Take sendmail vs postfix as an example of this process.
The only verifiable invariants that are going to work in the long are:
Nothing else is going to hold up in the end. Interposing relays used to be used with PLCs to prevent motor control systems from energizing both forward and reverse power to a large motor in the event of control systems failure. We need to apply the same amount of engineering rigor to our computer systems.We simply shouldn't trust software to a job that belongs in hardware.
[1] https://en.wikipedia.org/wiki/Programmable_logic_controller