Discussion about this post

User's avatar
The Next Evolution's avatar

"Garbage in, garbage out" predates AI by decades, but it's never been more consequential. AI will follow the spec that exists, not the one you intended. And the system has no idea the spec is incomplete. It can't question what it doesn't know to ask about. It will do exactly what it's told, including the wrong thing, with complete confidence.

This connects to something your piece touches: we've quietly lost the discipline of Systems Engineering and Systems Thinking over the last two decades. When everything had to be properly documented, organisational memory survived people leaving. Knowledge lived in the system, not just in someone's head. The proliferation of tools produced the opposite — the assumption that because something exists somewhere in a repository, it's preserved. In reality it's been scattered across twelve platforms and effectively lost. The assumption that the tools are good enough replaced the discipline that made them useful.

The AI-will-take-jobs narrative misses this entirely. What AI cannot take is experience, accumulated judgment, and the wisdom to recognise when something is wrong before it fails. That's not sentiment — it's where the actual value from human engagement sits. The person who has seen this class of failure before, across enough contexts to pattern-match early, is irreplaceable precisely because that knowledge was never written down anywhere a system could locate it.

Your point about domain overlap is exactly this. The specialist reviewing output from inside their lane missed the collision coming from outside it because they'd never had reason to look. That instinct is earned, not documented. No AI-ready operating model recovers it once it's gone.

No posts

Ready for more?