AI adoption
What can go wrong?
...everything. A senior engineer once pushed back to me, "What do you think can go wrong?" when I asked them to attend a major go-live.
…everything. A senior engineer once pushed back to me, “What do you think can go wrong?” when I asked them to attend a major go-live.
Thankfully (or worryingly) they were from a large, external software organisation. What can go wrong? All the things. All of them. It’s Murphy’s Law“If there are two or more ways to do something and one of those results in a catastrophe, then someone will do it that way.” Edward A. Murphy Jr.. It’s a complex system. A system with dozens of integrations and millions of users. What am I worried about? What I am not worried about would be quicker.
Chaos is the normal state
In a major re-platforming cutover, asking me for specific risks is upside-down thinkingand not in an inverse thinking good way. Everything, anything, all the possibilities. It’s entropy. We are always descending towards chaosDr Richard Cook: complex systems run broken, held together by redundancy and people adapting. Chaos is the normal state. That’s the natural order of systems, and technology is no different.
AI in critical systems
So when I hear of people plugging untrained Chat AgentsYou’re only ever one prompt away from total disaster directly into mission-critical software, I know exactly what I am worried about: chaos. Maybe it will purr along without issue for days, weeks or even months. But then it will do something unexpected. Then again. Then again. Ironically, if you understand AI (and programming generally), you know the only thing you can expect with certainty is the unexpected.
Always assume that anything a system can do, it eventually will dothe principle of least privilege has never been more important. Giving AI full control rights into your critical business systems? Or even just more rights than it needs? I fear you might become a future anecdote about recklessness and AI.
Don’t let fear decide
I really empathise with the fear of being left behindFOMO is a real pressure for people and businesses in the AI age in the AI arms race - but don’t let that fear make you do something reckless. In haste, if you give an AI agent too much access then, at some point, Chicken Little will be right and the sky really will fall in.
Entropy. The universe is on my side on this one.