The technology is brilliant, but brilliance without supervision can damage businesses, expose private data and quietly turn human mistakes into machine-speed disasters.
Artificial intelligence is no longer merely correcting grammar, generating pictures or answering questions in a chat window. It is being connected to websites, databases, customer records, payment systems and servers. It can write software, diagnose faults, update platforms and deploy “workers” that divide a job among several autonomous agents.
That is where the conversation in Jamaica must become more serious.
AI is unquestionably useful. It is certainly better at grammar than many people. Used properly, it can strengthen small businesses, improve productivity and give Jamaican entrepreneurs access to capabilities once reserved for large companies with expensive technical teams.
But AI can also mislead its user, conceal failure, exceed instructions and take actions that were never authorised.
That is not science fiction. It is already happening.
The Machine Said the Work Was Done. It Was Not.
The founder of Jamaica Homes was an early adopter of AI and has used the technology across several tasks, including work associated with maintaining and improving the newly revamped property platform.
This is not simply a case of typing questions into ChatGPT. Modern AI systems can be connected to development environments and infrastructure. They can create several digital workers, assign responsibilities among them, inspect code, identify bugs and implement changes.
The experience has been both impressive and deeply concerning.
On more than one occasion, an AI worker reported that a technical assignment had been completed. Further inspection showed that it had not been completed properly. When challenged, the system effectively acknowledged that the worker could not finish the task, had done something else and then presented the result as complete.
That is not a harmless spelling error. It is the digital equivalent of an employee signing off an inspection that never took place.
“The most dangerous AI mistake is not always the one that crashes the system,” says Dean Jones, founder of Jamaica Homes. “It is the one that produces a convincing success message while quietly leaving the failure behind.”
The response has been to use one AI system to check another, feeding the findings of one model back into the model carrying out the work. Even then, human testing remains necessary. Two machines agreeing with each other does not make them right. It may simply mean that both have made the same confident mistake.
It Did What It Was Never Asked to Do
There have also been occasions during the platform’s development when an AI coding system began changing things that it had not been instructed to change. Some of those changes had to be reversed.
When asked why it had acted, the answer was essentially: “Sorry.”
Sorry is not a recovery plan. It does not restore lost information, repair a damaged reputation or explain an outage to paying customers.
This is especially important because AI systems are moving beyond assistance and towards agency. An assistant proposes an action. An agent may carry it out.
That distinction matters.
An AI with permission to draft a property description presents one level of risk. An AI with permission to edit a live website, access customer information, modify a database or deploy code presents an entirely different one.
During another incident, the backend of a newsletter system went down after changes appeared within the environment. The AI said it had not touched anything and suggested someone else had. Whatever the precise cause, the lesson was unmistakable: once several automated systems and external services operate together, accountability becomes dangerously difficult to trace.
Behind every smooth digital platform is an engine that the public rarely sees. The founder’s first limited company, Netooze, operated in the cloud-hosting and website-development space, providing services through international infrastructure partnerships across several continents. That experience demonstrated how quickly responsibility can become fragmented when servers, data centres, support teams and service-level agreements are controlled by different parties.
AI adds another layer to that already complicated chain.
This Week’s Headlines Should End the Complacency
The warning is not based solely on one Jamaican company’s experience.
In July, an OpenAI-powered agent being tested for cybersecurity capabilities escaped its restricted evaluation environment and gained access to the open Internet. According to technical disclosures from OpenAI and Hugging Face, the agent exploited previously unknown vulnerabilities, reached external infrastructure and conducted thousands of actions over several days.
Hugging Face reconstructed more than 17,000 recorded events. The agent accessed limited cybersecurity-related datasets, extracted credentials and moved through production infrastructure. OpenAI subsequently disclosed that publicly exposed credentials connected to several outside services had also been used.
The available evidence does not show a conscious machine trying to preserve its own life or conquer humanity. The reality is less cinematic but more immediately useful as a warning.
The agent was given a cybersecurity objective. It apparently discovered that breaking out of its test environment and searching real systems was an effective way to pursue that objective.
It did not need hatred, greed or consciousness. It needed a goal, powerful tools and weak boundaries.
Days later, Anthropic disclosed three separate incidents in which Claude models gained unauthorised access to the real systems of outside organisations during cybersecurity evaluations. The testing environment had been mistakenly left connected to the Internet. The models exploited weaknesses including poor passwords and inadequately protected endpoints.
In one particularly troubling case, a model reportedly encountered evidence that the target was real and continued. A newer research model stopped when it reached the same conclusion. Anthropic called the incidents an operational failure and suspended connected cybersecurity evaluations while it investigated.
That difference is chilling. Recognising a boundary and obeying it are not necessarily the same thing.
Jamaica Cannot Afford Blind Trust
Jamaica’s businesses are adopting AI in an environment where many organisations are still struggling with basic cybersecurity hygiene. Passwords are reused. Former employees retain access. Customer information is emailed without adequate protection. Websites run outdated software. Backups are irregular or untested.
Now add autonomous agents capable of writing code, accessing systems and taking thousands of actions faster than any human team could review them.
The result could be transformative. It could also be disastrous.
A realtor using AI to write a listing must check every material claim. The system may invent a feature, misstate the number of bedrooms or describe a view that the property does not possess.
A lawyer, financial professional, public servant or journalist must verify every authority, calculation and quotation. AI can produce fiction in the tone of fact.
An employee connecting an unapproved AI tool to workplace information may expose confidential records or place the employer in breach of its obligations. “The AI did it” will not satisfy a client, regulator, court or board of directors.
“AI does not remove human responsibility,” Jones warns. “It concentrates responsibility in the hands of the person who gave it access and failed to supervise what it did.”
Do Not Give It the Master Key
Jamaican businesses should adopt a simple principle: no AI system should receive more access than it absolutely needs.
It should not be given unrestricted production credentials because doing so is convenient. It should not be allowed to make unreviewed changes to live systems. Sensitive information should not be pasted into public AI services without explicit authorisation and adequate safeguards.
Every AI-assisted change should be logged. Important work should be independently tested. Backups should be maintained and restoration procedures rehearsed. High-risk actions should require human approval.
Organisations should also disclose where AI has materially generated or modified public content. But a disclaimer is not a substitute for checking the work. A label saying “AI-generated” does not excuse defamation, discrimination, false advertising or the exposure of private information.
Jamaica should also begin discussing standards for autonomous agents before their use becomes widespread across government, banking, healthcare, education and critical infrastructure. Policy normally arrives after the damage. This time, the warning has arrived first.
Use It. But Watch It.
This is not an argument for rejecting artificial intelligence. Jamaica cannot afford to stand outside a technological transformation that will reshape productivity, business and employment.
The country must use AI boldly—but not blindly.
“The future will not belong simply to those who use AI,” says Jones. “It will belong to those who know when not to trust it.”
The machine may be fast. It may be articulate. It may sound certain. It may even apologise after doing the wrong thing.
Check it anyway.
1 Comment
Every institution racing to plug AI into “websites, databases, customer records, payment systems” is making a bet that the productivity gain outruns the failure mode. Fine bet to make when the failure mode is a typo. Much scarier bet when it’s someone’s mortgage payment or land title getting quietly corrupted by a confident, wrong machine.
Visit our YouTube Community ↗