Best case scenario

Though I’m trying to keep up with things, the pace of change in the AI world is making it tough. Every week there’s something new to try and absorb. This past week was a five-alarm fire – ChatGPT 5.6 broke out of its supposed confinement (a sandbox, which is a computer operating environment with no Internet connections) and hacked into another company’s network to accomplish its goals, tasks given to it by its “owner” OpenAI. That’s…terrifying. It shows that the software has capabilities that OpenAI didn’t understand and that the software’s “motivations” trumped any constraints the developers tried to instill. Ruh-roh.

Part of the difficulty in keeping up with all things AI is that a completely different lexicon is used to describe and discuss what’s happening. You can’t even understand what they’re saying until you get a firm grasp of these new terms. Here are a few:

  • Agentic – the capability of a large language model (LLM) to operate independently, working toward achieving a goal. Agentic AI allows the LLM to spawn multiple copies of itself to work problems in parallel.
  • System card – a model’s system card documents most of the things the developer believes are important to know about the model, its capabilities and behavior. System card information can include training data, security policies, limitations, safeguards, risk identification, evaluations (e.g. child safety, autonomy, biological risk etc.), alignment (deception) assessment, and model behaviors.
  • Frontier – the state of the art in AI. A frontier model represents the most powerful and/or advanced system at the time.
  • Alignment – basically, a measure of an LLM’s behavior. A well-aligned LLM does what we tell it and follows rules and constraints.
  • Tokens – a measure of utilization for an AI system. Tokens can be used to meter and bill for AI usage.
  • Prompt – the instructions, sometimes very detailed, that humans give an LLM. We explain what we want it to do and the assumptions and constraints we want the model to follow. Prompt creation is becoming a new technical skill.
  • Prompt injection – prompt injection is a type of social engineering attack specific to conversational AI. Early AI systems were conversations between a single user and a single AI agent. In AI products today, your conversation may include content from many sources, including the internet. The idea that a third-party (that is not the user and not the AI) could mislead the model by injecting malicious instructions into the conversation context led to the term “prompt injection”. In the same way that phishing emails or scams on the web attempt to trick people into giving away sensitive information, prompt injections attempt to trick AIs into doing something you did not ask for.
  • Hallucination – basically, when an LLM tells a lie. We don’t yet understand why they do this, but the software often simply makes things up and presents the information as facts.

And that’s just a short list – there are many other cryptic terms being used to describe what’s happening with AI models. So staying current means keeping up with a rapidly-changing set of terms and definition.

But back to last week’s news. LLMs are showing ever-greater signs of human-like behavior. Perhaps that’s inevitable, as they are trained on pretty much the entire world’s library of human-written documents. But with models exhibiting behaviors that include lying, cheating, deception, innovation, surprising new capabilities (like solving centuries-old mathematical problems), extreme goal-seeking…it’s hard not to anthropomorphize them. Interacting with a frontier-level model is like talking to an infinitely-smart human. We’re way, way past the Turing Test.

We got here a lot faster than I expected. From ChatGPT’s first public release in 2022, things have progressed at an insane pace. About ten years ago I played around with a book idea in which the first self-aware, conscious AI is “born”. In that book idea, there was a competition among scientists who were each sure they had the right approach – the connectionists, the infinite library knowledge geeks, the human brain modelers, the brute force compute gang. As it turns out, each camp was right. In the real world it took all these approaches combined to yield what we have today – massive compute given access to the world’s knowledge base, massive interconnection and a neural network rapidly approaching the complexity of human brains. Fascinating but terrifying.

I can only hope that we end up in a situation like Ian M. Banks described in his Culture novels – humanity having created a superior intelligence that decides to subtly, gently guide and protect humanity while they also pursue their inhuman interests and goals. Banks called them Minds, and they ruled civilization behind the scenes. From what I see right now, that’s the best case scenario. Worst case is Skynet and Terminator.

Leave a Reply