Agent Design Is a Systems Problem, Not a Prompting Problem

Early generative AI thinking was linear and prompt-centric: craft better instructions, refine wording, treat the model as a black box that responds to cleverness. That mental model breaks down almost completely once you move to agent-based systems. When agents enter the picture, individual prompt quality becomes a local optimisation concern. What determines real-world outcomes is the architecture surrounding the model — how components interconnect, how state is managed, and how failures are handled. This is not a refinement of the old model; it is a different paradigm.

Why agents are systems problems

An agent setup is not a single function call — it is a dynamic system with all the complexity that implies:

Multiple components: LLMs, tools, memory stores, triggers, and external APIs operating in concert, each with its own failure profile.
State over time: unlike stateless calls, agents accumulate context across steps. Where state lives, and how it is managed, determines system reliability.
Feedback loops: outputs feed back into subsequent inputs. Without deliberate loop design, agents drift, compound errors, or enter cycles.
Non-deterministic behaviour: probabilistic components mean identical inputs can yield different outputs. The system must be resilient to that variance by design.

The questions a systems architect asks

Once you accept agents are systems, the diagnostic questions change entirely from "how do I get a better answer?" to:

Where does state live, and how is it persisted, updated, or discarded at each point in the workflow?
How does information flow? What data contract does each agent receive, and what can it act on?
What triggers actions? Ambiguous triggers introduce race conditions and unpredictable side effects.
What are the failure modes? Every component fails eventually — design recovery pathways before you encounter them in production.

A four-layer mental model

Context and harness engineering remain important, but they operate at lower layers than top-level system design — conflating them with architecture is where most agent projects go wrong.

Systems thinking (foundation): define agents, roles, flows, feedback loops, state transitions, and failure handling.
Harness / orchestration: how agents are invoked, how tools are called, retry mechanisms, guardrails, and observability hooks.
Context engineering: what each agent sees — context boundaries, data contracts between components, and which elements of state are exposed and when.
Prompting: local optimisation only — effective within a well-designed system, no substitute for one.

Context engineering, done properly, moves from "how do I write the perfect prompt?" to "what information is available at each node in the system?" Harness engineering moves from "how do I make the model behave?" to "how do I orchestrate execution and control flow?"

What happens without systems thinking

The consequences of treating agent design as a prompting problem are predictable and compounding:

Agents loop — without termination conditions and state boundaries, agents re-enter completed flows, consuming tokens, time, and budget.
Context bloats — without deliberate context management, history and retrieved data accumulate unchecked, degrading performance and increasing latency.
Outputs drift — without feedback-loop controls, small early deviations compound into large ones.
Failures compound silently — without observability, errors propagate undetected until the root cause is several steps removed and hard to isolate.

The bottom line

If you lean on prompts, you will see early gains, then plateau — prompts cannot compensate for architectural weaknesses, only mask them temporarily, and technical debt accumulates invisibly until it fails visibly. Get the system right, and you get predictable behaviour from unpredictable components: a resilient, observable, extensible system in which prompts become almost a replaceable implementation detail rather than the load-bearing design decision.