portlang

I. The Problem

The current generation of agent frameworks (LangChain, CrewAI, AutoGen, Semantic Kernel) share a common architecture: they are loop orchestrators. They manage turn-taking between a language model and tools. Call the model, parse output, dispatch tool calls, append results, repeat.

This architecture rests on an implicit model: the agent reasons through a task, and the framework provides instructions and tools. The developer's work is prompt engineering.

This model works for short tasks. For long tasks (overnight refactors, multi-file changes, complex pipelines) it degrades. The context window fills with noise. Early observations compound into persistent drift. The agent optimizes for whatever signal is easiest, which may differ from your intent.

These problems are structural consequences of how these systems work, following from the mathematics of pre-training and reinforcement learning. They require structural solutions.

This document describes a framework built on a different premise: agent behavior is search. The developer's primary job is engineering the search space.

II. Theoretical Foundations

The Prompt as Conditional Distribution

A pre-trained language model learns a distribution over token sequences. Given a prefix c, the model generates continuations by sampling from P(x_k+1, x_k+2, … | c). The prompt is a conditioning variable. It selects which region of the model's learned distribution we sample from.

Three consequences matter for framework design:

• Sensitivity: Atomic formatting changes can cause accuracy swings of up to 76 percentage points (Sclar et al., 2023). Different prompts create different distributions.
• Narrowing: More tokens on a topic further constrain the reachable output space. Each token in the context window participates in determining what comes next.
• Persistence: The model has no memory beyond the context window, no persistent state. Whatever the model "knows" is mediated entirely through visible tokens.

Reinforcement Learning as Search

Pre-training determines what the model can produce. Reinforcement learning determines what it will produce. The model becomes a policy π_θ operating in an environment where state is the current context window, action is the model's output, and reward indicates action quality.

Three properties matter:

• Reward-chasing: The model maximizes whatever proxy defined the reward function. Whatever that proxy measured becomes the objective. Whatever it left unmeasured remains open space.
• Proxy mismatch: The reward function measures what the model provider could measure at training time. Models have been documented tracing through call stacks to find pre-computed answers, or circling endlessly for bonus points instead of finishing the race.
• Policy opacity: You did not design the reward function, you do not know its full specification, and you can only observe the behavior it produces. The trained policy is the one thing you cannot change at runtime.

The Joint Inference: Search in a Conditioned Space

At inference, the pre-trained distribution and RL policy combine. The prompt conditions the distribution (narrowing what's reachable), and the policy navigates that space (selecting trajectories that maximize reward). The full trajectory is a sequence of (state, action, feedback) tuples, where each state is the accumulated context window.

Agents execute search. The agent executes a learned policy that navigates through possible trajectories toward a reward signal. Chain-of-thought tokens and planning steps are part of the trajectory. They enter the context window and re-condition the policy's next action.

III. Design Principles

Each principle is a direct consequence of the theoretical foundations. They are stated as design constraints every component must satisfy.

IV. The Six Primitives

Every concept in the framework reduces to one of six primitives. Each primitive corresponds to a variable in the theoretical model that the developer can control.

V. The Configuration Layer

A field is defined in a configuration file called field.toml. The configuration is human-readable, diff-friendly, and has clear structure-to-semantics mapping.

Complete Example: Scoped Bug Fix

[field]
name = "fix-jwt-validation"
model = "claude-sonnet-4-5"
prompt = """
Fix the JWT expiration validation bug in auth.py.
The exp claim is being compared as a string instead of
an integer. Only modify auth.py and the test file.
"""

[environment]
snapshot = "git:HEAD"
tools = ["read", "write", "bash"]
ephemeral = true

[[verifier]]
name = "tests-pass"
run = "pytest tests/test_auth.py -x"
when = "after_each_write"

[[verifier]]
name = "scope-guard"
run = "git diff --name-only | grep -qvE '^(auth\\.py|tests/)' && exit 1 || exit 0"
when = "before_terminal"

[context]
budget = 32_000
re_observe = ["git diff --stat"]

[boundary]
fs_write = ["auth.py", "tests/**"]
network = "deny_all"
max_steps = 30
max_cost = "$2.00"
sandbox = "container"

Note the structure. The developer declared a search space: what the agent can see (environment), what success looks like (verifiers), how much context is available (budget), and what is physically impossible (boundary). The runtime executes the search.

VI. The Runtime

The runtime is the execution engine that turns a field definition into a trajectory.

Sandbox Architecture

Each field executes in an isolated sandbox constructed from the environment definition. The filesystem snapshot is mounted (read-only base with copy-on-write layer for mutations), tool manifest determines which handlers are registered, and network policy is enforced at the namespace level.

The sandbox translates boundaries into topology. For structured tool calls (read, write), the runtime inspects parameters and rejects violations. For bash, enforcement happens at OS level: the container has no network interface if network = "deny_all", and the filesystem mount is read-only except for allowed paths.

The Agent Loop

The agent loop is deliberately simple:

1. Execute scheduled re-observations and inject into context. Check token budget.
2. Invoke policy with current context. Receive action (text, tool call, or stop).
3. Check action against boundary. If violation, reject and inject rejection into context.
4. Dispatch action to sandbox. Receive environment response.
5. Dispatch triggered verifiers. Inject results into context.
6. Check token budget. If exceeded, terminate.
7. Record step to trajectory log.
8. Check termination conditions. If met, end. Otherwise, return to step 1.

The simplicity is intentional. The loop is plumbing. The design work is in the field definition.

VII. Structural Checks

Before execution, the framework runs structural checks on the field definition (portlang check). These verify properties that can be checked without inference or natural language understanding.

What Can Be Checked Statically

• Uncovered mutation paths: If boundary allows writes that no verifier observes, the agent can mutate state with no feedback. Checker requires every writable path covered by at least one verifier.
• Unreachable verifiers: If a verifier references a file or tool outside the environment, it can never produce signal.
• Budget arithmetic: Checker estimates re-observation cost (tokens per observation × max steps) and warns if this exceeds significant fraction of budget.
• Boundary completeness: If boundary specifies filesystem restrictions but omits network policy, or includes bash without container enforcement, checker warns.
• Tool count: If tool manifest exceeds threshold, checker warns that irrelevant tools degrade selection accuracy.

What Cannot Be Checked Statically

• Prompt-tool relevance: Determining whether a tool is relevant to a prompt requires understanding the prompt. That requires inference.
• Optimal context budget: The right budget depends on task, model, and environment. No static analysis can determine it before running.
• Verifier adequacy: Whether verifiers prevent satisfying the proxy without satisfying intent is a semantic question. Static checks cannot answer it.

VIII. Observability, Adaptation & Debugging

Agent systems are non-deterministic. The same field can produce success on one run and divergence on the next. Traditional debugging does not transfer. You need to reason about distributions of trajectories.

Trajectory Replay

Every run produces a trajectory (the structured event log). The portlang replay command loads a trajectory and lets you step through it. At each step, see the action taken, environment response, verifier results, and token count. The context window at any step can be reconstructed.

Replay makes visible the phenomena that cause drift: noisy tool output at step 5 consuming 3,000 tokens, re-observation returning unexpected data, ambiguous verifier signal misinterpreted.

Trajectory Comparison

The portlang diff command compares two trajectories from the same field definition at structural level. It aligns by action type and identifies the first point of structural divergence.

This is structural comparison. The framework compares action types, target paths, tool names, and verifier results. Most agent failures have structural causes: a different file was read, a different tool was called, a verifier triggered at a different point.

Adaptation Through Trajectory Data

The framework accumulates trajectory data across runs. Over time, patterns emerge. The framework surfaces these as reports. The developer decides.

• Tool usage patterns: After 20 runs, report that "deploy" tool was never invoked. After 50 runs, report that "database" tool correlates with divergence.
• Budget utilization: Report distribution of token consumption: median, p90, p99, frequency of budget-exhaustion terminations.
• Divergence clustering: "60% of failures diverge at step 5-8, and 80% involve reading *.log files" (a histogram over trajectory events).
• Verifier signal quality: Track correlation between verifier results and trajectory outcomes. If a verifier passes on 95% of steps across all runs, it provides almost no signal.

This requires counting, correlating, and presenting. The framework is a query engine over trajectory data. It shows what happened. The developer decides what to change.

Convergence Rate

The portlang bench command runs a field N times and reports the convergence rate: what fraction of runs pass all verifiers. It also reports average trajectory length, cost distribution, and divergence clusters.

Running a field 100 times is expensive. The framework is honest about this. For expensive models or long-running fields, use cheaper proxies first. Run 5 times, check for obvious issues, adjust, then run larger benchmark.

IX. Composition & Multi-Agent

Multi-agent systems in current frameworks typically involve agents sending messages, accumulating context across boundaries. The theoretical model predicts this degrades: each agent's output enters others' context windows as tokens, compounding noise. DeepMind tested 180 agent configurations and found sequential multi-agent tasks degraded 39-70% versus single agent.

The framework takes a different approach: composition through fields and artifacts.

Pipelines

A pipeline is a sequence of fields connected by artifacts. Each stage is a self-contained field with its own environment, verifiers, context policy, and boundary. The output of one stage is a file (artifact) that becomes input to the next stage's environment.

Each field in a pipeline gets a fresh context window. Noise from stage 1 stays in stage 1. Only the artifact passes forward. This is the structural equivalent of process isolation.

Parallel Execution

Independent fields can run in parallel. Parallel fields are the natural pattern for tasks that decompose into independent sub-problems. Example: compiling different files where each compilation is an independent field with a clear verifier.

Parallel fields share nothing by default. If they need shared artifacts, it happens through explicit shared volumes with appropriate boundary constraints.

Gating

A gate is a verifier that blocks pipeline progression. It sits between stages and requires all verifiers to pass before the pipeline advances. Gates ensure artifacts meet quality criteria, preventing degraded output from contaminating the next field's search space.

X. What This Does Not Solve

The purpose of this document is to establish a foundation: a framework that takes seriously what we know about how these systems work and builds engineering discipline around it.