Search from Index to Browser

Index durable session content, expose ranked and freshness-aware results through HTTP, and render them safely in the web application.

Search is a full-stack feature: content becomes index input, a query crosses HTTP, ranking returns evidence-shaped results, and the browser must display both text and trust state safely. This chapter keeps the deterministic lexical/hybrid baseline, then connects it to /api/search and the browser search panel before comparing more complex retrieval adapters.

Chunking defines what can be found

A chunk has a recoverable document id, ordinal, source text, and content fingerprint. Code chunks need enough context to explain a symbol; session-history chunks need enough context to preserve a user or assistant turn. Chunk size, overlap, symbol boundaries, generated-file policy, and event-kind inclusion are workload variables rather than universal constants.

from autocode.search.chunkers import chunk_text

chunks = chunk_text("cli.py", "\n".join(f"line {i}" for i in range(5)), lines_per_chunk=2)
assert [chunk.ordinal for chunk in chunks] == [0, 1, 2]
assert chunks[0].document_id == "cli.py"
print("chunk sizes:", [len(chunk.text.splitlines()) for chunk in chunks])

chunk sizes: [2, 2, 1]

The chunk ids recover the source document and ordinal. Production search results should also carry revision, path or session cursor, symbol, line range, and fingerprint so the UI and agent can distinguish current evidence from a stale match.

Carry a query across the stack

AutocodeApplication indexes durable user and assistant messages after repository append. GET /api/search?q=... calls that application boundary and serializes document_id, score, text, and method. The browser URL-encodes the query, renders each result with textContent, and shows an explicit empty state.

The local index is process memory so the deterministic lesson stays small. Chapter 09 turns rebuild into a background job; a production adapter persists vectors or postings and records index version and freshness.

from tempfile import TemporaryDirectory

from fastapi.testclient import TestClient

from autocode.runner import DemoAgentRunner
from autocode_service.api import create_app

with TemporaryDirectory() as directory:
    app = create_app(database_path=f"{directory}/sessions.db", runner=DemoAgentRunner())
    with TestClient(app) as client:
        session_id = client.post("/api/sessions", json={"title": "search"}).json()["session_id"]
        with client.websocket_connect(f"/ws/sessions/{session_id}") as socket:
            socket.send_json({"type": "user_message", "content": "explain journal recovery"})
            event = socket.receive_json()
            while event["kind"] != "run_finished":
                event = socket.receive_json()
        hits = client.get("/api/search", params={"q": "journal recovery"}).json()

assert hits
assert hits[0]["document_id"].startswith(f"session:{session_id}")
assert {"document_id", "score", "text", "method"} <= hits[0].keys()
print("top browser result:", hits[0]["document_id"], round(hits[0]["score"], 3))

top browser result: session:664c2bcc-1019-48de-8b23-657e429f66cc:1#0 1.383

The query crosses the same session run that produced the content, then returns through HTTP as a ranked result. This proves wiring, not retrieval quality. A held-out lookup set still determines whether the ranking helps users or the agent; the endpoint cannot make a weak index accurate.

Separate relevance from freshness

The offline baseline scores token overlap and exact lexical presence. It is not a neural embedding model; it is a reproducible comparator for recall, reciprocal rank, latency, and cost. Freshness is an independent result. A high-scoring stale chunk should trigger invalidation or reindexing and carry a visible stale state rather than silently enter an agent prompt.

from autocode.search.index import LocalSearchIndex

index = LocalSearchIndex()
source = "class SessionRepository:\n    def recover(self):\n        replay_journal()"
index.add("store/repository.py", source)

hits = index.search("recover journal")
assert hits[0].document_id == "store/repository.py#0"
assert index.freshness("store/repository.py", source) == "fresh"
assert index.freshness("store/repository.py", source + "\n# changed") == "stale"
print("result:", hits[0].document_id, "fresh -> stale after source change")

result: store/repository.py#0 fresh -> stale after source change

The browser should show source identity, score or ranking explanation when useful, and freshness state. The agent tool should receive the same evidence fields. Registering search changes available evidence, not the harness’s think–act–observe loop. Chapter 12 reports retrieval metrics separately from downstream task success so an attractive result card does not become proof of impact.

Exercises

Create three held-out browser search questions with relevant document ids. Report Recall@1 and Recall@3, verify the HTTP response shape, and make one modified document appear as stale or disappear until reindex completes.

[P06.1] Evaluate a search API honestly

Create three held-out queries and relevant document ids, measure Recall@1 and Recall@3 through /api/search, and include one changed document. Explain what the browser and agent should do with stale evidence.