From Zero Context to Shipping

What AI changes about understanding unfamiliar systems

I spent a morning reasoning carefully about a code path that could never run.

The explanation was clean. The types lined up, the downstream logic made sense, and I had enough evidence to keep going. I was also wrong.

This was shortly after I joined Kepler. Joining a new engineering organization has always come with a context tax: you need a model of where state lives, how data moves, which boundaries matter, and which assumptions are intentional. Traditionally, that model accumulates over weeks of reading code, shipping changes, and talking to the people who built it. This time, much of that context accumulated alongside the work. I was moving across Rust services, Python data pipelines, agent infrastructure, and Kepler-specific orchestration while still building my model of how they interacted. AI didn’t give me that model. It made it cheaper to construct and test one as I went.

The obvious use of AI here is code generation, and yes, it can write plenty of code. But that turned out to be one of the less interesting sources of leverage. I found it more useful for interrogating the system: trace this value back to where it originated. Show me every path that transforms it. What consumes this downstream? Where is this property actually enforced? If the representation changes here, what assumptions break later? Questions that once required manually tracing several paths through the repository became cheap enough to ask continuously.

I saw this clearly while tracing provenance through our citation infrastructure. At Kepler, we want every claim an agent makes to remain traceable to the underlying financial evidence, so provenance has to survive the entire path from source data to answer. Certain XBRL labels were producing malformed citations, so I used AI to trace the representation backwards: every place citations were constructed, the callers that fed them, and the assumptions each path made about source data. Nothing was wrong with the financial data itself. Characters perfectly valid in an XBRL label also had structural meaning inside our citation format. Several construction paths allowed source data to enter that representation without preserving the distinction. The answer could still look right while the provenance attached to it was wrong. The immediate failure was narrow, but the property it touched wasn’t: source data should never be able to change the semantics of the mechanism used to prove where it came from. Enumerating the construction paths and tracing their callers was the easy part. The harder question was where to enforce that property once, rather than depend on every caller getting it right.

The same speed made it easy to run with a good explanation too early. While tracing how we handled Anthropic API failures, I asked AI to walk the path end to end. The explanation was convincing: the provider returned an error, our error type deserialized it, and the resulting status drove retry and fallback behavior. I spent the morning reasoning from that model. It held until I compared it against an actual provider response. The payload was nested differently from the type we were deserializing it into, which meant real provider errors could fail before ever reaching the retry and fallback logic I had been reasoning about. The type existed. The downstream code existed. AI had connected them correctly. What neither of us had established was that the contract at the boundary was real.

AI hadn’t invented an API or hallucinated a call path. It had produced a coherent explanation from code that was actually there. It was still wrong about the system’s behavior because one plausible assumption had quietly been promoted to fact. Coherent wasn’t the same as established. And I don’t think that failure is separate from the speedup. The same synthesis that let me reconstruct an unfamiliar execution path quickly also made it easier to miss where the evidence stopped. Reading the code manually would have been slower, but I would have had to establish each transition myself. AI could hand me the whole path at once, including the transition it had inferred. Some of the friction, it turns out, was doing epistemic work.

I started keeping a mental distinction between what I’d observed in code, tests, state, or runtime behavior, what I’d inferred, and what remained unknown. AI is very good at making inferred feel like observed. The type existed: observed. The downstream code behaved a certain way: observed. That real provider responses matched the type: inferred. It also changed the questions I asked. Instead of asking AI only to explain the system, I started asking it to attack the explanation: Show me the actual external contract. Find another path where this assumption doesn’t hold. Show me where this property is enforced — the actual line, not the intent. What would make this explanation wrong? Nothing moved from inferred to observed because an explanation sounded convincing. It moved when I saw the code, ran the test, inspected the state, or watched the behavior.

Once I started looking at the boundary this way, other distinctions became harder to ignore. A streaming response could stop partway through generation and travel through the same completion path as a successful one. From downstream, a half-written answer could look finished. I followed the event forward through our Rust types, deserialization, stream handling, retries, and event propagation. Then I worked backwards: given the state we observed at the end, what paths could have produced it? That exposed other places where information was being collapsed. An unparseable error could acquire a status the provider had never returned. Different termination conditions could converge on the same application state. The problem wasn’t that a model provider could fail. That was expected. The problem was losing the information that told us how it had failed. AI made those paths faster to reconstruct. Which distinctions belonged in the contract was still an engineering decision. A partial stream, a provider error, and a clean completion aren’t implementation details if downstream behavior depends on telling them apart.

The parallel to what we’re building at Kepler was hard to miss. A fact pulled from a filing, a financial metric calculated from known inputs, and an agent’s interpretation of why that metric changed can all appear in the same answer. A model can make them sound equally certain. They aren’t. The fact needs provenance, the calculation should be reproducible, and the interpretation needs to remain grounded in the evidence underneath it. I was applying the same standard to AI’s explanations of the codebase: what was grounded in something I could inspect, and what merely followed convincingly from everything around it?

I expected AI to make me faster at writing software, and it did. What I didn’t expect was how quickly it could get me from unfamiliar code to the question of where a property should be enforced once. The speed is real; the question is which parts of the path I’ve actually established and which arrived already assembled for me. The distance from zero context to shipping is getting shorter. Knowing what should ship is still the hard part.