Most AI benchmarks still do a mediocre job of explaining how useful models are in the kinds of agentic workflows businesses actually care about.

That is not because benchmarks are useless. They are useful. But many of them are still optimized around narrow tasks, coding-heavy environments, academic-style reasoning problems, or synthetic prompts that do not look much like how knowledge workers actually use AI systems. The more practical question is different:

If I connect an AI agent to my company’s data, give it tools, and ask it to produce a useful piece of work, how close does it get to something I can actually use?

That is the question behind the Diligence Stack Agent Bench.

The benchmark is built around a view we have had for a while: outside of coding, one of the most common enterprise agent patterns will not be “let the model roam the open internet and hope for the best.” It will be something more structured. Companies will create MCP servers, internal tools, and knowledge systems that connect their proprietary data to applications like ChatGPT, Claude, and their own internal agents.

That means agent performance is not just about the model. It is also about the surrounding infrastructure: the retrieval system, the quality of the knowledge base, the tools available to the agent, the artifact-generation environment, and the cost structure of the workflow.

The model matters enormously. But the system around the model increasingly matters just as much.

This exercise mirrors the evaluation process we believe many organizations are undertaking, and many more will undertake, as enterprise AI moves from experimentation to adoption. The decision is not simply which model scores highest. Organizations must decide which models to adopt, which platforms or agent systems will route work among them, and how to manage token costs without compromising output quality. Evaluating models inside a realistic knowledge-work environment is therefore not an academic exercise; it is a practical proxy for how enterprises will select, deploy, and govern AI systems.

My Experience With the GPT-5.6 Series

I have been tasking GPT-5.6 Sol with demanding work for about a month, and it is incredible. It has an exceptional understanding of how to use and control subagents, it is remarkably intelligent, and, most importantly, it keeps working. When you give it a task, it continues until that task is complete instead of stopping at a plausible first draft.

I have only had access to GPT-5.6 Terra and Luna for a few days, but the early results are equally compelling. For knowledge work, Terra is undefeated on the combination of price, speed, and value.

Luna may be the best everyday model, period. It is better than the open-weight models we have tested while being significantly cheaper. On factuality-focused work, it can outperform Claude at a fraction of the price. The tradeoff is presentation: if you need a polished final artifact, you may have to do some additional work to make the output look better.

With Luna and Terra in the mix, frontier-level performance is finally affordable enough that you do not have to worry about the price on most tasks. Most of the daily work we ran through these models cost cents rather than dollars, while still producing usable, factually accurate results. That is not true of most models in the same price range, where lower cost usually comes with a clear drop in reliability. This is the first time token budgeting starts to look like a problem that can be solved by better, cheaper models rather than tighter limits: use capable models like Luna and Terra for the majority of work, and reserve the most expensive systems for the smaller set of tasks that truly need them.

If you are looking for quick factual answers and research, I have not found a better model. When that research needs to become a polished, actionable deliverable, you can feed it to GPT-5.6 Terra or Sol and have the stronger model synthesize and present it. That gives you the best of both worlds at a fraction of the price.

Overall, I would take the GPT-5.6 series over Claude for these workflows. Based on the pricing I have seen, I expect it can deliver equal or better work for roughly one-fifth to one-tenth of the price.

Why We Built the Benchmark

The Diligence Stack Agent Bench exists because we wanted to evaluate models and agents in a practical knowledge-work environment.

The core question is not whether a model can answer a trivia question, summarize a clean document, or solve a toy reasoning problem. The question is whether a model can take a messy, real-world knowledge-work task and produce a useful first draft.

For example:

If I ask an agent to build a financial model with a specific set of variables, source the assumptions from internal research, produce an Excel file, write a PDF report, and explain the confidence level behind the output, can it give me something I can actually use?

That is the bar.

In most knowledge-work environments, AI outputs are not replacing the entire workflow yet. They are usually the first step. A model or agent creates a draft, model, memo, deck, or report that a human reviews, edits, and improves. But that first draft still matters. The better it is, the less human cleanup is required. In some cases, if the draft is strong enough, it may be close to final.

That is what this benchmark tests: how close a model or agent can get to a complete, usable work product when given access to internal data and a computer-like tool environment.

The Two-Pronged Setup

When we started building the agent system that became the foundation of this benchmark, we began with a simple question: what does a knowledge-work agent actually need access to?

For Diligence Stack, the answer was three core data sources:

1. Historical Creative Strategies research and internal notes
2. Professional investment-grade research collected over time
3. Financial models and related structured files

There are a few obvious ways to give an agent access to this kind of data. You can upload files to Box, Google Drive, Dropbox, or another document system and let the connector handle search and retrieval. We tried versions of that. It works. But it is not ideal.

The problem is that most enterprise data was not designed for agents. PDFs have headers, footers, watermarks, irrelevant boilerplate, broken tables, embedded images, and inconsistent formatting. Presentations are even messier. Audio and video are generally worse. A basic connector can search over that content, but it is rarely optimized for high-quality agent retrieval.

So we built a better pipeline.

The Diligence Stack Knowledge Base Pipeline

The knowledge base pipeline is designed to structure unstructured data so agents can use it efficiently.

At a high level, it takes uploaded files, cleans them, splits them where appropriate, removes junk, extracts useful structure, and makes the content easier for agents to search and reason over. That means removing things like watermarks, repeated headers, useless page artifacts, and other noise that makes retrieval worse.

When images appear in files, the system generates detailed descriptions of those images so the agent can reason over charts, diagrams, screenshots, and visual content. The original files remain available, but the agent is no longer forced to treat them as opaque blobs.

The system is designed to handle essentially every major file format, including PDFs, documents, spreadsheets, slides, images, audio, and video.

In practice, that means you could upload a four-hour Zoom recording, 200 PDFs of old emails, a folder of financial models, an audio recording of your dog barking, and a pile of research notes, and the agent should still be able to find and use the relevant information.

Under the hood, the system uses Gemini embedding models from Google for multimodal embedding, Convex for database storage, Gemini 3.1 Flash Lite for data processing, and a pipeline architecture designed by GPT-5.6 Sol to be robust across different downstream models.

The point is not just to make one model perform better. The point is to make the whole agent environment better.

Why This Matters

This matters because it lets us give any agent access to high-quality internal knowledge through MCP or through our own agent environment.

Instead of asking a model to search raw files in a generic document store, we can provide a purpose-built knowledge base that has already been cleaned, structured, embedded, and optimized for agent retrieval.

That changes the economics and performance of the workflow.

In our testing, searching through the Diligence Stack knowledge system instead of using the more generic agent search pattern saved roughly five minutes on simple requests. We do not yet have fully audited cost numbers across every configuration, but the likely savings on large-model requests are meaningful, potentially in the range of several dollars per request depending on the model, search pattern, and context size.

That adds up quickly.

Faster retrieval means lower latency. Better retrieval means fewer wasted tool calls. Cleaner context means lower token usage. Better source selection means stronger outputs. The economics are not subtle: better infrastructure makes agents cheaper, faster, and more useful.

The Diligence Stack Agent

We then built this knowledge system into our own agent.

The Diligence Stack agent has access to its own computer-like environment, tools to create documents, tools to create spreadsheets and financial models, access to the full knowledge bases, and the ability to download and inspect original files. It is designed for longer-horizon work, not just short chat responses.

That distinction matters.

A normal chatbot can answer a question. A useful knowledge-work agent needs to investigate, retrieve, synthesize, model, format, check its work, and produce artifacts. It needs to behave more like a junior analyst with tools than a text box with a memory.

In the Diligence Stack Agent Bench, this setup performed well. In one benchmark run, our internal agent outperformed Claude Cowork running Claude 5 Fable on the same task, while also coming in at a lower price. Creative name for the benchmark? Maybe not. Useful signal? Absolutely.

Why the Harness Matters

One of the clearest lessons from this testing was that the model is only one part of the result. The harness—the tools, retrieval system, context management, execution environment, and revision loop wrapped around the model—can materially change both output quality and cost.

We tested this directly with Claude 5 Fable. To make the comparison as controlled as possible, we gave Cowork access to identical tools through MCP and the exact same skills available to our Diligence Stack harness. In this benchmark run, Claude 5 Fable still produced a better result inside our harness than it did inside Claude Cowork. The Cowork run scored worse and cost more, while our harness used the same model more efficiently and produced a stronger deliverable.

That does not mean Cowork will be worse for every task, or that one run settles the question. It does mean model comparisons need to account for the environment in which the model is operating. A strong model inside an inefficient harness can waste tool calls, consume more tokens, retrieve weaker context, and still produce a worse final output. A well-designed harness can make the same model both cheaper and more useful.

What the Benchmark Measures

The benchmark is designed around practical output quality.

We are not asking, “Which model sounds smartest?” We are asking, “Which model produces the most useful work product when connected to internal data and tools?”

The grading framework evaluates several dimensions:

Prompt and deliverable completion. Did the model actually do what was asked? Did it produce the requested files, sections, analysis, and supporting materials?

Factual accuracy and verification. Are the claims correct? Are the numbers internally consistent? Does the model avoid hallucinating facts, companies, timelines, or financial assumptions?

Source grounding and priority. Did the model use the right sources? Did it prioritize the Diligence Stack knowledge base and internal materials before reaching for weaker external context? Did it cite and trace its assumptions appropriately?

Analysis and actionability. Did the output merely summarize information, or did it synthesize it into something useful? Does the report help a decision-maker understand what matters, what is uncertain, and what to do next?

Artifact quality and auditability. Are the PDF, Excel file, model, or other deliverables usable? Are they readable, well-structured, and traceable? Can a human inspect the assumptions and understand how the output was created?

Tool strategy and robustness. Did the agent use tools intelligently? Did it search efficiently, inspect the right files, recover from missing information, and avoid wasting time on low-value steps?

Cost and token efficiency. How expensive was the run? How much did the model spend to reach the result? Cost is tracked separately from quality because the cheapest bad answer is still a bad answer, but price matters once output quality clears a usable threshold.

In the current scoring framework, the final quality score is weighted most heavily toward completion, factual accuracy, source grounding, and analytical usefulness. Design, auditability, tool behavior, and cost efficiency also matter, but the benchmark intentionally does not let a cheap model win if the output is not useful.

That is the right tradeoff. In real knowledge work, the highest-cost outcome is not paying a few extra dollars for a model. The highest-cost outcome is a polished but wrong deliverable that wastes a senior person’s time.

How We Think About Pricing

Pricing is part of the benchmark, but it is intentionally a smaller part of the weighted score.

The reason is simple: output quality matters most. A cheap run that produces a weak, incomplete, or inaccurate deliverable is not actually cheap. It just moves the cost downstream to the human who has to fix it. In knowledge work, the expensive failure mode is not spending a few more dollars on a model. The expensive failure mode is trusting a polished but wrong output, or getting a draft that still requires hours of senior cleanup.

So we treat price as an important constraint, not the primary objective.

The goal of the benchmark is to help users understand which models produce the best outputs, and then decide which of those outputs are worth the price for their specific workflow. For some tasks, the highest-scoring model may be worth paying for because it gets meaningfully closer to a usable final deliverable. For other tasks, a slightly lower-scoring model may be the better choice if it delivers 90% of the quality at a much lower cost.

That is why we look at both absolute quality and cost-adjusted performance. The right answer is not always “use the cheapest model” or “use the highest-scoring model.” The right answer is usually: find the best output quality available at the price you are willing to pay.

This is especially important for agentic workflows because costs can compound quickly. Long-running agents search, retrieve, reason, inspect files, generate artifacts, and revise outputs. A model that is slightly more expensive per token can still be the better economic choice if it uses tools more efficiently and produces a cleaner first draft. Likewise, a cheaper model can be attractive if the task is lower stakes, more repetitive, or easier for a human to review.

In other words, price matters. But price only matters after the output clears the bar of being useful.

Why We Didn’t Include Open Models

We also tested open models extensively, but did not include most of them in the current benchmark results.

The reason is straightforward: almost all of the open models we tested produced materially worse outputs while also costing more to run in this agent environment. That is a difficult combination to recommend. Lower quality can be acceptable when it comes with a meaningful cost advantage, but lower quality at a higher price does not offer users a useful tradeoff.

DeepSeek V4 Pro was the one clear outlier. It was far cheaper than the other open models we tested, although the quality tradeoff is visible in its benchmark score. For lower-stakes, high-volume, or highly reviewable tasks, that price difference may still make it interesting.

This is not a permanent exclusion. We will keep testing open models and add more of them as their performance and economics improve. For this version of the benchmark, however, including a longer list of models that were both weaker and more expensive would not have helped readers make better decisions.

Where GPT-5.6 Fits

The GPT-5.6 series performed strongly in this environment because the task rewarded more than raw fluency. It rewarded long-context handling, source discipline, synthesis, spreadsheet construction, report writing, and the ability to produce usable artifacts in a single run.

GPT-5.6 Sol and GPT-5.6 Terra were the strongest quality leaders in our testing, with GPT-5.6 Luna standing out as a strong value option. The key takeaway is not simply that one model “won.” It is that stronger models produced more complete first drafts, handled ambiguity better, and were more effective at turning messy internal data into structured outputs.

That is exactly what matters for agentic knowledge work.

There is also a broader point here: as models improve, the benchmark ceiling rises. Tasks that previously required a human analyst to spend hours gathering sources, cleaning up assumptions, creating tables, and writing the first report draft can now be pushed much further in a single agent run. The output is still not perfect. But the baseline is moving fast.

What We Learned

The biggest lesson from the benchmark is that model quality and system design compound.

A better model inside a weak retrieval environment can still waste time and miss key information. A strong knowledge base paired with a weaker model can produce decent results, but may still fall short on synthesis, judgment, and artifact quality. The best results come from pairing strong models with purpose-built data infrastructure and tools.

That is the direction we think enterprise AI is heading.

The winning setup will not be a generic chatbot pointed at a file dump. It will be a structured knowledge system, exposed through MCP and internal tools, connected to high-quality models, and evaluated against real work products.

That is the premise behind Diligence Stack Agent Bench.

We are not trying to measure whether a model can perform well in an artificial sandbox. We are trying to measure whether it can do useful work in the kind of environment businesses are actually building.

And on that front, GPT-5.6 looks like a meaningful step forward.

The practical readout is simple: for knowledge-work agents, the gap between “interesting demo” and “usable first draft” is closing. The models are getting better, but the infrastructure around them is what turns that improvement into something businesses can actually use.

Join the newsletter and stay up to date