LucentBench.

What LucentBench tests

LucentBench is a benchmark for one hard question: can a language model read a company’s latest quarterly filing and forecast what happens next? It was designed by Lucent, a Levy Street venture building an AI investment research platform, and published in the Lucent whitepaper (version 1.0, August 2025). The whitepaper calls it “the first forward-looking financial prediction benchmark based on quarterly earnings forecasting”. That is Lucent’s own claim; we have not audited it against every financial benchmark in existence.

The structure, as the whitepaper sets it out:

Input

“Previous quarter’s 10-Q report (complete document)”.

Task

“Forecast next quarter’s earnings, revenue, EBITDA, and stock price”.

Outputs

Written equity analysis, point estimates for key metrics, probabilistic ranges (lower quartile, mean, upper quartile) and an investment recommendation.

Sample

“100 representative stocks across NYSE and NASDAQ”, across technology, healthcare, financial, consumer and industrial sectors, with market capitalisations from $500M to $500B+.

Period

“Q1-Q2 2025 (ensuring no model training contamination)”.

Written analysis scoring

A judge LLM applies a six-part rubric (fundamental analysis 25%, analytical rigour 20%, forward-looking insights 20%, risk assessment 15%, technical competence 10%, communication 10%) on a 1 to 5 scale.

Feeding a full 10-Q also makes this a long-document test, which is why we treat it alongside our work on evaluating long-context models.

Why forward-looking evaluation avoids contamination

Contamination is the main reason AI benchmarks overstate ability. If a test’s answers appeared in a model’s training data, a high score may reflect memory, not reasoning. Finance is especially exposed: past earnings, price moves and analyst commentary are all over the public web.

A forward-looking benchmark sidesteps this by scoring forecasts of a period that post-dates the models’ training data. The model cannot have seen the answer because, when its training data was collected, the answer did not exist. The whitepaper places LucentBench in the same family as other live evaluations it reviews, noting “LiveBench’s commitment to contamination-free evaluation through regularly updated questions” and Prophet Arena, which “measures real-world forecasting capability through live prediction markets”.

The protection has two conditions that the whitepaper does not report on. Every model’s training data must end before the forecast period, and the model must not be able to search for the actual results during the test. Anyone building a similar benchmark should record both.

The results, as the whitepaper reports them

The tables below reproduce the whitepaper’s figures exactly, including the rows it chose to show. Ranks are the whitepaper’s; gaps in the rank column mean the whitepaper did not print those models.

Table 2: Fundamentals forecasting accuracy (revenue and EBITDA)

Table 3: Stock price prediction accuracy

The whitepaper’s reading: “The tight clustering of top performers suggests frontier models have achieved near-parity in price forecasting capabilities.”

Table 4: Written analysis quality (judge LLM, 1 to 5)

Table 5: Range estimation accuracy (stock price and revenue in range)

The pattern matters more than any row. No model led every task. Claude Opus 4.1 ranked third on fundamentals but twelfth on written quality; GPT-5 led written quality but ranked eleventh on ranges.

How the results shaped Lucent’s model choice

Lucent did not pick the top scorer. It also measured how correlated the models’ predictions were, because an ensemble of models that agree adds little. The whitepaper reports GPT-5 and Claude Opus 4.1 as a “Highly Independent” pair at a correlation of 0.007, and identifies an “Anthropic Family” cluster with average internal correlation of 0.516.

The resulting consortium was GPT-5 (“Highest written quality (4.88/5)”), Claude Opus 4.1 (“Excellent fundamentals (90.13%), highly unique perspective”), Grok 4 (“Solid overall performance, moderate independence”) and Gemini 2.5 Pro (“Strong written analysis (4.57/5), best range estimation for risk assessment”), with an average pairwise correlation of 0.282. These are mid-2025 models; any re-run today would involve newer ones.

The general lesson for AI model selection: choose models for complementary errors, not just headline scores.

What is not published, and what that means

Read LucentBench as a documented internal evaluation, not a public benchmark.

• No public dataset, code, leaderboard or per-stock results. Searches on 19 October 2026 found none. The results cannot be reproduced outside Lucent.
• Scoring formulas are not defined. The whitepaper names “Diff Score”, “Stock Return Score” and range scores but does not say how they are calculated, so read them as relative rankings. In Table 2 the combined figure equals the sum of the two component columns.
• The judge model is not named for the written-analysis rubric.
• Total models tested is not stated. Printed ranks run to at least 18.
• One six-month window. Results from Q1 and Q2 2025 reflect one market period.
• Tool access is not described. The whitepaper does not say whether models could search the web during the test.

The whitepaper itself cautions that benchmarks “cannot fully capture the nuanced requirements of investment analysis”. We agree, and the gaps above are the first things we would ask any vendor to close.

Lessons for evaluating AI in financial services

LucentBench’s design carries over to any team testing AI for research or advice work, including in New Zealand, where regulators are asking sharper questions. On 15 October 2026 the Reserve Bank and the FMA said that “Boards and senior management need to know where AI is being used in their business, understand the risks it brings, and be clear about who is accountable for managing them.” The FMA’s exploratory review of AI in financial advice opened on 6 August 2026.

• Score forecasts against outcomes that occurred after the models’ training data ends
• Disable or log web access so recall cannot pass for reasoning
• Measure ranges and calibration separately from point accuracy
• Judge written reasoning with a published rubric, and name the judge model
• Check correlation between models before combining them
• Publish enough of the method for someone else to repeat it

Range estimation is where calibration matters most; see our research on temporal probability calibration. For the wider method, read our guide to AI agent evaluations, the sector context in AI for financial services, or talk to us about LLM evaluation.

Written by the Levy Street team in Wellington. Last updated 19 October 2026.