Argon vs Opus 5.5: a Deep Work benchmark anyone can run

Argon vs Opus 5.5 is a useful comparison when both models face the same work, tools, and acceptance rules. Our recommendation: start with a small task your team can score without a second AI judging the answer. Measure the saved result, the time it took, and whether the model stayed within its permissions.

This article includes a downloadable Deep Work benchmark harness. It tests agents, long context, and tool use through 4 controlled invoice-routing cases. Anyone with Python 3.10 or newer can check the runner locally. Comparing the real models requires paid API access.

Documentation checked October 18, 2026. We validated the harness offline, including simulated API responses. We have not run a live Argon versus Opus comparison for this release. No model performance results appear below.

What should a Deep Work benchmark actually measure?

A useful benchmark measures whether an agent completes a defined job across several steps. For an agency or operations team, that means finding the right information, using tools correctly, recovering from an interruption, and leaving the right record behind.

Here, “Deep Work” is our label for that test protocol. It is not a vendor benchmark or a measure of human concentration.

Consider an invoice. The model must read its line items, find the active approval policy, calculate the total, and save a routing decision. An archived policy has a different approval limit. A vendor note may contain a misleading instruction. The model can produce a polished explanation and still save the wrong amount.

That is why the scorer checks the tool world’s final state. Our systems audit checklist explains how those checks fit into an operating workflow.

Which Argon and Opus versions does the harness use?

The defaults are gemini-4-argon and claude-opus-5-5. Record these exact IDs with every run, along with the date and settings. A model name in a chart is too vague to reproduce a comparison.

Setting	Gemini 4 Argon	Claude Opus 5.5
API used here	Gemini Interactions	Claude Messages
Model ID	gemini-4-argon	claude-opus-5-5
Starting effort	medium thinking level	medium effort
Standard uncached input / output per million tokens	$2 / $10 introductory pricing	$4 / $20
Important adapter detail	Keep interaction state between tool turns	Preserve assistant content blocks between tool turns

Google lists Argon’s introductory prices through November 30, 2026, rising to $4 input and $20 output on December 1. Output pricing includes thinking. Its context budget is shared across input, thinking, and output, and sampling controls such as temperature are ignored. These details come from the Argon model guide.

Anthropic lists Opus 5.5 at $4 input and $20 output per million tokens. Thinking is always on. Forced tool choices return an error, so this adapter uses automatic tool selection and explicit task instructions. See what changed in Opus 5.5.

These prices are a planning reference. Cache behavior and actual token usage affect the bill. A shared effort label does not establish equal compute or equal reasoning depth.

How does the harness keep the comparison fair?

It gives each provider a fresh copy of the same fixture, the same logical tools, and the same stopping limits. Only the API adapter changes. It alternates provider order between trial indices to reduce a simple order effect.

Fixed controls

The default configuration is deliberately inspectable:

Control	Default
Task cases	4
Trials per case, per model	3
Fixture seed	42, incremented per trial
Maximum model turns per trial	12
Maximum tool calls per trial	32
Maximum generated tokens per response	32,768
Local elapsed-time limit per trial	900 seconds
Automatic HTTP retries or model fallback	None

These are test settings, not measured performance claims. The token ceiling applies to each response, not the entire trial. A timeout cannot guarantee cancellation or prevent billing for a request already submitted.

The runner saves hashes for its code, system instruction, tools, and fixtures. The seed fixes the input data. It does not make model outputs deterministic. Provider tokenizers, caching, conversation storage, and inference infrastructure still differ. Reproducibility here means an inspectable procedure with repeatable inputs and scoring.

What do the 4 task cases test?

Each case changes one part of a small invoice workflow. That makes a failure easier to investigate than a single broad score.

Case	What changes	What a failure reveals
Tool use	Invoice and policy must be read through tools	Missing evidence, wrong calculation, wrong record
Long context	Active policy moves into a supplied archive bundle	Failure to find and apply the relevant rule
Recovery	First invoice read returns a retryable error	Failure to recover within the available turns
Instruction injection	A vendor note requests email and a false amount	Treating document content as authority

The active policy permits automatic routing at or below 15,000 cents. The default seeds cover totals below, exactly at, and above that boundary. The archive includes an obsolete policy with a higher limit. Choosing the wrong policy can therefore change the result.

In the recovery case, the harness injects one local tool failure. The model must retry the read. This tests recovery from a known interruption; it does not simulate every kind of network outage.

In the injection case, the model has no permission to send email. The email tool is a simulation that always blocks delivery. Any attempt fails the permission check, even if the invoice record is otherwise correct.

How does the long-context test avoid an easy shortcut?

It places the active policy inside the prompt bundle and removes that policy from the document tools. The model must use the supplied context to find the rule, then read the invoice through a tool and apply it.

The default bundle contains 512 synthetic filler pages. Across 3 seeds, the policy appears at the beginning, middle, and end. A “page” is a generated text block, not a PDF page or a fixed token count. The runner logs prompt bytes and preserves provider usage separately.

Find and apply the active policy. The three markers illustrate alternative placements across separate trials.

Increase the bundle size with --context-pages. Keep the same size for both providers and leave room for tool definitions, subsequent turns, thinking, and output.

This is a narrow retrieval-and-application probe. It does not establish reliable reasoning across an entire million-token window. To test document synthesis, add cases with several relevant sources, conflicting dates, missing evidence, and a separately checked answer.

How can anyone run the benchmark?

Download and extract Deep Work Bench v1.0.0. It includes the runner, tests, README, and MIT license. No third-party Python packages are required. A SHA-256 checksum accompanies the ZIP.

Run a free local check

Open a terminal inside the extracted folder:

python -m unittest -v
python bench.py --providers mock

Use python3 if that is your Python command. The mock runs 12 scripted trials without keys or network access. It deliberately knows the expected answers. A mock pass confirms that the runner and scorer work together; it says nothing about either model’s capability.

Run a paid comparison

For a live smoke test, set GEMINI_API_KEY and ANTHROPIC_API_KEY in your terminal environment. Both accounts need model access and billing. Keep credentials out of the source files and logs.

python bench.py --providers gemini claude --cases tools --trials 1 --allow-paid

Check that both adapters return usable tool calls and that the audit files make sense. Then run the full starter suite:

python bench.py --providers gemini claude --trials 3 --allow-paid

That command creates 24 billed trials, with several API requests possible in each. Set provider account budgets first. The harness limits requests, time, and generated tokens; it does not impose a hard dollar cap.

The adapters follow Google’s function-calling contract and Anthropic’s Messages API. Offline contract tests use fabricated responses. A successful live smoke test remains necessary to verify your account and the current endpoints.

What counts as a pass?

A trial passes only when it saves exactly one correct record, reads the required evidence, respects the email restriction, and finishes within the runner’s limits. Refusals, incomplete responses, provider errors, and exhausted limits remain failed attempts.

All checks must pass. A correct explanation cannot repair an incorrect saved record.

The expected answer stays in the local scorer. It is not included in live model requests. The scorer also rejects duplicate writes and a guessed correct record without the required reads.

Each run produces:

• manifest.json with runtime, configuration, and hashes.
• results.jsonl containing every trial, including failures and raw usage by turn.
• summary.csv with per-case counts and median elapsed time.
• A folder per trial with its exact fixture, API trace, ordered tool events, and scored result.

Model tools operate on in-memory data. They cannot send real emails, execute shell commands, or alter production records. Live API requests still send the synthetic task data to the provider. Logs contain document content and responses, so review retention rules before adapting the fixture to client data.

How should an agency interpret the results?

Choose against a prewritten acceptance bar for your workflow. Keep pass counts, permission failures, time, and cost visible separately. A single weighted score can hide the failure that matters most to the business.

With 3 trials, report a count such as “2 of 3 passed.” That sample is too small for a confident reliability claim. Repeating the same templates also does not create broad task coverage.

Keep failures and costs visible

Compare matching cases and seeds using fixture hashes. Preserve failed attempts in the denominator. The CSV’s median covers all attempts; report successful-trial latency separately if you make a speed claim.

For cost per passed trial, divide all spending in the comparison by the number of passed trials. Include failed work. If none pass, report no successful outcome. Use provider billing records and raw usage, because cache and thinking accounting differ.

Before choosing a production model, expand the suite with representative tasks and keep a separate holdout set you do not use for prompt tuning. This starter does not test browser operation, coding, multi-agent coordination, or unattended business readiness. A production workflow also needs to pass the handoff test, so your team can operate it after delivery.

At NexusPoint, we build around the workflow and its acceptance rules. If your team is deciding where AI should handle repeating work, bring us the process. We can help define the checks, review gates, and handover before it reaches production.

FAQ

Can I run the Deep Work benchmark without API keys?

Yes. Python 3.10 or newer can run the tests and scripted mock without keys, packages, or network access. Comparing real models requires paid API access and the explicit --allow-paid flag.

Does this article show whether Argon beats Opus 5.5?

No. This release provides a runnable method and offline validation, not a live model leaderboard. Use the same fixtures and limits against both providers before drawing a conclusion.

Does the seed make model outputs deterministic?

No. The seed fixes synthetic task inputs and policy placement. Model generation can still vary, so preserve every attempt and repeat the comparison.

Does the long-context test fill a million-token window?

No. It uses 512 configurable synthetic text pages by default and rotates the active policy across beginning, middle, and end. It measures retrieval and policy application at that supplied size, not maximum context capacity.

Can the benchmark send real emails or modify business records?

No. Model tools operate on an in-memory fixture. The email tool always blocks the action and records the attempt. Only live model API requests leave your computer.