• HuggingFace Blog (rss) · 6 gundur sessiz

A new study called HarnessTax shows that swapping coding-agent harnesses like Claude Code, Codex CLI, or Pi can double or quintuple inference costs for nearly identical success rates, meaning the harness—not just the model—is often the real cost driver.

• HarnessTax tested 21 model-harness combinations (7 models x 3 harnesses: Claude Code, Codex CLI, Pi) across 60 tasks from SWE-bench Lite and Terminal-Bench 2.0
• On SWE-bench Lite, a Claude model solved 97.8% of tasks with Claude Code at $1.33/attempt vs 96.7% with Pi at $0.67/attempt — nearly 2x cost for a 1.1-point accuracy gain
• Pi's minimal 4-tool harness (read, write, edit, bash) hit the cost-success Pareto frontier on both benchmarks, beating feature-rich alternatives on efficiency
• Vendor-default pairing is not always optimal: across Anthropic and OpenAI models, an alternative harness beat the vendor's own harness in 9 of 12 comparisons
• On Terminal-Bench 2.0, an OpenAI model scored 83.3% with Pi at $0.42/attempt vs 78.9% with Codex CLI at $0.76/attempt
• Co-author Melissa Pan argues developers should treat 'model x harness x workload' as the unit of evaluation and track cost-per-successful-task, not just aggregate success rate

llm-keys-ui 0.1

• <p><strong>Release:</strong> <a href="https://github.com/simonw/llm-keys-ui/releases/tag/0.1">llm-keys-ui 0.1</a></p> <p>This plugin solves a very specific problem.</p> <p>I've started using <a href="https://learn.chatgpt.com/docs/remote">Codex Remote</a> to run coding agents on various machines whi

Rokt's CTO says AI has collapsed the engineering career ladder, prizing judgment and strategy over coding execution.

• Rokt CTO Sam Dozor says engineers who aren't orchestrating teams of AI agents 'can't keep up with the pace of development.'
• Rokt merged PM, engineer, and designer roles into a single 'builder' title, flattening the traditional career ladder.
• Dozor cites a 'J-curve of productivity': teams see temporary output drops while learning AI workflows, then sharp gains afterward.
• Company is now training builders directly on systems design and product strategy instead of manual coding/debugging skills.
• Piece is sponsored content from Rokt (ecommerce tech co.), framed as thought leadership rather than a product announcement.