Sarmadi AI Digest October 12, 2026 Updated 6:40 AM CT Today Archive Topics Saved Subscribe RSS

Claude Code wins 6 golds in an adversarial game benchmark; inference-efficiency papers pile up

Research today clusters around two practical themes rather than new capability claims. One group of papers chips away at inference cost: faster speculative decoding, parameter-free teacher-student distillation, and a finding that offline student rollouts often beat on-policy distillation outright. A second group asks whether agents can turn raw experience, demonstrations, or competitive play into reusable skill, with Opus5.5 running on Claude Code taking six gold medals in a long-running adversarial game benchmark while still losing most human ladders. A third thread is methodological hygiene: two papers show biological and spatial reasoning models lean on their language priors more than on the structured inputs they are supposed to use, and a third introduces a benchmark for data-video understanding that most multimodal models still fail. On the industry side, Karpathy's comment about a widening gap in shared understanding of LLM capability drew outsized attention, while QGIS's ban on AI-assisted contributions and Silicon Valley's uptake of the White House's "superintelligence" rebrand point to diverging reactions to AI inside institutions.

20 papers 6 news 4 sources ← Latest

News

6 items

Institutions reacting differently to AI, from bans to rebrands

QGIS, the open-source geospatial project, banned AI-assisted code contributions over provenance and review-burden concerns. Separately, a Karpathy post on the widening gap in public understanding of current LLM capability drew outsized attention (492 points), while coverage shows parts of Silicon Valley adopting the White House's push to rebrand AI as "superintelligence." Smaller stories cover data-center siting NDA friction and skepticism that voice AI has reached its breakout moment.

News Hacker News

QGIS bans AI assisted contributions

The QGIS open-source geospatial project adopted an enhancement proposal banning AI-assisted code contributions.

Why it matters
  • A widely-used open-source project formally restricting AI-generated code signals growing institutional friction over provenance and review burden.

Papers

17 items

Squeezing more throughput and transfer out of fixed model weights

Six papers cut inference and training cost without changing base weights. DCOCO restores token dependencies in parallel diffusion drafters for up to 6.6x speedups; PreSTO and a layer-selection method speed speculative decoding further. Universal Textual Teaching distills teacher-student gaps into a reusable text "Primer" with no parameter updates, a 17-pair study finds offline-rollout distillation often beats on-policy distillation, and SynthSelf brings self-distillation to small models.

Paper Hugging Face

Contextual Correction of Parallel Drafts for Speculative Decoding

DCOCO restores intra-block token dependencies for parallel diffusion drafters by scoring candidate tokens' contextual compatibility, reaching up to 6.6x speedups over vanilla decoding.

Max speedup vs vanilla decoding 6.6xGain over best competitor under load 6-23%
Why it matters
  • Highest average throughput across four verifier backbones and ten benchmarks, including under extended reasoning.
  • Outperforms the strongest competing corrector by 6-23% under concurrent serving loads of 4-32 requests.
Paper Hugging Face

Universal Textual Teaching for LLMs

UTT distills teacher-student knowledge gaps into a reusable natural-language "Primer" with no parameter updates, raising a student's KernelBench accuracy from 9.4% to 48.6%.

KernelBench accuracy gain 9.4% to 48.6%Math reasoning accuracy gain 27.6% to 51.7%
Why it matters
  • Works for API-only or costly-to-train teacher models since it requires no weight access on either side.
  • A Primer synthesized for one teacher-student pair generalizes to other students that never participated in its creation.
Paper Hugging Face

When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better

Across 17 teacher-student pairs from 1.5B to 235B parameters, distilling from offline student rollouts (Semi-OPD) beats on-policy distillation in 14 of 17 cases.

Pairs where Semi-OPD wins 14 of 17Max training speedup 11.4x
Why it matters
  • Up to +13.6% accuracy and 11.4x training speedup over on-policy distillation when teacher and student are misaligned.
  • On-policy distillation only wins when teacher and student output-token overlap is already high, giving a concrete criterion for choosing a method.
Paper Hugging Face

PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling

PreSTO prefetches batches of future Metropolis-Hastings proposals for power sampling, cutting model calls roughly 3.9x while leaving the sampler's transition kernel unchanged.

Median speedup range 1.71x-2.33xFewer model calls 3.9x
Why it matters
  • Median speedups of 1.71x-2.33x across PowerMH and its EntropyCut and MultiTryMH follow-ups on seven datasets and five base LLMs.

Agents that convert competitive play, demonstrations, and GUI use into reusable skill

Papers formalize how agents turn experience into reusable competence. AAArena has agents revise game policies from replays; Opus5.5 on Claude Code wins six golds but tops no remaining human ladder, weaker as rule complexity rises. Work2Skill induces reusable GUI skills from human desktop demos. Encore reads a few robot demos as evidence, not training data, to write manipulation code. SoftwareGym synthesizes 182 runnable environments for computer-use agent training.

Paper Hugging Face

Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition

AAArena benchmarks AI agents revising game-playing policies from replays across 12 adversarial games against 1,920 archived human programs; Opus5.5 with Claude Code wins 6 gold medals but no configuration tops the remaining 6 human ladders.

Gold medals won 6 of 12 gamesHuman ladders topped 0 of 6 remaining
Why it matters
  • Shows agentic policy revision from replays can beat archived human competitors in some games, under real competition-style budgets.
  • Performance is weaker in games with more complex rule specifications, pointing to rule-understanding as the current bottleneck rather than search or learning.
Paper Hugging Face

Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction

Work2Skill records screen, mouse/keyboard, and accessibility-tree data from 350 human desktop demonstrations, then induces reusable skills that separate workflow rules from task-specific detail.

Why it matters
  • Mean score improvements of 4.67-10.54 points on MyPCBench Bounded Action and 3.11-7.50 points on a 286-task OSWorld-Verified subset over a no-skill baseline.

Evaluation work finds models relying on priors instead of the evidence they're given

Two papers find reasoning models score well without actually using inputs built for them. Perturbation tests on six biological reasoning models find leading genomic/protein foundation models contribute little to accuracy, with the LLM following text over biological evidence in over 97% of conflicts. DataVista, the first data-video-understanding benchmark, finds the best model (Gemini-3.1-Pro) reaches only 70% accuracy, weakest on causal reasoning and narrative structure, well below humans.

Paper Hugging Face

When Do Biological Reasoning Models Use Their Biological Inputs?

Testing six biological reasoning models with input perturbations and evidence conflicts finds Evo2 and ESM3 contribute little to BioReason's final accuracy, which instead follows the text input in 97.9-99.7% of conflicts.

Conflicts where text wins over DNA evidence 97.9%Conflicts where text wins over protein evidence 99.7%
Why it matters
  • Accuracy gains across SFT and RL checkpoints do not imply growing reliance on the biological foundation-model inputs the systems were built around.
  • Reasoning traces misstate nucleotide changes and describe functions omitted from the final prediction, showing post-training does not guarantee faithful use of specialized inputs.
Paper Hugging Face

DataVista: Diagnosing Multimodal LLMs on Data Video Understanding

DataVista, the first benchmark for data-video understanding, tests 19 MLLMs on 961 real-world data videos and 6,775 questions; the best model reaches only 70.0% accuracy, weakest on causal reasoning and narrative structure.

Best model accuracy 70.0%Questions in benchmark 6,775
Why it matters
  • More frames and subtitles mainly help data perception and temporal reasoning, with little gain on narrative understanding, isolating where current models actually fall short.

Robotics and multimodal models learning more from less raw data

Four papers push embodied/spatial learning toward structure over scale. Dex-One2Many abstracts one demo video into scene graphs guiding RL for manipulation, widening its lead to 71% unseen. TouchScale's 500-hour visual-tactile dataset lifts manipulation success from 22.5% to 57.5%. SAVER trains sparse-frame video reasoning using dense views only as training-time references. S-Space finds a linear subspace encoding spatial coordinates, exposing viewpoint transformation as the key bottleneck.

Also today