A papers-only day: agent benchmarks find fresh cracks, one recsys paper is already live in production

August 5 was a papers-only day: Hugging Face's daily feed produced 36 arXiv submissions and no qualifying news, consistent with the backfill process used for this date after press feeds could not be fetched retroactively. Benchmarks Keep Finding Agent Reasoning Is Shakier Than It Looks is the strongest thread — MerchantBench shows the best LLM merchant agent reaches only 27.3% of human net assets over a simulated year, and FinIndices shows financial-statement reasoning collapses once formula hints are removed. New Recipes for Teaching Models from Their Own Mistakes groups five self-distillation and credit-assignment papers (PCSD, TurnSight, ReflectRL, DASH, Any-OPD), all extracting denser training signal from agent rollouts rather than discarding failed trajectories. World Models and Video Agents Push Toward Real-Time, Long-Horizon Use covers MiniWorld, ST-WAM, and Video-DeepResearch, the last reporting it beats Claude-4.5-Sonnet and GPT-5 on a new video benchmark — treat that as self-reported until independently reproduced. Efficiency work (LLaDA MoE v2, OmniPack, RestoreKV) keeps cutting inference cost without giving up capability. The most concrete result sits in Applied and Niche Tools: Knowledge-Geometry Decoupling is already live on Shopee, lifting homepage-search GMV per user 1.75% in a production A/B test.

36 papers 0 news 1 sources ← Latest

Papers

29 items

Benchmarks keep finding agent reasoning is shakier than it looks

Six new benchmarks probe agent reliability under long-horizon, financial, causal, and privacy-sensitive conditions, and each finds a gap between claimed and actual capability. MerchantBench and FinIndices are the sharpest: agents that look competent on bounded tasks fall apart once evaluated over a simulated year of decisions or a real uncropped financial statement.

Paper Hugging Face

MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operations

MerchantBench simulates a 365-day e-commerce merchant role and finds the best of eight tested LLMs reaches only 27.3% of human net assets.

Best LLM vs human net assets 27.3%Simulated horizon 365 daysProduct records 98,843

Why it matters

Long-horizon coherence, not single-turn accuracy, is the harder bar and current agents fail it badly.
48 runs across two agent frameworks show the gap is consistent, not a fluke of one setup.

Source → Arc

Paper Hugging Face

Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements

FinIndices shows LLM financial-statement reasoning collapses without explicit formula hints, e.g. Gemini-3.1-Pro drops from 70.70% to 38.22% on table tasks.

Gemini-3.1-Pro w/o hints 38.22% (from 70.70%)SFT gain (table) +3.82%

Why it matters

Models that memorize formulas during pretraining still fail to apply them reliably under real statement complexity.
Multi-metric table generation drains reasoning capacity, causing regressions to shallow heuristics even on tasks the model can solve in isolation.

evaluation benchmarks reasoning

Source → Arc

Paper Hugging Face

ExplainBench: Evaluating Code Explanations from Agents

ExplainBench shows coding-agent explanations often claim a patch is correct when it isn't, and ranks agents differently than SWE-bench Verified.

agents code evaluation benchmarks

Source → Arc

Paper Hugging Face

ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?

ContinualSkillBench finds in-context adaptation, not reusable skill libraries, drives most of the gains agents show across sequential tasks.

agents evaluation benchmarks memory

Source → Arc

Paper Hugging Face

When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills

AntiSkillBench finds persona-skill distillation leaks private attributes and enables impersonation, with existing defenses generalizing poorly across strategies.

agents safety evaluation benchmarks

Source → Arc

Paper Hugging Face

When Many Answers Are Valid, Voting Fails: Symbolic Verification for Best-of-K Causal Reasoning in LLMs

CALVER, a symbolic causal verifier, beats self-consistency voting and LLM judges on best-of-K causal reasoning, reaching 42.1% vs roughly 30% on ambiguous queries.

reasoning evaluation benchmarks

Source → Arc

World models and video agents push toward real-time, long-horizon use

Six papers move video and world modeling from offline generation toward streaming, real-time, and agent-usable feedback. Video-DeepResearch's headline result — beating Claude-4.5-Sonnet and GPT-5 on a new video QA benchmark — is the most attention-grabbing but also the least independently verified.

Paper Hugging Face

Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent

Video-DeepResearch's 35B-A3B agent hits 64.0% on a new multi-hop video QA benchmark, ahead of Claude-4.5-Sonnet (59.0%) and GPT-5 (52.5%).

Video-DR-35B-A3B 64.0%Claude-4.5-Sonnet 59.0%GPT-5 52.5%Gemini 2.5 Pro 57.5%

Why it matters

Forces cross-frame visual grounding before web search, addressing agents' tendency to bypass vision tools for text search.
Outperforming frontier proprietary models on a fresh, self-authored benchmark is a notable claim, though it needs independent reproduction before treating the ranking as settled.

agents video-generation multimodal benchmarks

Source → Arc

Paper Hugging Face

ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts

ST-WAM pairs DINOv3 semantic features with VAE dynamics to fix a 'training-distribution hallucination' failure mode, more than doubling real-world manipulation success under visual shift, from 25.8% to 61.5%.

Real-world success under shift 25.8% to 61.5%LIBERO-Plus zero-shot gain +21.3pp

Why it matters

Identifies a concrete failure mode — world models hallucinating training-domain content instead of the current scene — that generalizes across setups.
The gain shows up in real-world robot trials, not just simulation benchmarks.

robotics world-models multimodal

Source → Arc

Paper Hugging Face

MiniWorld: Democratizing the Training of Video World Models from Scratch

MiniWorld is an open, reproducible framework for training streaming video world models from scratch on a single 8-GPU server.

world-models video-generation open-weights

Source → Arc

Paper Hugging Face

JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion

JoyAI-Video-Edit is a 16B autoregressive diffusion model for real-time, open-ended video editing, hitting 720p at roughly 30 FPS on one B200 GPU.

Parameters 16BResolution / speed 720p @ ~30 FPS

video-generation diffusion

Source → Arc

Paper Hugging Face

Quo Vadis, World Modeling?

A survey reframes world models around agent-usable feedback — execution, memory, verification — rather than raw physical-state prediction.

world-models agents

Source → Arc

Paper Hugging Face

GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience

GROVE grows a training-free, stratified memory from streaming video for both reactive QA and proactive assistance.

memory video-generation multimodal

Source → Arc

New recipes for teaching models from their own mistakes

Five papers on self-distillation and RL credit assignment share a pattern: instead of discarding failed or noisy agent rollouts, each extracts denser training signal from them, whether through hindsight, reflection, or cross-family distillation.

Paper Hugging Face

PCSD: Persistent Consistency for Self-Distillation in Agentic Reinforcement Learning

PCSD derives token-level distillation weights from persistent teacher-favoring signal, beating GRPO by 13-16 points on ALFWorld.

reinforcement-learning distillation agents

Source → Arc

Paper Hugging Face

TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning

TurnSight uses turn-level hindsight self-distillation for tool-integrated reasoning, improving credit assignment over trajectory-level RL.

reinforcement-learning distillation tool-use agents

Source → Arc

Paper Hugging Face

ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning

ReflectRL treats failed expert trajectories as material for reflection rather than discarding them, improving reasoning across 9 benchmarks and 4 backbones.

reinforcement-learning reasoning distillation

Source → Arc

Paper Hugging Face

Know When to Stop: Segment-Level Credit Assignment for Reducing Overthinking

DASH assigns segment-level credit to curb unproductive overthinking, edging out GRPO and Dr.GRPO on competition math while cutting wasted reflection.

DASH accuracy 59.45%GRPO accuracy 56.95%

reasoning reinforcement-learning math

Source → Arc

Paper Hugging Face

Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging

Any-OPD distills across mismatched model families by comparing outputs in a shared frozen vision space, lifting a 2.5B student's PickScore to near-teacher quality distilled from a 12B model.

Student size 2.5B (SD3.5-Medium)Teacher size 12B (FLUX.1-dev)PickScore 0.846 to 0.884

distillation image-generation diffusion

Source → Arc

Efficiency work: diffusion LMs scale up, inference costs come down

Six papers push on the cost side of the stack: scaling laws for MoE diffusion language models, training-free token compression for omni-modal models, and cache- and quantization-level fixes, alongside a reminder that a widely-deployed positional encoding has a quiet numerical bug.

Paper Hugging Face

LLaDA MoE v2: Scaling Mixture-of-Experts Diffusion Language Models

LLaDA MoE v2, a 30B-A3B diffusion LM trained on 23.5T tokens, approaches Qwen3 on reasoning and coding using about 65% as many pretraining tokens.

mixture-of-experts diffusion training

Source → Arc

Paper Hugging Face

When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings

ALiBi's linear bias scaling underflows floating-point precision in state-of-the-art pretrained models, silently zeroing out attention weights and impairing token retrieval.

attention long-context training

Source → Arc

Paper Hugging Face

OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models

OmniPack's training-free token compression keeps 92.9% of Omni-LLM performance at just 6.8% of original FLOPs.

Performance retained 92.9%FLOPs used 6.8%

multimodal inference quantization

Source → Arc

Paper Hugging Face

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

RestoreKV adds a learned restoration pass to KV-cache eviction, raising RULER-4K accuracy from 38.2 to 73.2 at a 5% cache budget.

RULER-4K accuracy at 5% budget 38.2 to 73.2

long-context inference

Source → Arc

Paper Hugging Face

AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling

AURORA-LM separates a decodable text latent from its diffusion distribution, topping other continuous and diffusion-based LMs on OpenWebText and XSum.

diffusion training

Source → Arc

Paper Hugging Face

ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

ARCHead compresses the LM output head separately from quantized transformer blocks, cutting head storage 3.7-3.9x with minimal perplexity loss.

quantization inference

Source → Arc

Applied and niche tools: design, robotics, retrieval, recsys

A grab-bag of domain-specific systems, the strongest of which is already validated in production: Knowledge-Geometry Decoupling is live on Shopee with a measured revenue lift, while the rest are research-stage design, cleaning-robot, and retrieval tools.

Paper Hugging Face

Knowledge-Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation

KGD decouples refreshable pretrained knowledge from task-specific geometry for streaming recommenders; live on Shopee, it lifted homepage-search GMV per user 1.75% and ad revenue 1.53%.

GMV per user +1.75%Ad revenue +1.53%Offline benchmark gain 4-12%

Why it matters

A rare production A/B result rather than an offline benchmark claim.
Addresses a real deployment problem: continually refreshing a pretrained model without breaking downstream task adaptation.

training products market

Source → Arc

Paper Hugging Face

UniWorld-Design: From Pixel Generation to Layer-Native Design

UniWorld-Design generates and edits images as semantic RGBA layers instead of flat pixels, cutting per-layer error 37% over Qwen-Image-Layered.

image-generation multimodal

Source → Arc

Paper Hugging Face

Search, Inspect, Fetch: Exploiting Boolean Retrieval for Deep-Research Agents

SIEVE lets deep-research agents query document fields via Boolean retrieval instead of reading whole pages, beating standard search-visit while using 20-50% fewer tokens.

agents retrieval rag

Source → Arc

Paper Hugging Face

PosterMELD: Multi-Agent Paper-to-Poster Generation for Controllable Design Diversity with Editable Print-Ready Outputs

PosterMELD's multi-agent pipeline turns papers into editable, print-ready posters at 81.3% success and $0.38 per request.

agents image-generation

Source → Arc

Paper Hugging Face

Push-Wiper: Toward General-Purpose Robotic Cleaning across Varied Stains and Surfaces with Segmented Pushing Trajectories

Push-Wiper reframes robotic stain cleaning as an aggregation problem, removing up to 130% more stain area than baseline wiping methods.

robotics imitation-learning

Source → Arc

Paper Hugging Face

Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing

Hunyuan3D-Buffalo unifies 3D understanding, generation, and editing in one model trained on an 87M-sample multimodal corpus.

multimodal image-generation

Source → Arc

Also today

Paper · Hugging Face CAPEval: A Decoupled Caption Evaluation across Understanding and Generation — CAPEval decouples caption quality into coverage and precision, finding coverage predicts understanding tasks and precision predicts generation tasks.
Paper · Hugging Face Decoding Children's Gait Behavior — A new 1,100-video dataset of pediatric gait shows current gait foundation models and MLLMs fail to resolve clinically relevant nuances.
Paper · Hugging Face LegalPincite: Multi-level Legal Information Retrieval Dataset — LegalPincite is a paragraph-level legal citation retrieval dataset built from CJEU judgments, addressing leakage and oversimplification in prior benchmarks.
Paper · Hugging Face ChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels — ChronoLens measures 220 years of parliamentary language change across five traditions, finding linguistic levels shift by similar magnitudes but diverge in direction.
Paper · Hugging Face Multi-Task Multi-Frame Visual Piano Transcription — V2N sets new state-of-the-art on visual piano transcription by jointly predicting onset, offset, key hold, and velocity from video.
Paper · Hugging Face Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation — STAMPlus resolves the segmentation-dialogue-speed trilemma for MLLMs, cutting 12-category latency from 13.5s to 5.16s in one unified checkpoint.
Paper · Hugging Face CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning — CURV's curriculum learning framework improves chart question answering up to 20.5% by teaching models step-wise visual grounding instead of relying on prompting alone.