# Research Papers — 2026-07 · bundle page 150 of 222

> Full-text markdown of papers #7451–7500 of 11075 published in 2026-07, newest first. AI-readable mirrors of the full 11075-paper 2026-07 archive (50 papers per file).

Canonical page: https://aipod.dev/papers/archive/2026-07/page/150
[← Previous bundle](https://aipod.dev/papers/archive/2026-07/page/149.md)
[Next bundle →](https://aipod.dev/papers/archive/2026-07/page/151.md)

### When Classical Baselines Are Tuned as Carefully as the Quantum Model, Does Quantum Reservoir Computing Still Win?

Authors: Tushar Pandey
Published: 2026-07-10
Categories: quant-ph, cs.LG
arXiv: https://arxiv.org/abs/2607.09905v1

Can a small quantum computer forecast a changing signal better than an ordinary classical method? Many studies say yes, but the classical methods they compare against are often left in a basic, untuned state while the quantum model is carefully optimised. We ask what happens when the classical competitor is given exactly the same care: the same size and the same amount of tuning effort. We study two popular reasons a quantum reservoir is thought to help, using exact simulations of small quantum systems (up to eleven qubits) on prediction tasks. In both cases the quantum advantage disappears once the comparison is fair. In the first, extra quantum measurements add nothing that a simple classical formula of the same size does not already provide. In the second, a feedback loop genuinely helps the quantum model, turning a useless setup into a working predictor, yet a well-tuned classical network still predicts slightly more accurately, and the gap is statistically reliable. Our point is not that quantum reservoirs can never win, but that two of their commonly cited advantages do not hold up against fair classical competitors at this scale. We provide these matched comparisons as a simple, reusable checklist for honest benchmarking. All results are fully reproducible from fixed random seeds.

---

### Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Code

Authors: Chunqiu Steven Xia, Courtney Miller
Published: 2026-07-10
Categories: cs.SE, cs.AI
arXiv: https://arxiv.org/abs/2607.09902v1

Agentic coding tools are increasingly used to make autonomous repository-level changes to real-world projects. Prior work has largely evaluated these contributions at the pre-merge stage, through outcomes such as pull request acceptance and review effort. Far less is known about what happens to agentic code post-merge. Yet merge success alone does not reveal whether a contribution will remain stable or require bug fixes and other corrective maintenance downstream. We conduct a longitudinal empirical analysis of agentic and human contributions across 182 repositories, tracking their post-merge fate over time, characterizing the intent of subsequent modifications, and analyzing the defects and vulnerabilities they introduce. While the overall maintenance rates are similar, agentic contributions require significantly higher rates of corrective maintenance and introduce more security weaknesses and dependency vulnerabilities. We also find statistically significant evidence that agentic maintenance burden is associated with repository characteristics. In particular, each 10 percentage-point increase in a project's no-review rate is associated with roughly a 6% increase in agentic maintenance burden on average. As coding agents become pervasive in software development, our findings highlight the need to evaluate and design agentic tools not only to produce mergeable changes, but to produce contributions that remain secure and maintainable.

---

### An End-to-End Hybrid Quantum--Classical Sampling Workflow for Discrete Markov Random Fields: A Reproducible Case Study

Authors: Arul Rhik Mazumder
Published: 2026-07-10
Categories: quant-ph, cs.LG
arXiv: https://arxiv.org/abs/2607.09893v2

Sampling from discrete Markov random fields (MRFs) is a hard problem. We study amplitude-encoded i.i.d. sampling for small MRFs where $2^n$ target probabilities are precomputed classically. This removes quantum exponential speedup but allows a clean comparison against classical MCMC based on independent circuit samples ($τ\approx 1$). Across 60 instances spanning five graph families (1k-step burn-in, 3k retained samples), the mean ESS ratios of Quantum to Single-Site Gibbs, Block Gibbs, Tuned-Block, and Parallel Tempering are $16.35$, $7.29$, $1.82$, and $1.79$, showing modern classical samplers substantially close this gap. Amortizing $O(2^n)$ preprocessing into wall-clock time, exact inverse-CDF sampling yields $17.7\text{M}$ ESS/s versus $488\text{K}$ ESS/s for the quantum sampler ($36\times$ mean rate, $153\times$ per-instance), confirming no wall-clock advantage. We characterize MCMC autocorrelation costs and benchmark amplitude-encoded state preparation at $n \in \{8,10,12\}$. An MPS scaling study ($n \le 40$) shows bond dimension $χ=32$ achieves $F=0.721\pm0.059$ at $n=40$. Finally, a matched-budget VQC vs. MPS comparison at $n \in \{8,10,12\}$ shows VQC fidelities fall far below MPS: $(F_{\mathrm{VQC}}, F_{\mathrm{MPS}}) = (0.31, 0.99), (0.21, 0.96), (0.17, 0.88)$ at compressions $10.7\times$, $34.1\times$, and $113.8\times$.

---

### An End-to-End Hybrid Quantum--Classical Sampling Workflow for Discrete Markov Random Fields: A Reproducible Case Study

Authors: Arul Rhik Mazumder
Published: 2026-07-10
Categories: quant-ph, cs.LG
arXiv: https://arxiv.org/abs/2607.09893v1

Sampling from discrete Markov random fields (MRFs) is a hard problem. We study amplitude-encoded i.i.d. sampling for small MRFs where $2^n$ target probabilities are precomputed classically. This removes quantum exponential speedup but allows a clean comparison against classical MCMC based on independent circuit samples ($τ\approx 1$). Across 60 instances spanning five graph families (1k-step burn-in, 3k retained samples), the mean ESS ratios of Quantum to Single-Site Gibbs, Block Gibbs, Tuned-Block, and Parallel Tempering are $16.35$, $7.29$, $1.82$, and $1.79$, showing modern classical samplers substantially close this gap. Amortizing $O(2^n)$ preprocessing into wall-clock time, exact inverse-CDF sampling yields $17.7\text{M}$ ESS/s versus $488\text{K}$ ESS/s for the quantum sampler ($36\times$ mean rate, $153\times$ per-instance), confirming no wall-clock advantage. We characterize MCMC autocorrelation costs and benchmark amplitude-encoded state preparation at $n \in \{8,10,12\}$. An MPS scaling study ($n \le 40$) shows bond dimension $χ=32$ achieves $F=0.721\pm0.059$ at $n=40$. Finally, a matched-budget VQC vs. MPS comparison at $n \in \{8,10,12\}$ shows VQC fidelities fall far below MPS: $(F_{\mathrm{VQC}}, F_{\mathrm{MPS}}) = (0.31, 0.99), (0.21, 0.96), (0.17, 0.88)$ at compressions $10.7\times$, $34.1\times$, and $113.8\times$.

---

### Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling

Authors: Chicago Y. Park, Jialin Mao, Xiaojian Xu, Taha Kass-Hout, Ulugbek S. Kamilov, Cao Xiao
Published: 2026-07-10
Categories: eess.IV, cs.AI
arXiv: https://arxiv.org/abs/2607.09892v1

We introduce DenseAR, a new generative paradigm that reformulates autoregressive image generation as coarse-to-fine next-dense-stride prediction using a compact single-scale tokenizer. Our key insight is that traversing a single-scale latent grid with progressively denser strides naturally captures the transition from global structure to fine detail. This addresses two limitations of existing autoregressive models at once: the slow inference of raster-order autoregression, which DenseAR avoids by predicting multiple tokens in parallel, and the heavy cost of multi-scale approaches, which need long, multi-resolution token sequences to achieve coarse-to-fine prediction. Building on our efficient framework and the flexibility of autoregressive modeling, we further extend DenseAR to a unified model that handles multiple modalities and imaging tasks within a single backbone. We validate DenseAR on both medical and natural images. On multi-contrast brain MRI, a single DenseAR model unifies cross-modal translation, modality-conditioned generation, and tumor segmentation, while remaining competitive with task-specific methods. On ImageNet, DenseAR improves class-conditional generation quality (FID and IS) over both a single-grid baseline without stride ordering and a multi-scale tokenizer-based baseline.

---

### What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection

Authors: Aishwarya R. Fursule, Vamshi Nallaguntla, Shruti Kshirsagar, Anderson R. Avila
Published: 2026-07-10
Categories: cs.SD, cs.AI
arXiv: https://arxiv.org/abs/2607.09891v1

Audio deepfake detection models determine whether speech is genuine or artificially generated, but high overall accuracy can mask substantial performance disparities across demographic groups. In this work, we investigate gender bias in audio deepfake detection using the ASVspoof5 dataset. We use ASVspoof5 under a controlled custom split designed to isolate gender-composition effects. We train attack-specific models on nine training sets with different gender compositions, ranging from female-only to male-only. We use a ResNet18 classifier with LogSpectrogram and WavLM-Base+ features, and we evaluated six post-hoc threshold calibration methods. Experimental results show that training data composition strongly predicts bias direction, with the underrepresented gender performing worse at test time. WavLM-Base+ features are shown to produce gender performance gaps 3.0 to 4.3 times larger than LogSpectrogram under identical training conditions, and balanced training is found to reduce LogSpectrogram bias but leave WavLM bias largely intact. Moreover, all six calibration strategies, including Oracle calibration with full test-set label access, leave the Equal Error Rate gap unchanged at 1.317 pp, confirming that threshold adjustment cannot correct underlying score distribution disparities. Overall, these findings suggest that gender fairness in audio deepfake detection must be addressed at training time, as post-hoc methods can only partially mitigate the resulting disparities

---

### Do Transformer Temporal Heads and Post-Pooling Motion Gates Help CorrNet-based CSLR? An Empirical Study

Authors: Lisi Wang, Zhidong Xiao, Jianjun Peng
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09890v1

CorrNet is a strong baseline for continuous sign language recognition (CSLR) because it models inter-frame correlations inside the visual encoding stage. In this paper, we study two natural extensions of a reproduced CorrNet system: replacing the BiLSTM temporal head with a Transformer encoder, and injecting motion cues after temporal pooling. We find that the Transformer head does not outperform the BiLSTM baseline, even with a training strategy adjusted for the Transformer, and the two heads have almost the same computational and runtime cost. For the second extension, we design a lightweight module called MotionGate. In our experiments, MotionGate consistently collapses to an identity-like mapping: the gate loses motion selectivity, and the injected residual becomes a weak, non-selective perturbation of the pooled features. These results suggest that explicit motion injection after CorrNet's correlation-based encoding is largely redundant, and that natural-looking architectural extensions in CSLR should be tested carefully instead of being assumed to help.

---

### Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention

Authors: Siddharth Pal, Viktoria Rojkova
Published: 2026-07-10
Categories: cs.LG, cs.AI, cs.CL, cs.NE
arXiv: https://arxiv.org/abs/2607.09889v1

Fixed-state sequence models compress an unbounded past into a bounded state, which caps their associative recall at roughly the state dimension; attention escapes the cap by keeping a key-value entry for every token, at quadratic compute and a cache that grows with the sequence. We study the middle ground: a sparse cache that allocates a slot only when an input is novel, so its size tracks the number of distinct items rather than the number of tokens. The allocation rule is the DP-means clustering rule, the small-variance limit of a Dirichlet-process mixture, used not as latent-variable inference but as the key-value memory operator for a deep recurrent backbone. We develop it in two forms, a static cache with a fixed concentration and a surprise-adaptive variant whose concentration follows the recent novelty rate. On a controlled associative-recall benchmark with redundancy we show that the cache matches full-attention recall while storing only the distinct items, that it dominates a fixed-budget eviction cache on the recall-versus-size frontier, and that on a state-space backbone it answers both a recall query and a long-range aggregate at the lowest memory of any model tested. The allocation is learnable end to end: a two-parameter novelty-threshold gate trained on the task loss alone recovers the rule exactly, whereas an over-parameterized gate fails, so the operative ingredient is the inductive bias rather than capacity. The evidence is a family of controlled mechanism studies at modest scale, with the distinct-items property confirmed on four real streams (recommendation, systems logs, clinical events, and insurance claims); a real-backbone, real-corpus language validation is pursued in a companion study.

---

### Bridging the Catalog-to-Real Gap: Scalable Product Recognition via Multi-Stage Contrastive Learning

Authors: Anyi Zhang, Joy Mazumder, Kiril Lomakin
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09888v1

Automated product recognition is a cornerstone of modern retail intelligence; however, accurately matching real-world, in-store images against extensive corporate catalogs remains a major scalability bottleneck for large-scale applications. In this work, we address this challenge by reformulating the task as an embedding-based cross-domain retrieval problem rather than a standard closed-set classification task. Specifically, we define the objective as retrieving the most corresponding catalog reference image for a given real-world product query crop from an expansive inventory. To bridge the severe domain gap between pristine studio packshots and noisy in-store queries, we introduce a novel catalog-to-real multi-stage contrastive learning paradigm (Cat2Real). This framework fine-tunes a vision backbone by systematically exploiting both item-level and image-level similarities to drive targeted hard negative mining. Extensive empirical evaluations demonstrate that our paradigm scales seamlessly to unseen products and categories, yielding outstanding zero-shot generalization performance even in the complete absence of real-world training images for novel inventory.

---

### Nonparametric Bayesian Inverse Reinforcement Learning with Data-Parallel Gibbs Sampling

Authors: Sai Anirudh Katupilla, Shreeya Dasa Lakshminath
Published: 2026-07-10
Categories: cs.LG
arXiv: https://arxiv.org/abs/2607.09886v1

Inverse Reinforcement Learning recovers reward functions from expert demonstrations, but standard formulations assume that all demonstrations come from a single expert. When demonstrations are pooled from multiple experts with distinct preferences, parametric methods recover an averaged reward that fits no individual expert well. We implement Nonparametric Bayesian Inverse Reinforcement Learning with a Dirichlet Process prior over reward functions, allowing the number of latent reward types to be inferred jointly with the rewards themselves. Inference uses a collapsed Gibbs sampler combining a Chinese Restaurant Process update for cluster assignments with a Metropolis-Hastings update for reward weights, and soft value iteration as the inner planning routine. We evaluate on a 10x10 ObjectWorld grid with two and three ground-truth reward types. The serial sampler recovers K=2 with Adjusted Rand Index of 1.000, substantially outperforming a Maximum Entropy IRL baseline (ARI=0.000). Extension to K=3 shows that the sampler correctly identifies the number of clusters in all runs; assignment ARI of 0.48-0.58 reflects behavioral overlap between expert types that persists across grid instantiations, revealing that reliable K=3 evaluation on ObjectWorld requires controlled object placement rather than random seeding. We further parallelize the sampler across CPU cores using Ray on HPC hardware, achieving a peak speedup of 4.79x at 8 workers, and characterize a throughput-versus-accuracy tradeoff arising from the consensus merge heuristic used during state aggregation. Code and a containerized environment are available at https://github.com/dasashreeya/np_bayes_irl.

---

### Index SLM Technical Report

Authors: Tianjiao Li, Lusheng Zhang, Shien He, Xiaojing Liu, Tianxing Yan, Mengran Yu, Ziang Cui, Kai Zhao, Xipeng Wang, Yang Liu, Yuxin Li
Published: 2026-07-10
Categories: cs.CL
arXiv: https://arxiv.org/abs/2607.09885v3

We present Index-1.9B, a series of open small language models developed at Bilibili. The series comprises four models: Index-1.9B-Base, a foundation model with 1.9 billion non-embedding parameters pre-trained on 2.8 trillion predominantly Chinese and English tokens; Index-1.9B-Pure, a control variant trained with an identical recipe but with all instruction-like data strictly filtered from the corpus; Index-1.9B-Chat, aligned from the base model with supervised fine-tuning and direct preference optimization; and Index-1.9B-Character, which augments the chat model with retrieval-augmented generation for few-shot role-playing customization. Pre-training employs a Warmup-Stable-Decay learning-rate schedule in which the concentration of curated data is raised substantially during the decay phase, together with a Norm-Head output layer that stabilizes training under large learning rates. On a suite of standard benchmarks covering examination, reasoning, mathematics, and code, Index-1.9B-Base attains an average score of 64.92, competitive with or exceeding open models of several times its size. We further report controlled studies on model depth, learning-rate magnitude and scheduling, the interaction between learning-rate decay and data quality, and the effect of including instruction data during pre-training, and we document an unexplained surge in benchmark performance midway through the constant-learning-rate phase. All models, together with evaluation code, are released at https://github.com/bilibili/Index-1.9B.

---

### Index SLM Technical Report

Authors: Lusheng Zhang, Shien He, Tianxing Yan, Mengran Yu, Ziang Cui, Kai Zhao, Xiaojing Liu, Tianjiao Li
Published: 2026-07-10
Categories: cs.CL
arXiv: https://arxiv.org/abs/2607.09885v1

We present Index-1.9B, a series of open small language models developed at Bilibili. The series comprises four models: Index-1.9B-Base, a foundation model with 1.9 billion non-embedding parameters pre-trained on 2.8 trillion predominantly Chinese and English tokens; Index-1.9B-Pure, a control variant trained with an identical recipe but with all instruction-like data strictly filtered from the corpus; Index-1.9B-Chat, aligned from the base model with supervised fine-tuning and direct preference optimization; and Index-1.9B-Character, which augments the chat model with retrieval-augmented generation for few-shot role-playing customization. Pre-training employs a Warmup-Stable-Decay learning-rate schedule in which the concentration of curated data is raised substantially during the decay phase, together with a Norm-Head output layer that stabilizes training under large learning rates. On a suite of standard benchmarks covering examination, reasoning, mathematics, and code, Index-1.9B-Base attains an average score of 64.92, competitive with or exceeding open models of several times its size. We further report controlled studies on model depth, learning-rate magnitude and scheduling, the interaction between learning-rate decay and data quality, and the effect of including instruction data during pre-training, and we document an unexplained surge in benchmark performance midway through the constant-learning-rate phase. All models, together with evaluation code, are released at https://github.com/bilibili/Index-1.9B.

---

### ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

Authors: Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak, Minh Nguyen, Ruining Deng, Mert R. Sabuncu
Published: 2026-07-10
Categories: cs.CV, cs.LG
arXiv: https://arxiv.org/abs/2607.09884v1

Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over-relies on a highly predictive modality and undertrains complementary sources, resulting in poor robustness under partial availability. While training-time modality knockout improves missing-modality robustness, existing approaches use static masking rates that cannot adapt to evolving modality utility during training. We introduce ShapKO (Shapley-Adaptive Modality Knockout), a dynamic training strategy that learns modality-specific knockout probabilities based on validation utility. ShapKO periodically evaluates performance across modality subsets, estimates modality importance via Shapley values, and updates masking probabilities to suppress dominant modalities more frequently. This adaptive process promotes complementary representations, while requiring no architectural modifications. We evaluate ShapKO on three datasets covering multitask clinical classification, survival prediction, and cancer detection. ShapKO consistently improves performance under modality absence and yields interpretable trajectories of learned masking behavior. Code is available at: https://github.com/sumona00/ShapKO

---

### A Dual-Stream Challenge-Response Protocol for Ocular Liveness Verification

Authors: Ismail Kably
Published: 2026-07-10
Categories: cs.CV, cs.CR
arXiv: https://arxiv.org/abs/2607.09883v1

Ocular biometric systems face sophisticated presentation attacks, including high-resolution video replays and real-time generative deepfakes, which easily bypass static liveness checks. Current Presentation Attack Detection (PAD) frameworks typically rely on isolated physiological metrics, such as gaze tracking or the Pupillary Light Reflex (PLR), which can be spoofed independently. This paper proposes a Spatio-Luminance Sensor Fusion protocol, which introduces a dual-stream challenge-response framework for ocular liveness verification by uniting these metrics into a simultaneous authentication challenge. By generating a randomized, time-varying visual stimulus that fluctuates in both spatial trajectory and luminance intensity, we construct a mathematically coupled state-space likelihood model, termed the Synchronization Matrix, to evaluate the continuous cross-correlation between the expected biological latencies of smooth pursuit tracking and pupillary constriction. Using Monte Carlo simulation grounded in literature-derived latency distributions, we demonstrate theoretical separability between genuine and simulated attack conditions, and show that a multi-round challenge design improves the detection of generative deepfakes when a non-zero rendering-latency gap exists. This work provides a simulation-supported theoretical framework for next-generation dynamic spoofing defense in ocular and iris biometrics; human-subject validation is identified as necessary future work before deployment claims can be made.

---

### CLIR-Bench: Benchmarking Multimodal Question Answering over Irregular Clinical Time Series

Authors: Frank Nie, Ethan B. Liu, Yuan Zhu, Loe Yan, Wei Fan, Jindong Han
Published: 2026-07-10
Categories: cs.CL, cs.AI
arXiv: https://arxiv.org/abs/2607.09880v1

Clinical time series are central to patient monitoring, risk assessment, and clinical decision support. However, they are often sparse, irregularly sampled, and asynchronous, making it difficult for models to identify the temporal evidence required for clinical Question Answering (QA). Existing benchmarks primarily focus on regularly sampled time-series QA or medical QA over static data, and therefore rarely assess whether models can faithfully ground their answers in irregular temporal observations. To fill this gap, we introduce CLIR-Bench, a benchmark for irregular clinical time series QA constructed from de-identified ICU records through a principled four-stage pipeline. CLIR-Bench contains 6,600 QA instances spanning 11 clinical variables, organized into four capability dimensions and 11 tasks. Each question is linked to explicit temporal evidence and task-specific answer derivation rules, enabling evaluation of both answer accuracy and evidence use. Experiments show that existing generalist models struggle to retrieve and reason over sparse clinical evidence, highlighting the need for stronger irregular time-series reasoning methods. Our code and data are available at https://huggingface.co/datasets/winall/CLIR-Bench.

---

### Serving the Long Tail: Training-Free LLM Candidate Generation for Vacation Rental Marketplaces

Authors: Syed Mohammed Arshad Zaidi, Eric Rincon, Shayan Hassantabar
Published: 2026-07-10
Categories: cs.LG, cs.IR
arXiv: https://arxiv.org/abs/2607.09877v1

Vacation rental marketplaces face a structural imbalance on the supply side: a small fraction of properties receive most user interactions, while the long tail of new, niche, and seasonal listings generates too little behavioral signal for collaborative filtering to serve effectively. At Vrbo, item-based k-nearest neighbors (IBKNN) is a core candidate generation channel, but leaves tens of thousands of properties with no candidates and produces weak neighborhoods for sparsely interacted ones. We present a training-free, LLM-based candidate generation pipeline that complements IBKNN using static property metadata alone. An off-the-shelf LLM synthesizes diverse semantic queries per property, a pre-trained text encoder embeds them, and an approximate nearest-neighbor index retrieves candidates from an 11.7M-property catalog. A Union fusion strategy merges these with IBKNN while preserving the behavioral channel's ordering, guaranteeing no degradation on well-served properties, and a downstream learning-to-rank model re-scores the fused pool. Evaluated on 1.6M focal properties, the system extends candidate coverage to tens of thousands of properties IBKNN cannot reach, delivers its largest gains on the long-tail segment where behavioral methods are weakest, and matches or beats IBKNN at every K on shared properties. A downstream learning-to-rank stage further lifts the fused pool, yielding a complete candidate generation and re-ranking stack that serves the long tail without regressing well-served properties. We additionally show that Union fusion collapses the recall gap between a 3B open-weights LLM and frontier API-based models from 27-46% to under 1%, supporting self-hosted small-model deployment at marketplace catalog scale.

---

### Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data

Authors: Valentin Gabeff, Baptiste Maquignaz, Jennifer Shan, Sepideh Mamooler, Gencer Sumbul, Blair Costelloe, Devis Tuia, Alexander Mathis
Published: 2026-07-10
Categories: cs.CV, cs.AI, q-bio.NC, q-bio.QM
arXiv: https://arxiv.org/abs/2607.09876v1

Automatically retrieving videos from large camera-trap datasets remains challenging. Text-to-Video retrieval (TVR) methods based on large video-language models (VLMs) have potential to retrieve events of interest by describing them with simple text queries. However, current methods often lack spatiotemporal understanding and do not generalize well to ecological data. In this work, we introduce Prompting-MammAlps, the first camera-trap TVR benchmark, and propose a fine-grained and interpretable TVR method. Specifically, we trained a vision transformer to perform spatiotemporal action localization, and convert its output to structured text, describing each video. Independently, ethology-inspired queries are processed by a Large-Language Model (LLM) based coding agent to parse the structured text per video and retrieve videos accordingly. We harnessed the LLM to use functions from a custom parsing library to minimize the risk of LLM hallucinations and to improve method interpretability. This retrieval approach applied on the Prompting-MammAlps benchmark achieved a set-based F1-score of 34\% on a test set of 135 ecologically-relevant queries and 775 candidate videos. In comparison the best zero-shot VLM achieved a F1-score of 18\%, while also lacking interpretability. Project page: https://cnai.epfl.ch/prompting-mammalps

---

### Nonlinear Axiomatic Attribution for Cooperative Games

Authors: Weida Li, Zhuanghua Liu, Yaoliang Yu, Bryan Kian Hsiang Low
Published: 2026-07-10
Categories: cs.LG
arXiv: https://arxiv.org/abs/2607.09869v2

The Shapley value is a widely used concept in attribution problems, as it uniquely satisfies the axioms of linearity, consistency, equal treatment, and efficiency. Often, the inclusion AUC metric is used to evaluate the quality of player rankings, in order to identify positively participating players. However, it can be established that the Shapley value is not always reliable for this purpose. The core issue lies in its linearity: the Shapley value acts as a linear operator with an excessively large null space, which is likely to contain non-negligible perturbations that remain indistinguishable to the operator. To address this limitation, we explore the design of nonlinear axiomatic attribution methods. Inspired by the least core, which is a popular nonlinear substitute for the Shapley value, we introduce a class of nonlinear attribution methods that retain the remaining necessary axioms. Each method yields a contribution vector that is the unique optimal solution to a minimization problem, which aims to approximate utility functions as faithfully as possible. In terms of the inclusion AUC metric, our experiments demonstrate the potential effectiveness of these methods compared to Shapley value variants that relax only the efficiency axiom. Our code is available at https://github.com/watml/nonlinear-axiom.

---

### Nonlinear Axiomatic Attribution for Cooperative Games

Authors: Weida Li, Zhuanghua Liu, Yaoliang Yu, Bryan Kian Hsiang Low
Published: 2026-07-10
Categories: cs.LG
arXiv: https://arxiv.org/abs/2607.09869v1

The Shapley value is a widely used concept in attribution problems, as it uniquely satisfies the axioms of linearity, consistency, equal treatment, and efficiency. Often, the inclusion AUC metric is used to evaluate the quality of player rankings, in order to identify positively participating players. However, it can be established that the Shapley value is not always reliable for this purpose. The core issue lies in its linearity: the Shapley value acts as a linear operator with an excessively large null space, which is likely to contain non-negligible perturbations that remain indistinguishable to the operator. To address this limitation, we explore the design of nonlinear axiomatic attribution methods. Inspired by the least core, which is a popular nonlinear substitute for the Shapley value, we introduce a class of nonlinear attribution methods that retain the remaining necessary axioms. Each method yields a contribution vector that is the unique optimal solution to a minimization problem, which aims to approximate utility functions as faithfully as possible. In terms of the inclusion AUC metric, our experiments demonstrate the potential effectiveness of these methods compared to Shapley value variants that relax only the efficiency axiom. Our code is available at https://github.com/watml/nonlinear-axiom.

---

### Neural Posterior Estimation for Inferring Weak Lensing Shear

Authors: Tim White, Dingrui Tao, Camille Avestruz, Jeffrey Regier, the LSST Dark Energy Science Collaboration
Published: 2026-07-10
Categories: astro-ph.IM, astro-ph.CO, cs.CV, stat.AP
arXiv: https://arxiv.org/abs/2607.09867v1

The prevailing approach to inferring weak gravitational lensing shear from images involves detecting galaxies, estimating their ellipticities, and calibrating these estimates to correct for image noise, selection bias, and model misspecification. Characterizing the statistical model and assumptions underlying this pipeline is challenging, which makes it difficult to propagate uncertainty through its various stages. As an alternative, we propose to infer shear using neural posterior estimation (NPE), a type of simulation-based inference. We train a deep neural network to map a simulated multiband image to a variational distribution over the underlying shear field, thereby folding galaxy detection, deblending, measurement, and calibration into a single implicit inference step. Once trained, the network accounts for all features present in the simulated images, including potential sources of bias. In experiments on simulated constant-shear images with increasingly complex observational effects, NPE produces accurate and well-calibrated posterior approximations for both shear components in the presence of blended galaxies, spatially varying point spread functions, stars, and detector artifacts. These results demonstrate that NPE can be a viable shear estimation method in settings where all anticipated features and artifacts can be simulated, a requirement that will become increasingly feasible as simulation fidelity improves in the coming decades.

---

### Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning

Authors: Wenke Xia, Pei Ren, Wenbo Yu, Yizhuo Zhang, Jifan Li, Yixue Zhang, Yinuo Zhao, Qingyang Gao, Jianlong Fu, Jian Tang, Ji-Rong Wen, Zhengping Che et al.
Published: 2026-07-10
Categories: cs.RO, cs.AI
arXiv: https://arxiv.org/abs/2607.09866v1

Offline-to-online reinforcement learning is promising for generalizable robotic manipulation, yet its full-stack complexity obscures reproduction and diagnosis. Within such systems, value estimation plays a central role in prioritizing heterogeneous data for policy improvement. Despite its importance, the central question remains underexplored: how value-function reliability shapes policy optimization in offline-to-online reinforcement learning. To answer this question, we propose Robo-ValueRL, a unified framework that enables reliable value estimation and systematically traces its downstream effects on policy pretraining and online improvement. Concretely, Robo-ValueRL learns a history-conditioned value estimator and evaluates its reliability through global-progress and local-preference metrics. These resulting value estimates are propagated into quality-conditioned consistency-policy pretraining and a residual adaptation module on online rollouts, providing a unified testbed for analyzing how value reliability shapes downstream policy performance. Across 240 hours of offline demonstrations and over 3,000 online rollout trajectories, our extensive experiments show that downstream performance is strongly associated with value reliability. Reliable value functions provide better action-quality estimates, allowing value-guided offline RL to scale more effectively than quality-agnostic behavior cloning, and stabilize online improvement by prioritizing high-quality rollout data. Integrating reliable value guidance through offline pretraining with online improvement, our system achieves 86% success on millimeter-level precise chip insertion and 84% on generalizable block disassembly. We hope these findings highlight the importance of value-guided data utilization for effective policy improvement from heterogeneous robotic experience.

---

### From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

Authors: Jorge A. Castillo, Marco Torres Yévenes, Juan Carlos Lanas
Published: 2026-07-10
Categories: cs.LG, cs.CL
arXiv: https://arxiv.org/abs/2607.09842v2

We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models spanning four post-training regimes: no training (Gemma-4-E4B base), multimodal RLHF (Gemma-4-E4B-it), RL distillation (DeepSeek-R1-Distill-Qwen-7B), and SFT (Qwen2.5-7B-Instruct). Three prompt conditions (an identity-specifying axis prompt, a length-matched generic-assistant prompt, and a 26-token vanilla baseline) are compared via five geometric metrics, principally the 1-Wasserstein distance between edge-wise distributions of Ollivier-Ricci curvature on k-NN trajectory graphs. Claims rest on trajectory-level permutation tests with multiple geometric controls (teacher-forced content controls, temporal-chain vs k-NN topology, ABT-projected k-NN, angular vs Euclidean graph construction, B=5000 permutations on borderline statistics). The central finding is a qualitative reorganization of identity encoding across the instruction-tuning boundary: in the base model the fingerprint is direction-coded (separation 0.034, p=0.002 under angular k-NN); in the multimodal instruction-tuned model it migrates into the magnitude (angular separation collapses to p=0.439 while Euclidean survives at p=0.042, and the mean norm of the first generated state inverts its length-ordering, being lowest for the identity prompt). This direction-to-magnitude reorganization is specific to the multimodal instruction-tuning regime, absent under RL distillation and SFT. A teacher-forced control attributes ~30% of the free-running cosine signal to prompt-driven effects. We position W_1 on edge-wise Ollivier-Ricci distributions on k-NN trajectory graphs as a methodological contribution of independent interest.

---

### Exploring Agentic Workflows for Generating High Quality Math Visual Aids

Authors: Rizwaan Malik, Ashna Khetan, Isabel Sieh, Samin Khan
Published: 2026-07-10
Categories: cs.AI, cs.CV, cs.HC
arXiv: https://arxiv.org/abs/2607.09839v1

Mathematical diagrams play a crucial role in K 12 education, both as problem components and as scaffolding for student comprehension. However, current AI tools, including Large Language Models (LLMs), struggle to reliably generate accurate and pedagogically sound visual diagrams, even when provided with detailed descriptions. A significant gap therefore remains in the reliable generation of diagrams for middle school mathematics. To address this, we introduce an agentic workflow that enables LLM agents to evaluate the quality of generated visuals and use this feedback to iteratively improve their outputs. This self improvement loop aims to enhance the accuracy and educational appropriateness of AI generated diagrams. Our research investigates two questions. First, can LLMs accurately generate quality assurance questions for a visual aid given specific criteria for visual quality? Second, given valid quality assurance questions, can Vision Language Models effectively evaluate generated K 12 visual aids and use the resulting feedback to improve them iteratively? We conduct an exploratory evaluation of our agentic workflow and identify key areas for improvement, including stronger spatial reasoning and more comprehensive coverage of diagram features in the generated quality assurance questions. Our results provide preliminary evidence that this approach can improve the reliability and educational value of AI generated mathematical diagrams.

---

### Reliability-Aware Ensemble Classification Under Class Imbalance: A Calibration Study on Liquid-Based Cervical Cytology

Authors: Nisreen Albzour, Sarah S. Lam
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09837v1

Cervical cytology classification models are typically evaluated on curated, class-balanced benchmarks, but real-world liquid-based cytology (LBC) collections are often small and class-imbalanced. This paper presents a class-imbalance-aware and calibration-aware ensemble classification study on the Mendeley LBC dataset, using its native four-class Bethesda taxonomy (NILM, LSIL, HSIL, SCC) rather than a collapsed binary formulation. Three lightweight architectures (Swin-Tiny, TinyViT-5M, DenseNet121) are trained directly on Mendeley LBC using weighted random sampling to counteract class imbalance, and compared against two soft-voting ensembles (Hybrid-2, Hybrid-3). Post-hoc temperature scaling is fit on a held-out calibration subset carved out of the training portion of each cross-validation fold, distinct from both the training data used to fit model weights and the evaluation fold used for final metrics, avoiding the optimistic calibration estimates that result when the same data is used for both purposes. Calibration substantially reduces expected calibration error, Brier score, and negative log-likelihood for every model and ensemble configuration tested, while discrimination metrics (accuracy, macro-F1, macro-AUROC) remain essentially unchanged. Ensemble size shows no consistent additional reliability benefit over the best individual model once all configurations are properly calibrated. Confusion matrices show that all classification errors, across every configuration, are confined to the boundary between high-grade lesions (HSIL) and carcinoma (SCC); no errors involve the negative (NILM) or low-grade (LSIL) categories. These results suggest that, for this dataset, calibration is the dominant lever for reliability, not ensemble size, though this conclusion should be read in light of the dataset's modest size.

---

### Does YOLO26 Truly Offer Advantages Over Its Predecessors for Edge Deployment? A Benchmark Study in Aquaculture

Authors: Rakesh Ranjan, Gajanan S. Kothawade, Kata Sharrer, Scott Tsukuda, Christopher Good
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09835v2

The recently introduced YOLO26 architecture incorporates NMS-free end-to-end inference and is optimized for deployment on resource-constrained CPU-based devices, making it well-suited for edge-based aquaculture applications. However, its performance, operational efficiency, and deployment suitability have not been systematically validated in aquaculture-specific scenarios. This study presents a comprehensive benchmark of YOLO26 against three Ultralytics predecessors (YOLOv5u, YOLOv8, and YOLO11) across nano, small, and medium model scales for fish mortality detection, a critical indicator of fish population health and welfare. Twelve model variants were evaluated for detection accuracy, training efficiency across seven dataset sizes, and inference performance on high-performance NVIDIA A100 GPUs and a CPU-only Raspberry Pi 5 edge platform. All models achieved comparable performance on the full dataset, with mAP50 differing by only 1.04 percentage points, indicating that architectural generation has little influence on final detection accuracy when sufficient training data are available. However, clear trade-offs emerged in data efficiency and deployment performance. YOLOv8 achieved 90% mAP50 with only 400 training images, whereas the YOLO26 nano and small variants required 1,000 images to reach comparable accuracy. Conversely, YOLO26n achieved the highest inference speed on the Raspberry Pi 5 (7.51 FPS), while YOLOv5mu outperformed all contemporary medium-scale architectures on CPU-based hardware. These results show that architectural novelty alone is insufficient for model selection and that training data availability, target hardware, and inference requirements should be considered jointly when selecting object detection models for practical edge AI deployment in aquaculture.

---

### Does YOLO26 Truly Offer Advantages Over Its Predecessors for Edge Deployment? A Benchmark Study in Aquaculture

Authors: Rakesh Ranjan, Gajanan S. Kothawade, Kata Sharrer, Scott Tsukuda, Christopher Good
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09835v1

The recently introduced YOLO26 architecture incorporates NMS-free end-to-end inference and is optimized for deployment on resource-constrained CPU-based devices, making it well-suited for edge-based aquaculture applications. However, its performance, operational efficiency, and deployment suitability have not been systematically validated in aquaculture-specific scenarios. This study presents a comprehensive benchmark of YOLO26 against three Ultralytics predecessors (YOLOv5u, YOLOv8, and YOLO11) across nano, small, and medium model scales for fish mortality detection, a critical indicator of fish population health and welfare. Twelve model variants were evaluated for detection accuracy, training efficiency across seven dataset sizes, and inference performance on high-performance NVIDIA A100 GPUs and a CPU-only Raspberry Pi 5 edge platform. All models achieved comparable performance on the full dataset, with mAP50 differing by only 1.04 percentage points, indicating that architectural generation has little influence on final detection accuracy when sufficient training data are available. However, clear trade-offs emerged in data efficiency and deployment performance. YOLOv8 achieved 90% mAP50 with only 400 training images, whereas the YOLO26 nano and small variants required 1,000 images to reach comparable accuracy. Conversely, YOLO26n achieved the highest inference speed on the Raspberry Pi 5 (7.51 FPS), while YOLOv5mu outperformed all contemporary medium-scale architectures on CPU-based hardware. These results show that architectural novelty alone is insufficient for model selection and that training data availability, target hardware, and inference requirements should be considered jointly when selecting object detection models for practical edge AI deployment in aquaculture.

---

### Generative Testing of Automated Speech Recognition Systems

Authors: Yanis Xabier Wilbrand Peña, Oliver Weißl, Andrea Stocco
Published: 2026-07-10
Categories: cs.CR, cs.LG
arXiv: https://arxiv.org/abs/2607.09833v1

Automatic speech recognition (ASR) systems have achieved high accuracy with transformer-based models, enabling deployment in critical applications. However, they remain vulnerable to adversarial manipulation, particularly in black-box settings where attacks must preserve perceptual naturalness. This work introduces GATAS, a black-box testing approach that generates failure inducing inputs by operating in the phoneme-level latent space of a text- to-speech model. Instead of perturbing waveforms directly, the approach interpolates latent representations to induce transcription errors while remaining within the manifold of natural speech. The attack is formulated as a multi-objective optimization problem balancing semantic divergence and perceptual quality. Our empirical evaluation against both white-box and black-box baselines shows that GATAS achieves a 98% success rate while producing lower distortion and higher perceptual quality, as confirmed by human studies. Despite operating without gradient access, GATAS remains competitive against white-box methods, highlighting that representation and perceptual alignment are more critical than access to model internals. Overall, our results demonstrate that untargeted latent-space optimization enables the efficient generation of realistic and effective test cases for ASR systems.

---

### A Strong Balanced-Softmax Classifier-Retraining Baseline for Long-Tailed Recognition

Authors: Juan Terven, Diana Margarita Córdova Esparza, Julio Alejandro Romero Gonzalez, Edgar Arturo Chávez Urbiola, Francisco Javier Willars Rodriguez, Juan Bautista Hurtado Ramos, Alfonso Ramirez Pedraza
Published: 2026-07-10
Categories: cs.LG, cs.CV
arXiv: https://arxiv.org/abs/2607.09832v1

Long-tailed recognition methods often modify losses, margins, or representations to reduce the dominance of frequent classes. We ask whether, after Balanced Softmax training, the remaining tail error can be reduced by retraining only the classifier. We evaluate BS-cRT, a two-stage procedure that trains a backbone and cosine classifier with Balanced Softmax, freezes the backbone, and updates only the classifier on balanced episodic batches. The second stage keeps the empirical-prior Balanced Softmax objective and uses raw cosine logits at inference. Across CIFAR-100-LT, CIFAR-10-LT, ImageNet-LT, and Places-LT, this classifier-only step consistently improves Few-shot accuracy over the matched Balanced Softmax checkpoint. At imbalance factor 100, Few-shot gains are +5.15 points on CIFAR-100-LT and +5.83 on CIFAR-10-LT; on ImageNet-LT and Places-LT, gains are +6.92 and +9.78 points, respectively, with a Top-1/Few-shot trade-off on ImageNet-LT. We also analyze Counterfactual Boundary Risk Minimization (CBRM), a boundary-probe extension using prototype-based features near decision boundaries. CBRM identifies two failure modes: scaled-logit cosine margins destabilize training, and corrected hardest-negative probes remain head-class anchored. The results support BS-cRT as a practical classifier-side baseline and indicate that boundary supervision must account for class frequency.

---

### Slide-Level Active Learning Reduces Annotation Burden in H&amp;E images

Authors: Mahsa Vali, Zhilong Weng, Noémie Moreaua, Yuri Tolkach, Katarzyna Bozek
Published: 2026-07-10
Categories: eess.IV, cs.CV
arXiv: https://arxiv.org/abs/2607.09831v1

Deep learning-based segmentation of histopathology whole-slide images (WSIs) requires large amounts of pixel-level annotations, which are costly and time-consuming to obtain. Active learning (AL) has been proposed to reduce this effort, but existing methods exhibit three key limitations. Uncertainty estimation is unreliable on partially annotated WSIs, patch-level acquisition is inconsistent with slide-level annotation workflows, and class imbalance in multi-class settings is not explicitly addressed. To address these challenges, we propose SHAL (Slide-level Hybrid Active Learning), a patient-level AL framework for annotation-efficient multi-class histopathology segmentation. SHAL integrates three complementary components: a foreground-aware strategy that suppresses bias from unlabeled background regions, a stage-adaptive mechanism that hybridizes predictive entropy and epistemic uncertainty across learning stages, and a class-aware strategy that prioritizes diagnostically relevant tissue classes. SHAL is evaluated on the TCGA colorectal cancer dataset. It achieves the highest Macro Dice at the full annotation budget (0.846) and reaches Dice greater than or equal to 0.80 using only 26 percent of the budget (50 of 190 slides), whereas competing methods reach this threshold only at 37 percent (70 slides). Across five independent external cohorts, SHAL attains the highest mean external Macro Dice (0.815) and the smallest internal-to-external generalization gap among all methods (0.025 at Round 3 and 0.026 at the full budget). The results indicate that patient-level hybrid uncertainty acquisition reduces annotation cost without sacrificing cross-domain generalization in computational pathology.

---

### Tracking Intermittent Particles with Self-Learned Visual Features

Authors: Raphael Reme, Victor Piriou, Alison Hanson, Rafael Yuste, Alasdair Newson, Elsa Angelini, Jean-Christophe Olivo-Marin, Thibault Lagache
Published: 2026-07-10
Categories: eess.IV, cs.CV
arXiv: https://arxiv.org/abs/2607.09829v1

In time-lapse fluorescence imaging, single-particle-tracking is a powerful tool to monitor the dynamics of objects of interest, and extract information about biological processes. However, tracked particles can be subject to occlusion and intermittent detectability. When these phenomena persist over a few frames, tracking algorithms tend to produce multiple tracklets for the same particle. In this work, we introduce self-supervised learning of visual features to compare tracked particles, and we exploit both visual and positional distances to robustly stitch tracklets representing the same particle. We demonstrate the performance of our stitching framework on time-lapse fluorescence sequences of Hydra vulgaris neurons. Results show high stitching precision, and reduction of errors made by previous algorithms on the same data by a factor of two.

---

### Robustness and Stability Analysis of Differentiable Shift-Variant FBP for Cone-Beam CT under Challenging Acquisition Settings

Authors: Chengze Ye, Linda-Sophie Schneider, Yipeng Sun, Mareike Thies, Siyuan Mei, Paula Andrea Pérez-Toro, Siming Bayer, Andreas Maier
Published: 2026-07-10
Categories: eess.IV, cs.CV
arXiv: https://arxiv.org/abs/2607.09828v1

The differentiable shift-variant filtered backprojection (SV-FBP) framework enables data-driven estimation of redundancy weights for cone-beam CT reconstruction under general source trajectories, removing the need for analytically derived weighting schemes. In this work, we present a systematic study of the robustness and adaptability of differentiable SV-FBP under challenging acquisition settings. We show that the framework remains stable across highly irregular and discontinuous trajectories, indicating that reconstruction performance is largely insensitive to trajectory ordering or continuity. Instead, the spatial distribution of sampling points plays a more dominant role. Under sparse-view conditions, differentiable SV-FBP achieves competitive reconstruction quality while providing an order-of-magnitude reduction in computation time compared to iterative reconstruction methods at moderate sampling densities. However, we identify a clear transition regime under severe undersampling, where the absence of iterative data consistency leads to performance degradation. Furthermore, we demonstrate that the framework remains applicable to non-planar multi-isocenter geometries, such as Lissajous-saddle trajectories, without requiring architectural modifications. These findings provide new insights into the behavior and limitations of the differentiable SV-FBP model and highlight it as a flexible and efficient solution for non-standard and robotic CBCT acquisition scenarios.

---

### TDSal: Task-Based Top-Down Saliency Prediction Model

Authors: Can Mizrakli, Tolga K. Capin
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09827v1

Visual saliency aims to predict the regions of an image most likely to attract human visual attention. While most saliency models assume free-viewing conditions, human attention is often shaped by explicit task goals. In this work, we address task-driven saliency prediction by proposing a model that conditions visual attention on natural-language task descriptions. The model produces task-dependent saliency maps that reflect how attention shifts under different viewing intents. Through quantitative and qualitative analysis, we show that incorporating explicit task semantics enables more faithful modeling of goal-directed visual attention.

---

### Towards Objective Dysgraphia Detection: A Multi-Branch Deep Learning Approach for Online Handwriting Analysis

Authors: Lydia Ouhib, Yassine Ouzar, Zoé Pinseel, Stéphane Bouilland, Mehdi Ammi
Published: 2026-07-10
Categories: cs.CV, cs.AI, eess.SP
arXiv: https://arxiv.org/abs/2607.09826v1

Dysgraphia is a specific learning disability that is prevalent among school-age children. It affects handwriting coherence, quality, fluency, and legibility, often hindering academic achievement and early learning development. This motor coordination disorder is typically diagnosed through subjective assessments based on clinician observation, which can be timeconsuming and prone to variability. In this paper, we introduce a deep learning-based framework for objective dysgraphia detection using online handwriting data captured via digitizing tablets. The proposed framework relies on two complementary branches: the first pipeline extracts both handcrafted and embedding-based kinematic features directly from raw temporal signals, while the second leverages image-based representations of the temporal signals generated using continuous wavelet transforms (CWT) and Gramian Angular Fields (GAF). The resulting features are then fused to leverage the complementary strengths of both representations. The four representations were evaluated separately and jointly using the publicly available DiaGraMo dataset, showing that the fusion of GAF, MOMENT, and hand-crafted kinematic features outperforms each individual representation, as well as other fusion schemes. These findings highlight the potential of the complementarity of image and signal based representations for more objective dysgraphia detection.

---

### More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning

Authors: Yi Li, Alexandre Chapin, Liming Chen, Jan Peters, Alap Kshirsagar
Published: 2026-07-10
Categories: cs.RO, cs.AI
arXiv: https://arxiv.org/abs/2607.09825v1

Robotic manipulation policies rely on pre-trained vision models that give either a global scene embedding or a dense patch grid. Both mix task-relevant and task-irrelevant features. Object-centric slot representations are a structured alternative: they group features into a few per-object slots. We test what this structure buys on ManiSkill3 PickCube-v1, with a frozen encoder and a held-out-seed evaluation. Holding the policy, goal token, rendering, and calibration fixed and changing only the encoder, a frozen object-centric SPOT representation (DINO ViT-B/16 + Slot Attention) reaches 55.0$\pm$2.9% success, 22.4% above a dense DINO global-feature baseline (32.6 $\pm$ 1.5%), with the same trainable policy and no encoder fine-tuning. More tokens alone do not help: a dense patch grid with 16x the tokens performs no better than the global feature. Adding an explicit 2D spatial goal and native-resolution rendering raises the full system to 68.7$\pm$4.2%, just below a privileged 3D-oracle upper bound (71.7$\pm$4.1%). An automated kinematic failure taxonomy then separates spatial-precision (Near-Miss) failures from object-tracking (No-Grasp) failures: spatial grounding reduces Near-Miss while leaving No- Grasp unchanged. The same taxonomy transfers to the harder StackCube-v1 and points to occlusion as the main bottleneck.

---

### Memory-Conditioned Tool Calling for Camera-First Visual Agents

Authors: Xiaofan Wu, Xi Zeng, Miaoxia Chen, Peishan Chen, Shuyan Li, Jiyun Yao, Hanyong Zhong, Jiahao Zhu
Published: 2026-07-10
Categories: cs.CV, cs.AI, cs.CL, cs.HC, cs.IR
arXiv: https://arxiv.org/abs/2607.09822v1

Recognition tells an agent what is in an image; personal memory affects what is worth looking up next. In a camera-first setting the user can send only an image, so the agent must form the lookups. We study whether personal visual memory improves agent-side tool choice and tool arguments, and thereby more user-aligned multi-tool lookups. The design uses a three-layer personal visual memory (profile, short-term focus, observations) that is loaded on each turn to condition an LLM tool-calling loop under camera-first intake, and includes conflict-aware write-back intended to refresh the user model for later captures. On 800 images paired with synthetic memory blocks constructed for controlled ablation, removing the full three-layer memory block reduces tool-query relevance by 0.47 points absolute (4.21 -&gt; 3.74 on a 5-point scale; 11.2% relative) and end-to-end utility by 0.082 absolute (0.842 -&gt; 0.760; 9.7% relative). These results measure memory conditioning of tool policy under image-only intake with fixed synthetic blocks, not multi-session write-back from live user histories.

---

### Performance Benchmarking and Optimisation of Clustering Algorithms for Local and Non-Local Similarity Measure in Medical Image Analysis

Authors: Sisipho Hamlomo, Marcellin Atemkeng
Published: 2026-07-10
Categories: eess.IV, cs.CV
arXiv: https://arxiv.org/abs/2607.09821v1

Medical imaging generates high-resolution images posing significant storage, transmission, and computational challenges. While low-rank matrix approximation (LoRMA) techniques offer efficient compression by exploiting structural redundancy, global approaches often fail to preserve local details critical for diagnosis. This paper focuses on clustering techniques that exploit non-local self-similarity to identify structurally similar regions in medical images. These clusters can be used for post-processing tasks such as adaptive image compression. We evaluate five clustering techniques: k-means, mini-batch k-means, agglomerative hierarchical clustering, balanced iterative reducing and clustering using hierarchies (BIRCH), and bisecting k-means across MRI, ultrasound, and chest X-ray modalities. All clustering techniques were optimised using random search, and cluster quality was assessed using the Silhouette score, the Davies-Bouldin (DB) index, and the Calinski-Harabasz (CH) index. Results demonstrate that standard k-means and bisecting k-means generally achieve strong cluster cohesion and separation across modalities. However, they tend to form a small number of clusters with high intra-cluster variability, limiting their effectiveness for post-processing tasks such as adaptive compression. Agglomerative clustering outperformed other techniques for MRI and ultrasound in terms of intra-cluster homogeneity, making it more suitable for preserving fine diagnostic details. For chest X-rays, mini-batch k-means achieved the best balance between clustering quality and intra-cluster compactness. BIRCH consistently underperformed across all modalities.

---

### Learning Predictive Ambiguity Sets for Decision-Focused Distributionally Robust Optimization

Authors: Junjie Guo
Published: 2026-07-10
Categories: cs.LG, q-fin.CP
arXiv: https://arxiv.org/abs/2607.09820v1

Predict-then-optimize systems usually compress uncertainty into a point forecast and then solve a downstream optimization problem as if the forecast were reliable. Distributionally robust optimization (DRO) offers protection against misspecification, but the ambiguity set is often centered at historical samples and uses a fixed radius. We propose \emph{learned predictive ambiguity sets} (LPAS): a deep contextual model outputs a finite nominal scenario distribution, a state-dependent Wasserstein radius, and optionally an anisotropic ground metric. These outputs define a contextual ambiguity set that feeds a DRO decision layer. The radius is trained by a combination of conditional quantile calibration, size regularization, and downstream decision loss, so that robustness is adaptive rather than globally fixed. We derive the finite dual form used by the decision layer, present a staged training algorithm, and evaluate the method on distributionally robust portfolio optimization with 20 S&amp;P 500 constituents from 2018--2026. The proposed method substantially improves over equal-weight, predict-then-optimize, and historical Wasserstein DRO baselines, achieving 26.28% annualized return, Sharpe ratio 1.30, final wealth 1.61, and lower tail loss than a deep fixed-radius DRO baseline while using a smaller average radius. The results show that learned ambiguity radii can recover most of the performance of strong fixed-radius DRO while reducing unnecessary conservatism and improving regime adaptivity.

---

### TS-Mask VLA: 2D Temporal-Spatial Masking for Vision-Language-Action Model with Effective Bridging

Authors: Shengzhuo Yang, Ronghao Yu, Chuanjie Lv, Linpeng Peng, Hang Yu, Jie Ren, Jiajun Lv, Yong Liu
Published: 2026-07-10
Categories: cs.RO, cs.AI, cs.CV
arXiv: https://arxiv.org/abs/2607.09818v1

Vision-language-action (VLA) models aim to understand natural-language instructions and visual observations, and to generate and execute corresponding actions as embodied agents. Recently, autoregressive token-based action generation has driven the development of many representative VLA models. However, this paradigm often reduces action generation to next-token prediction, thereby lacking explicit modeling of the spatiotemporal structure of action sequences and the disentanglement between vision-language representations and actions, which can limit performance in long-horizon and complex scenarios. In this paper, we propose TS-Mask VLA, a vision-language-action framework for robot manipulation. TS-Mask VLA is built upon two key designs: (1) a Discrete Diffusion Action Expert equipped with a Bridge Attention conditioning bridge, which enables multi-layer conditioning from the VLM and facilitates more accurate and stable action generation; and (2) a temporal-spatial 2D masking strategy for discrete action tokens that strengthens the model's understanding of cross-time dependencies and inter-dimensional coupling, leading to more structurally consistent action sequences. We conduct extensive experiments on simulation benchmarks and real-world tasks. On LIBERO, TS-Mask VLA achieves a 95.7 percent average success rate with only 0.5B parameters, outperforming significantly larger models. On CALVIN, it attains the best average sequence length of 4.19 and strong long-horizon performance. Comprehensive analyses and ablations further validate the effectiveness of our design.

---

### Estimation, Prediction, and Assortment Optimization for Markov Chain Choice Models with Panel Data

Authors: Yalcin Akcay, Gerardo Berbeglia, Young-San Lin
Published: 2026-07-10
Categories: cs.LG, stat.ML
arXiv: https://arxiv.org/abs/2607.09817v1

We propose a framework for the Markov chain (MC) choice model with panel data, including parameter estimation, personalized choice prediction, and personalized assortment optimization. In contrast to the traditional setting, which assumes that each transaction is independently drawn from a random utility model, our framework accounts for dependencies among transactions for the same customer in historical data, captured by partial-ordering preference information. To the best of our knowledge, our framework initiates the study of choice modeling with panel data under MC. As our primary result, we propose novel expectation-maximization (EM) algorithms for MC parameter estimation by incorporating partial-ordering-based customer preference information. On synthetic datasets and the sushi dataset, our EM algorithms outperform the traditional EM algorithm of Simsek and Topaloglu (Operations Research, 66, 2018) and multinomial-logit-based partial-order benchmarks adapted from Jagabathula and Vulcano (Management Science, 64, 2018). As our secondary contribution, we present hardness and computational results for conditional choice prediction and assortment optimization problems. These results complement our estimation framework and clarify the computational landscape of conditional choice and assortment optimization, which may be of independent interest.

---

### RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification

Authors: Yanxuan Yu, Dong Liu, Shu Wang, Wenxiao Zhao, Eric Jiang, Chang Liu, Jinxi Yu, Hui Pan, Ben Lengerich, Tong Geng, Renata Borovica-Gajic, Ying Nian Wu
Published: 2026-07-10
Categories: cs.LG
arXiv: https://arxiv.org/abs/2607.09816v3

Class imbalance poses a fundamental challenge in risk-sensitive applications such as fraud detection and medical diagnosis, where minority-class samples are scarce yet critical for accurate classification. Existing oversampling methods generate synthetic samples to rebalance class distributions; however, they often produce large numbers of low-quality candidates that distort decision boundaries or introduce artifacts, leading to overfitting and degraded generalization. In this work, we introduce RUBRIC, a generator-agnostic filtering framework that formulates synthetic sample selection as a quality-over-quantity optimization problem. RUBRIC ranks candidates using a realism-utility trade-off: realism is quantified by a learned discriminator that distinguishes real samples from synthetic samples, while utility captures proximity to the decision boundary through a concave margin-based scoring function. We show that, under mild regularity conditions, the proposed filtering strategy monotonically tightens the generalization bound for margin-based classifiers by jointly reducing distribution shift and suppressing near-negative tail contributions. Through extensive experiments on credit-card fraud detection and other imbalanced benchmarks, we demonstrate that RUBRIC improves F1-macro and recall while maintaining comparable ROC-AUC across several generators. We also provide explicit lambda-sensitivity analysis to show how users can recover AUPRC when ranking quality is prioritized.

---

### RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification

Authors: Yanxuan Yu, Dong Liu, Eric Jiang, Shu Wang, Wenxiao Zhao, Jinxi Yu, Shaoyi Lu, Hui Pan, Renata Borovica-Gajic, Ying Nian Wu
Published: 2026-07-10
Categories: cs.LG
arXiv: https://arxiv.org/abs/2607.09816v2

Class imbalance poses a fundamental challenge in risk-sensitive applications such as fraud detection and medical diagnosis, where minority-class samples are scarce yet critical for accurate classification. Existing oversampling methods generate synthetic samples to rebalance class distributions; however, they often produce large numbers of low-quality candidates that distort decision boundaries or introduce artifacts, leading to overfitting and degraded generalization. In this work, we introduce RUBRIC, a generator-agnostic filtering framework that formulates synthetic sample selection as a quality-over-quantity optimization problem. RUBRIC ranks candidates using a realism-utility trade-off: realism is quantified by a learned discriminator that distinguishes real samples from synthetic samples, while utility captures proximity to the decision boundary through a concave margin-based scoring function. We show that, under mild regularity conditions, the proposed filtering strategy monotonically tightens the generalization bound for margin-based classifiers by jointly reducing distribution shift and suppressing near-negative tail contributions. Through extensive experiments on credit-card fraud detection and other imbalanced benchmarks, we demonstrate that RUBRIC improves F1-macro and recall while maintaining comparable ROC-AUC across several generators. We also provide explicit lambda-sensitivity analysis to show how users can recover AUPRC when ranking quality is prioritized.

---

### RASR: Range-Aware Scale Recovery for Metric UAV Navigation

Authors: Hongtao Liang, Xinyu Shao, Chenxu Wang, Yiyao Wan, Jiahuan Ji, Fangwei Ye, Fuhui Zhou, Qihui Wu
Published: 2026-07-10
Categories: cs.RO, cs.CV
arXiv: https://arxiv.org/abs/2607.09815v2

A central challenge in image-goal UAV navigation under Global Navigation Satellite System (GNSS) denial is estimating metric distance and heading between current and goal views. Dense pairwise geometry models capture relative scene structure, but without a calibrated metric scale, they cannot directly provide reliable distance estimates for navigation. Although global scale calibration corrects the dominant scale bias, the remaining errors vary systematically with distance. In this paper, Range-Aware Scale Recovery (RASR) is proposed, which complements global scale calibration with range-aware residual correction. RASR encodes pairwise geometry extracted by a frozen Matching And Stereo 3D Reconstruction (MASt3R) backbone as a compact descriptor and separates the scale-recovery core from task-specific command calibration. On the official online evaluation of the UAVs in Multimedia 2026 PairUAV challenge, RASR achieved a total error of 0.003189, achieving a lower total error than global scale calibration alone. The results demonstrate that range-aware residual correction improves metric distance estimation beyond global scale calibration. Code and materials are available at https://github.com/lht-research/rasr-pairuav.

---

### CHM-Net: Center Heatmap-driven Macro-Micro Modeling Network for MRI-based Microbial Density Stratification

Authors: Jiaming Liang, Haolin Chen, Tingting Li, Bowen Yu, Qianyan Long, Tinghe Zhang, Xi Zhong, Xiaowei Hu, Xiaoqi Sheng, Hongmin Cai
Published: 2026-07-10
Categories: eess.IV, cs.CV, cs.LG
arXiv: https://arxiv.org/abs/2607.09812v2

Microbial density is clinically important for tumor assessment and treatment decision-making, and recent advances in deep learning suggest that it can be non-invasively inferred from multimodal MRI. In this work, MRI-based Microbial Density Stratification (MRI-MDS) is first investigated as a patient-level representation learning task, and Center Heatmap-driven Macro-micro modeling Network (CHM-Net) is introduced for this task. CHM-Net first establishes the link between imaging phenotypes and microbial states through center heatmap-guided small-lesion response localization. Building upon this, it constructs patient-level macro-micro evidence from localized heatmap responses for microbial density prediction. Experiments on the novel GBNPC 2026 dataset constructed for MRI-MDS demonstrate the effectiveness of CHM-Net, achieving superior performance over representative baselines with a 12.06% absolute ACC gain over the strongest competing result. Additionally, auxiliary validation on two 3D medical image datasets further verifies its robustness across volumetric medical image classification scenarios. The project is available at https://anonymous.4open.science/r/CHM-Net-942E/.

---

### CHM-Net: Center Heatmap-driven Macro-Micro Modeling Network for MRI-based Microbial Density Stratification

Authors: Jiaming Liang, Haolin Chen, Tingting Li, Bowen Yu, Qianyan Long, Tinghe Zhang, Xi Zhong, Xiaowei Hu, Xiaoqi Sheng, Hongmin Cai
Published: 2026-07-10
Categories: eess.IV, cs.CV, cs.LG
arXiv: https://arxiv.org/abs/2607.09812v1

Microbial density is clinically important for tumor assessment and treatment decision-making, and recent advances in deep learning suggest that it can be non-invasively inferred from multimodal MRI. In this work, MRI-based Microbial Density Stratification (MRI-MDS) is first investigated as a patient-level representation learning task, and Center Heatmap-driven Macro-micro modeling Network (CHM-Net) is introduced for this task. CHM-Net first establishes the link between imaging phenotypes and microbial states through center heatmap-guided small-lesion response localization. Building upon this, it constructs patient-level macro-micro evidence from localized heatmap responses for microbial density prediction. Experiments on the novel GBNPC 2026 dataset constructed for MRI-MDS demonstrate the effectiveness of CHM-Net, achieving superior performance over representative baselines with a 12.06% absolute ACC gain over the strongest competing result. Additionally, auxiliary validation on two 3D medical image datasets further verifies its robustness across volumetric medical image classification scenarios. The project is available at https://anonymous.4open.science/r/CHM-Net-942E/.

---

### Detangled: A Framework for Creating, Editing, and Inferencing Feature Rich Hair Strands

Authors: Sarah Jobalia, Yitong Deng, Carolyn Smith, Ronald Fedkiw
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09811v1

We present a framework for understanding and generating feature rich hair strands. Drawing upon both scientific and cultural expertise, we define strand texture as the various distinctive patterns (curling, switchbacks, twist, etc.) that are formed by forces internal to a hair strand. We begin by proposing a novel five-dimensional parameter space, intended to be a bijection with naturally occurring hair strand textures. This encoding is both qualitatively accessible, allowing users to readily locate their own hair in the parameter space, and quantitatively precise, allowing the generation of individual strands from texture inputs. Importantly, strand texture should be independent from the overall strand direction. In order to disentangle strand texture from the overall strand direction, we identify centerline geometry and use it to map strands into a canonical space (a strand texture space). We construct centerlines using a novel method that cleanly distills complex hair grooms, separating the strand texture from the overall style (parameterized by style guides). We enable the creation of new strands conforming to our parametric description of texture via a generative artificial intelligence approach supervised by a separate neural network trained to label candidate strands according to our five-parameter description. The ability to create new strands conforming to any desired texture enables groom editing using either texture transfer or user-provided inputs. We demonstrate results on a variety of hair types.

---

### PHINN-EEG: Topological Time-Series Analysis of Dream-State EEG -- Dynamic Betti Curves for Dream Content Classification and Topology-Conditioned Neural Signal Synthesis

Authors: Ren Takahashi, Emre Yusuf, Jayabrata Bhaduri
Published: 2026-07-10
Categories: q-bio.NC, cs.AI, cs.LG, eess.SP, math.AT
arXiv: https://arxiv.org/abs/2607.09662v1

Current electroencephalography (EEG)-based dream detection relies on power spectral density (PSD) and statistical moment features, achieving a state-of-the-art area under the receiver operating characteristic curve (AUC) of approximately 0.70 on the DREAM database (Wong et al., 2025, Nature Communications). We introduce PHINN-EEG (Persistent Homology Inspired Neural Network for EEG), the first topological time-series framework for dream mentation analysis. Using sliding-window Takens delay embeddings and Vietoris-Rips filtrations on multichannel pre-awakening EEG epochs, we extract Dynamic Betti Curves that characterize the geometric architecture of neural activity, not merely its energy. These topological invariants, combined with topology-conditioned flow matching, are analytically projected to outperform existing PSD and catch22 benchmarks, targeting AUC = 0.82-0.90 on the 1,462-awakening open-access subset of the DREAM database (drawn from a full registry of 3,191 total awakenings from 263 participants across 20 independent laboratories). We further introduce a topology-conditioned rectified flow model for dream-state EEG synthesis-with a spectral-conditioned flow model of comparable feature dimensionality as an additional ablation baseline to isolate the value of topological conditioning specifically-and propose a set of candidate Betti transition archetypes linking topology to phenomenological dream report categories, presented as an exploratory hypothesis space pending empirical validation. If validated, this work represents a paradigm shift from spectral energy to phase-space geometry in neural rare-event detection, with potential future implications for wearable BCI dream monitoring.

---

### PanoWorld: Real-World Panoramic Generation

Authors: Haoyuan Li, Dizhe Zhang, Yuemei Zhou, Xiangkai Zhang, Haoran Feng, Xiaofan Lin, Wenjie Jiang, Bo Du, Ming-Hsuan Yang, Lu Qi
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09661v1

In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and dataset will be publicly available. More information can be found on our project page: https://lihaoy-ux.github.io/panoworld-page/.

---

### Scalable Visual Pretraining for Language Intelligence

Authors: Yiming Zhang, Zhonghan Zhao, Wenwei Zhang, Haiteng Zhao, Tianyang Lin, Huanze Tang, Yunhua Zhou, Demin Song, Kuikun Liu, Haochen Ye, Haian Huang, Yuzhe Gu et al.
Published: 2026-07-10
Categories: cs.CV, cs.AI, cs.MM
arXiv: https://arxiv.org/abs/2607.09657v2

The rapid progress of large foundation models has been driven predominantly by pretraining on large-scale text corpora. However, many forms of knowledge are conveyed through visual representations, where figures, typeset equations, and page layouts carry rich information that cannot be faithfully or completely captured by text alone. Yet current pretraining approaches discard these visual cues by converting visually rich sources, such as documents and web pages, into plain text for learning language intelligence. This paper challenges the default assumption that language models must be trained on text-only representations and shows that Visual Pretraining is a scalable learner for foundation model intelligence. To this end, we conduct a systematic study of unsupervised visual pretraining paradigms that directly leverage visual documents without text extraction. Across multiple backbones and benchmarks, visual pretraining on the same underlying corpora consistently outperforms text-only pretraining, offering an efficient pathway to scalable language intelligence.

---

### OpenLongTail: Generative Scaling of Long-Tail Driving Data

Authors: Lulin Liu, Nuo Chen, Yan Wang, Bangya Liu, Wenyan Cong, Hezhen Hu, Boris Ivanovic, Hao Wang, Ziyao Zeng, Xinyu Gong, Yang Zhou, Zixiang Xiong et al.
Published: 2026-07-10
Categories: cs.CV
arXiv: https://arxiv.org/abs/2607.09655v1

Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.

---

### Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models

Authors: Shravan Murlidaran, Miguel P. Eckstein
Published: 2026-07-10
Categories: cs.CV, cs.AI
arXiv: https://arxiv.org/abs/2607.09654v1

Vision language models (VLMs) have made remarkable progress in visual reasoning during the last decade. Most evaluations have used simple scenes (MS-COCO) that do not showcase complex human interactions or behaviors, only a handful of non-curated human descriptions as a benchmark, and have not focused on understanding the model's error types. Here, we introduce the Complex Social Behavior (CSB) dataset, containing 100 images depicting complex social interactions/behaviors. We analyze the progression of scene descriptions over a decade (2017-2025) of VLMs (four pre-Multimodal Large Language Models, MLLMs, and five MLLMs). We evaluate the accuracy of the models and 20 human descriptions relative to a gold standard on the CSB dataset and on a sample from MS-COCO. We analyzed five visual-cognitive error types: object detection, recognition, hallucination, scene understanding, and spatial dependence. The CSB dataset showed a more pronounced improvement than MS-COCO in scene description accuracy, with pre-MLLMs achieving much lower accuracy than the bottom-ranked human descriptions and MLLMs attaining accuracies similar to the top-ranked human descriptions. We show that MLLMs have eliminated the gap in scene description accuracy between simpler MS-COCO scenes and scenes depicting complex behaviors (CSB). MLLMs have almost eliminated all error types in our tested datasets, except for occasionally relying on different image regions for scene descriptions than humans do (spatial dependence error). We also show that detection, recognition, and hallucination errors have the highest impact on scene description accuracy. Together, our findings provide a more thorough evaluation of how visual language models have advanced over the last decade.

---

