All papers

The paper analyzes model swapping in multi-session agents and proposes a history summarization method to preserve strong reasoning behaviors.

Documents safety alignment methods, negative preference tuning outcomes, and automated judge limitations for a Portuguese language model.

The paper uses intermediate transformer hidden states instead of semantic embeddings to improve classifier-based prompt routing across unseen tasks.

This paper introduces plug-in decision heads that read action choices directly from visual agent hidden states rather than generating text tokens.

The paper shows pruned reasoning models fail by looping, proposing an activation steering method to restore proper termination.

The paper analyzes retrieval failures in Mamba architectures under query-first settings and explores token representations and selection gates.

Benchmarks the inference energy efficiency of diffusion language models compared to standard autoregressive architectures.

Proposes a structural scaling method based on transformer unit roles to convert language models to ternary weights effectively.

Proposes a trajectory-aware retrieval method that indexes tool sequences in a trie to cut token costs and improve selection accuracy.

Introduces a benchmark evaluating single-forward-pass discrete decision models versus generative reasoning models across diverse closed-loop tasks.

The paper presents a decision model that performs deeper reasoning by looping its layers instead of generating output tokens.

The paper introduces an encoder architecture that processes input once to answer multiple structured classification queries simultaneously.

Introduces a method to filter unsupported compressor inferences from long-horizon agent summaries to avoid compounding execution errors.

The paper introduces a compact vision-language-action model using register tokens as persistent memory to reduce control oscillations in robotics.

Evaluates on-policy distillation versus supervised fine-tuning in vision-language models, analyzing how teacher reliability impacts student behavior.

Shares hardware-model co-design techniques and analytical kernel selection for training large recommendation models on AMD GPUs.

This paper investigates direct key-value cache substitution between frozen language models and finds geometric similarity fails to ensure consumability.

The study investigates whether removing small subsets of attention heads can improve language model performance on specific tasks.

This paper demonstrates that frozen short-context drafting models can effectively serve ultra-long context speculative decoding without fine-tuning on long sequences.

Gates reinforcement learning policy-shift distillation using intermediate checkpoints to ensure student models only learn consistent teacher shifts.

Uses teacher critiques on incorrect student rollouts to improve on-policy distillation and recover supervision during mathematical reasoning tasks.

The paper proposes ReMaRk, a reinforcement learning method that trains recommender models using all available reader replies simultaneously to reduce variance.

Allocates computation unevenly across passes in looped mixture-of-experts models to improve efficiency without losing performance.

Introduces an asynchronous prefix denoising framework for extreme latent compression in diffusion models.

Proposes a synthetic self-distillation method using a slightly larger aligner to enable continual post-training in small language models.

This paper proposes verbal regularization prompts for on-policy self-distillation to prevent teacher models from taking shortcut reasoning paths.

Proposes Simplified Matching Distillation, a regression-based on-policy method that enables four-step generation for diffusion models.

Introduces a lightweight correction mechanism to restore token dependencies during parallel draft generation in speculative decoding.

The paper proposes an agent architecture that dynamically switches between latent reasoning and text interaction while maintaining external tool access.

Proposes matrix-entropy and kernel alignment criteria to select optimal layer subsets of hidden states for speculative decoding.

Formulates relative policy difference transfer as an implicit reward in KL-regularized optimization for flexible on-policy knowledge distillation.

Presents a training method for latent recurrent language models that avoids performance penalties at single loops while scaling with compute.

The paper introduces a token-level sparse attention method using a learned query-key subspace and an indexed kernel to accelerate long-context decoding.

This paper investigates gradient spectral properties during reinforcement learning with verifiable rewards to design an adaptive-rank optimizer for reasoning models.

The paper presents an expressive and distributed attention kernel that compresses structured attention masks to enable efficient context parallelism on Hopper GPUs.

This paper introduces COUNT, scaling language model memory without training parameters by storing precomputed n-gram counts directly in host memory.

Compresses long-context key-value caches using test-time training on generic document prompts without task-specific labels.

The paper compresses video history into evolving lightweight model weights instead of competing for limited input context length.

The paper proposes a training-free structured pruning method that trims multilayer perceptron layers while preserving attention mechanisms for reasoning tasks.

The paper introduces a training-free contrastive decoding method where a small model guides a larger model toward shorter reasoning chains.

This paper investigates batch size, regularization methods, and runtime tradeoffs when pretraining language models on repeated finite data.

Introduces an elastic transformer pretraining architecture enabling simultaneous hyperparameter transfer and subnetwork execution within a single model.

The paper proposes a method to translate reverse reasoning paths into verified forward chains of thought for distillation.

Proposes a policy mechanism enabling language models to decide when, how, and which test-time training procedures to execute.

Examines how chronological versus shuffled pre-training knowledge sequencing influences the sample efficiency of subsequent model knowledge updates.

Compresses tabular in-context learning key-value caches into query-calibrated prototypes to improve throughput without sacrificing accuracy.

Introduces a spectral weight decay update inducing low-rank weight matrices to improve SVD-based post-training model compression.

The paper presents a profiling-guided framework to allocate precision budgets adaptively across layers and tokens during KV cache quantization.

The paper measures the energy consumption, latency, and thermal behavior of running vision-language models on edge hardware accelerators.

The paper presents an elastic vision architecture that dynamically reconfigures width and supports partitionable inference across heterogeneous edge devices.

Designs a vision transformer using a single recurrent block with depth-programmed experts to reduce stored parameters without accuracy loss.

The authors propose decoupled positional encodings enabling full-duplex speech models to run internal thinking streams at hardware-adaptive speeds.

The paper examines positive activation bias in neural networks, showing that reducing this bias slows initial training but improves final model performance.

The paper proposes disaggregating teacher and student models across separate nodes with a fused loss to accelerate knowledge distillation.

The paper introduces an attention conversion method enabling multi-head latent attention on modern architectures with reduced memory footprints.

Analyzes test-time compute strategies, including parameter adaptation and prediction aggregation, to improve predictions in tabular foundation models.

The paper presents Diffuplex, a full-duplex spoken language model that separates speech timing from content generation using discrete diffusion.

This paper introduces a training-free decoding rule that anchors multi-token format units to ensure parseable parallel generation in diffusion language models.

This paper proposes RIFT, a method using electrical resistance analogies to trace token influence through transformer attention graphs.

This paper presents Hashrope, a unified persistent data structure supporting incremental edits, branching, and cache reuse for language model serving.