All papers

The paper benchmarks ten self-distillation methods in agentic reinforcement learning across varied model sizes, tasks, and computational costs.

The paper analyzes performance bottlenecks when using remote key-value caches during context-parallel prefill in large language model serving.

Demonstrates that knowledge distillation transfers safety dispositions and behavioral tendencies from teacher models to student models on unrelated tasks.

Analyzes tabular foundation models and shows most transformer layers can be replaced with affine translators without sacrificing performance.

The paper defines evaluation metrics to show that language models continue generating reasoning traces even when instructed not to think.

This paper analyzes pretraining factors such as masking ratios and token budgets to optimize bidirectional transformer encoders.

The paper extends attention residuals across model depth by introducing multi-stream depth attention with causal convolutions.

The paper introduces INT4 KV cache quantization with selective INT8 protection to accelerate asynchronous rollouts in reinforcement learning.

The paper proposes an edge-cloud collaboration framework that selectively delegates complex agent steps from local models to cloud models.

Develops small multimodal models that output structured choices using masked language modeling to reduce inference latency in agent tasks.

Proposes a training-free frame packing method enabling vision-language models to process long videos with constant memory consumption.

Introduces a framework where a small model drafts tool-use trajectories and a larger model verifies them in single prefill passes.

The paper examines projector training in multimodal models and proposes a training-free initialization method using summary statistics instead of paired image-caption data.

The paper proposes a training-free episodic memory mechanism that extends linear attention language models to long-context tasks.

Virtual Head Attention uses two linear operations to compress key-value cache overhead without increasing training compute or sacrificing output quality.

Proposes Diff Diffusion, a diffusion-based model that predicts localized code edit sequences rather than regenerating entire source files.

Presents a retraining-free method to adapt transformer sizes dynamically across diverse computational budgets using joint soft pruning of components.

Introduces Contrastive Vision On-policy Distillation to prevent vision-language models from relying on answer shortcuts rather than visual evidence.

Allocates precision across MoE experts by separately handling resident storage footprint and active batch decode traffic via dynamic optimization.

Proposes a conformal routing method that accounts for model failure dependencies to optimize multi-model query coverage under risk budgets.

The authors develop an error-compensated 4-bit all-reduce algorithm to reduce communication overhead during distributed model training and inference.

This paper evaluates open-weight language models against closed frontier models on a cybersecurity detection benchmark using an interactive telemetry environment.

The paper studies compute allocation across multi-agent coding teams to identify Pareto-optimal trade-offs between cost and performance.

This paper presents CacheFuse, a training-free technique enabling heterogeneous language models to communicate directly by sharing key-value cache states.

MedDEx combines language models for patient data extraction with deterministic rule engines to execute clinical treatment guidelines accurately and efficiently.

Introduces an unsupervised method to identify and remove subliminally transferred traits from model parameter updates during knowledge distillation.

Trains an unsupervised vision-language reward model using intermediate layer representations and caption perturbations without human preference labels.

Proposes an interpretability framework using sparse autoencoders to explain mixture of experts routing based on superposed fine-grained features.

Presents a hypernetwork that maps document contexts directly into low-rank adapter weights for large language models.

Implements open-vocabulary speech recognition on a low-cost ESP32-S3 microcontroller without dedicated neural accelerators.

Develops a GPU-accelerated tokenizer using CUDA graphs to reduce time-to-first-token in agentic language model serving.

Proposes a method to select a compact subset of procedural skills that provide effective training signals during model distillation.

Develops an open-source testbed to evaluate segment-wise key-value cache reuse beyond exact prompt prefixes during language model serving.

The paper shows that whole-context KV cache rehearsal harms compression under tight budgets, proposing targeted rehearsal to preserve answer-relevant entries.

The paper presents a repair method that re-encodes causal suffixes to enable safe non-prefix KV-cache reuse in retrieval-augmented generation.

This paper demonstrates that sentence-level relevance retrieval can match full conversation history accuracy in personalized language models using only five percent context.

The paper studies safety behavioral deviations caused by cross-model key-value cache reuse and proposes an alignment technique to mitigate them.

Introduces a low-rank adaptation method that splits rank budgets into stable and dynamic branches to capture residual gradients outside the tangent space.

The paper presents a theoretical framework using nesting properties to calculate distance bounds and compare language model truncation sampling algorithms efficiently.

The paper proposes mixed precision scaling laws to model the trade-offs of low-precision quantization during model pretraining.

This paper evaluates lightweight scheduling policies for continuous batching in large language model serving across different GPU architectures.

This paper presents a system that speeds up multi-model language model serving by translating and reusing attention maps across models.

This paper investigates error compounding and scale-state feedback mechanisms during recurrent-state quantization in linear-attention and state-space language models.

The paper introduces a progress-aware distillation method to prevent small language model agents from stalling in repetitive loops during long tasks.

Proposes an adaptive reasoning framework that supervises intermediate retrieval states and stops reasoning steps dynamically to cut latency.

This paper introduces a framework combining persistent belief states with implicit world modeling to help long-horizon language agents recover from failures.

The paper presents an automated coding agent that tunes large model training configurations for throughput under user-defined compute budgets.

The paper evaluates whether self-distillation using privileged information improves reasoning capabilities or reflects overfitting to training tasks.

The study shows that training on-policy distillation exclusively on student errors degrades accuracy rather than improving it.

The paper re-evaluates the pretraining compute scaling laws between diffusion and autoregressive language models using modern diffusion training designs.

The paper provides theoretical mean-square error guarantees and bounds for layer-wise quantization error compensation across linear and non-linear networks.

Introduces an adaptive multi-agent debate framework that uses a leader-supporter structure to trigger debate only when simple aggregation is insufficient.

Measures teacher entity familiarity to filter out uninformative search trajectories when distilling deep-search capabilities into smaller agents.

This paper introduces a method to translate key-value caches between different language models, enabling context handoffs without reprocessing input tokens.

This paper presents a lattice vector quantization method that compresses attention key-value caches to two bits while preserving model quality.

Studies how depth-based early-exit inference affects language model safety, finding shallow exits resist jailbreaks better than deep exits.

This paper analyzes how layer ordering and selection affect performance across tasks in hybrid attention model architectures.

The paper analyzes model swapping in multi-session agents and proposes a history summarization method to preserve strong reasoning behaviors.

Documents safety alignment methods, negative preference tuning outcomes, and automated judge limitations for a Portuguese language model.

The paper uses intermediate transformer hidden states instead of semantic embeddings to improve classifier-based prompt routing across unseen tasks.