All papers

The paper introduces a dual-process memory framework coordinating small and large models for efficient long-context reasoning.

This paper introduces an on-policy residual learning framework with an entropy-contraction objective to improve generation speed and quality in diffusion language models.

The paper proposes a policy optimization method that treats reasoning transitions as learned coordinates in an energy space to internalize self-correction.

Derives token-level credit from single rollouts using hindsight future context rather than sampling multiple attempts per prompt.

The paper learns trajectory embeddings to predict the success or failure of language model agents early during execution.

The paper proposes weighting majority voting responses using a length-conditioned prior adapted at test time without model fine-tuning.

This paper investigates using Classical Chinese as a compact intermediate reasoning register to reduce token length in Chinese language models.

The paper presents a systematic framework for selecting optimal parallelism strategies to balance latency and throughput during distributed language model serving.

The paper presents a method to retrofit attention models with hierarchical test-time neural memory modules to improve long-context inference efficiency.

This paper proposes SCOUT, a cascaded two-track framework combining visual candidate filtering, offline semantic summaries, and query refinement for document retrieval.

This paper proposes a joint scaling law evaluating how adding experts in multilingual mixture-of-experts models affects language capacity and loss penalties.

The paper introduces a CPU front-end tokenizer that encodes only session deltas to speed up multi-turn inference.

The paper introduces a unified routing framework that learns both execution mode selection and model assignment across diverse tasks.

The paper presents a method using an external encoder to induce latent soft-token reasoning on frozen language model backbones.

This paper develops a self-supervised representation learning approach based on pointwise mutual information factorisation for scalable generative latent tokenization.

This paper introduces a two-stage priming framework combining reconstruction pretraining with distributional post-training for one-step generative modeling.

Presents a serving system that dynamically manages GPU memory and prefetches weights for multi-model agent workflows.

The authors propose Subspace Tuning, a fine-tuning method restricting weight updates to bottom singular subspaces to mitigate catastrophic forgetting in language models.

The paper presents a closed-form, training-free approach to merge specialized fine-tuned models into a single student using forward passes.

The paper compares the Muon optimizer against AdamW across controlled data, wall-clock time, and tuning budgets.

The paper introduces a system that retrieves document information into a fixed set of prompt embeddings for frozen language models.

The paper presents an agent orchestration framework that trains smaller language models to navigate, inspect, and reason over long multimodal documents.

Presents a framework that decouples speech generation and synthesis token spaces to speed up spoken dialogue model inference.

Investigates cost-aware policies for adaptively switching between language models of varying fidelity in stochastic optimization pipelines.

The paper trains a small language model to generate and refine GPU kernel optimization scripts using compiler traces.

Introduces a compression-based patching strategy for byte-level language models that allocates compute where upcoming bytes are unlikely to repeat.

The paper introduces a confidence-calibrated controller to optimize inference compute and test-time expansion decisions in reasoning models.

The paper introduces a hybrid generative-tagging architecture to improve efficiency and coverage in product attribute and value extraction.

The paper presents PocketTSLM, deploying an on-device soft-prompt time-series language model on smartphones for personal health data.

The paper proposes RoPERelay to reorganize positional geometry and rotary embeddings after sparse KV cache token eviction.

Proposes a compressed representation using Gaussian trellises to quantize conditional memory lookup tables down to two bits.

Compares abstract-only versus full-text inputs and few-shot versus fine-tuning approaches for generating biomedical lay summaries.

Shows across 200 models that proposed intrinsic tokenizer metrics do not reliably predict downstream language model performance.

Introduces a hybrid speculative decoding framework splitting draft tasks between mobile neural processing units and cloud servers.

Analyzes how calibration language choices affect block-influence depth pruning rankings in multilingual language models.

Proposes a framework that increases image diffusion throughput by fusing and processing multiple generation samples simultaneously in manifold space.

This paper explores internal-state sharing in multi-agent language models through a central pool rather than pairwise transfer bridges.

The paper presents a replay-driven decoupled reinforcement learning framework to improve system efficiency during language model post-training.

The paper introduces decision models that balance fast single-pass predictions with adaptive reasoning for uncertain cases using reinforcement learning.

This paper presents a distillation method using frozen language models to guide molecular flow generators toward natural language discovery goals.

The paper proposes adding an explicit copy-paste prediction head to language models to improve in-context retrieval and copying efficiency.

The paper personalizes mixture-of-experts models and analyzes which specialized experts encode distinct user-specific features.

The paper proposes data-space iteration, a few-step generative method removing flow discretization by directly refining noise predictions in data space.

Introduces an information-theoretic framework to reweight supervised fine-tuning data based on conditional information gain.

Investigates on-policy distillation for multi-turn agents by decoupling objective interaction states from internal student reasoning trajectories.

Evaluates static Transformer block skipping in time-series foundation models to reduce inference latency with minimal forecasting degradation.

Presents deployment experiences and design strategies for ByteExpress, an RDMA communication layer for distributed training and inference workloads.

Reports that initialization-time gradient masks underperform random masks on catastrophic forgetting when confined to the same LoRA parameter pool.

This paper presents a method for upcycling pretrained language models into sparse architectures using token-indexed embeddings in residual pathways.

This paper proposes a sample-adaptive zeroth-order optimization method that estimates sample importance online to accelerate large language model fine-tuning.

Proposes a market-level mixture-of-experts coordination framework using cost-aware gating and revenue allocation across heterogeneous model services.

The paper proposes confining feature distillation loss to language-routed adapters to balance performance in compact multilingual speech recognition models.

The paper studies gate learning in recurrent networks, showing how adding short-dependency samples helps models learn long-range temporal connections.

The paper introduces a benchmark to evaluate how typed decision models handle theory-of-mind reasoning tasks compared to standard generative language models.

Introduces lexical reference frames to interpret intermediate hidden states in language models using the model's own remaining layers.

The authors introduce a 2.5-billion-parameter bilingual Tamil-English language model trained from scratch with tool-calling capabilities.

The paper proposes a knowledge distillation method that transfers text language model capabilities to speech-to-speech models using internal text interfaces.

The paper introduces a lightweight guidance model to estimate parallel decoding inconsistency and accelerate diffusion language models.

The paper analyzes geometric properties of text embeddings to predict how many nearest neighbors an individual example loses under dimensionality reduction.

This paper studies cross-entropy surprisal metrics, showing that predictability rankings of texts remain consistent across multiple small language models.