All papers

This paper proposes a recurrent architecture that adds rotational degrees of freedom to prevent oscillatory bottlenecks in looped reasoning.

This paper examines neuron relevance characterization methods based on magnitude and distribution support for selecting instruction fine-tuning data.

HaloBlock allocates adaptive context halo regions across blocks to smooth context transitions and improve diffusion language model decoding.

Heterogeneous Structured Pruning selects sublayer-specific sparsity patterns guided by latency and memory models for hardware-efficient inference.

This paper proposes a low-rank bypass to accumulate small updates and reduce compounding errors in quantized recurrent model states.

The paper investigates progressive depth growth during self-supervised vision transformer pretraining to improve downstream accuracy while reducing computation.

This paper formulates a trace-minimization framework to design causal error-feedback coefficients for neural network weight quantization.

Proposes an attention mechanism that switches from linear complexity during prefilling to full attention during token decoding.

Proposes BoundaryEvo, dynamically adapting training data around a student model's evolving capability boundary during long-chain reasoning distillation.

The paper studies how calibration dataset size, language, and data distribution affect post-training expert pruning in mixture-of-experts models.

Proposes category-conditioned expert routing biases to mitigate data imbalance in mixture-of-experts language models.

Introduces a multilingual tokenizer that uses structural discovery and linguistic constraints to improve downstream model representations.

Presents a hypothesis testing framework using shadow models to audit whether a language model was distilled from a proprietary teacher.

The paper introduces an algorithm to optimize denoising timestep placement in continuous diffusion language models to improve generation quality.

The paper presents an empirical scaling law study measuring parameter and compute trade-offs in spiking neural network language models.

This paper presents a lightweight, interpretable neural network architecture for resource-constrained time-series classification on microcontrollers.

The paper introduces a router-free method that adaptively thresholds rank-one LoRA adapter responses to remove cross-talk noise per token.

Introduces a dynamic precision control framework that allocates weight chunks to mixture-of-experts models during inference.

This paper applies recursive training and discrete deduction procedures to message-passing neural networks to solve constraint-reasoning tasks with few parameters.

This paper presents an interpretability protocol to monitor latent reasoning dynamics and stability across multiple looped language model families.

The paper evaluates the energy efficiency, carbon intensity, and accuracy of ten post-training quantization methods across various language models and GPUs.

The paper demonstrates multi-fidelity training to optimize data generation compute and error scaling for neural PDE surrogates.

The paper compares single-head autoregressive beam search with parallel multi-token prediction for generating semantic IDs in recommendation systems.

The paper studies expert width and dimension balance in latent mixture-of-experts architectures to optimize parameter capacity across model layers.

The paper presents an in-context learning method that extracts compact concept vectors from demonstrations to guide sequential action choices for agents.

The paper presents an adaptive, training-free speculative decoding method to speed up inference for vision-language models.

This paper introduces CACTUS, an ADMM-based optimizer that speeds up curvature-aware pruning and preserves influence functions in compressed language models.

This paper presents a flow matching method to distill diverse continuous latent reasoning distributions from multiple chain-of-thought rationales.

The paper proposes an interface for typed decision models that selectively invokes and interrupts reasoning to balance accuracy and latency.

Investigates why looped transformers fail to retain or reuse earlier correct intermediate states when extended past training depth.

The paper presents a tiny learned policy controller to optimize token unmasking and remasking schedules in diffusion language models.

The paper proposes filtering high-drift responses to mitigate off-policy degradation in mixture-of-experts reinforcement learning.

The paper introduces a method that accelerates on-policy knowledge distillation by truncating student rollouts and applying inverse-probability weighting.

Proposes a fractional power-of-two quantization grid to enable efficient multiplier-free integer inference for large language models.

The paper uses latent eigenvectors from CLIP to select smaller, representative training datasets for diffusion models.

Studies how using UTF-16 byte order improves boundary prediction and representation efficiency in multilingual byte-level language models.

The paper surveys memory-centric compression techniques for managing inference resources and context windows in large language models.

The paper proposes a rank-local data-loading policy that rebalances variable-cost training batches without cross-node coordination to speed up distributed training.

The authors propose a theoretical and empirical framework to route token-level teacher selection during on-policy student distillation.

This paper presents a pathwise conformal prediction method that guarantees valid coverage across adaptive early-exit stopping policies without multi-exit score inflation.

Presents a distributed system architecture aggregating idle Apple Silicon devices into a virtual GPU cloud for inference.

Analyzes Mamba-3's input-dependent state rotations as oscillatory dynamics to explain their utility for tracking states and long-context language modeling.

Develops a category-theoretic framework to analyze and compare parameter-efficient fine-tuning methods as smooth maps in weight space.

This paper redesigns 4-bit AdamW optimizer-state quantization by performing stochastic rounding directly within preconditioner space.

Proposes separating continuous diffusion language generation into high-level flow-generated predictive states and block-level autoregressive token decoding.

Introduces Double-Take to mitigate capacity and cross-position binding failures in long-context test-time training architectures.

The paper introduces an evaluation framework to measure the trade-off between task quality and computation cost when replacing iterative neural modules.

Introduces ReSFlow, rectifying categorical flows along geodesic trajectories on statistical manifolds to enable one-step discrete generation.

Analyzes quantization error propagation in recurrent-depth language models and proposes adaptive mixed-precision schedules for faster prefill inference.

The paper proposes a robot control framework using predictive architectures and flow matching to generate action chunks without autoregressive decoding.

This paper analyzes prediction-derived error updates and test-time gradient objectives in trained test-time-training layers.

The paper analyzes depth transfer of hyperparameters in neural networks using an L-smoothness theoretical framework.

The paper presents an outcome-routed distillation and reinforcement learning method tailored for training mixture-of-experts diffusion language models.

The paper proposes a decoding algorithm that denoises consecutive blocks concurrently by promoting sampler states in diffusion language models.

The authors propose MirrorMoE to reconcile an exploration dilemma between fine-grained expert segmentation and shared experts in mixture-of-experts models.

This paper presents a transform-based framework for structured pruning in language models that optimizes basis selection using activation and weight statistics.

RePara trains diffusion language models using group-relative reinforcement learning on short token blocks to improve math generation speed and accuracy.

The paper proposes a context-efficient foundation model for generating realistic tabular data from limited context observations.

The paper analyzes pre-sink hidden-state geometry to detect and modulate attention sink formation in language models.

The paper analyzes the mathematical implicit bias of practical Muon optimization compared to gradient descent on linearly separable classification problems.