All papers

Studies how the choice of local optimizers affects model performance degradation when scaling distributed training replicas under fixed budgets.

The paper proposes Noise Policy Optimization to align few-step generative models with black-box rewards by steering their initial input noise space.

This paper designs and simulates an all-transistor physical circuit using sub-threshold MOSFETs to run transformer attention and in-context learning.

This paper combines bidirectional and error-based predictive coding to enable deep, 4-bit quantized predictive coding networks.

This paper presents a multi-GPU training framework for temporal graph neural networks that relaxes event dependencies to reduce synchronization.

The paper studies depth extrapolation in looped transformers for graph connectivity, identifying finite iterations beyond which test performance degrades.

This paper investigates using analog radio frequency computing over wireless networks to deliver low-power neural network inference to resource-constrained edge devices.

The paper studies the impact of separating prediction horizons from state update intervals during sampling in generative flow models.

Develops a closed-loop distillation pipeline where student feedback refines teacher models to improve adversarial robustness.

The paper proposes a one-step generative reinforcement learning policy that balances reward improvement and entropy without iterative sampling.

This paper provides a thermodynamic framework explaining energy efficiency and optimization issues in softmax-free spiking transformers.

The paper introduces a backpropagation-free training method for generative relaxation models suitable for analog neuromorphic hardware execution.

Presents a predictive subtree prefetching method to accelerate power sampling for test-time language model reasoning.

Introduces a native sparse attention mechanism that combines sliding windows and keypoint selection for linear-time context extrapolation.

This paper introduces HexaBee-4B, a translation model trained across six languages using joint multilingual answering and bidirectional supervision.

Combines privileged teacher signals with value estimation in policy optimization to provide denser token-level supervision for reasoning tasks.

This paper introduces a coarse-to-fine text generation method using discrete code hierarchies to accelerate parallel sampling over diffusion language models.

Presents a unified encoder that jointly models type definitions and text for cross-domain entity, relation, and event extraction.

The paper proposes a pair-norm guided mixed-precision quantization method for key caches during long-output transformer inference.

This paper presents a generative compression framework using implicit randomized Hadamard dictionaries to speed up diffusion-based image and video coding.

Proposes a memory-efficient cross-attention method for distilling large paired audio-visual datasets into compact synthetic sets.

Introduces a neural network architecture enforcing layer-wise polytopal constraints to improve latent representation interpretability without degrading task performance.

Proposes a hybrid linear and sparse attention framework to accelerate feed-forward 3D foundation models across many views.

Presents a compressed latent world-action model enabling faster robotic manipulation inference on resource-constrained edge hardware.

Investigates training mechanisms in continuous diffusion language models, proposing design modifications for self-conditioning and embedding geometry.

Develops a retrieval-enhanced routing framework that dispatches multimodal queries to specialized vision-language models based on local failure patterns.

Linearizes multilayer perceptrons using low-degree rational approximations of activation functions to reduce inference costs across 3D models and language models.

Introduces an end-to-end open pipeline, pretraining corpus, and family of decoder-only models trained from scratch for Romanian.

Proposes a dynamic structural pruning method using biologically inspired plasticity to reduce neural network parameters during training.

Introduces an attention-aligned recurrent architecture that maintains persistent state representations across sequences using rolling low-dimensional traces.

The paper formalizes layer-to-layer representation sharing as a memory storage and retrieval framework for deep transformer networks.

The paper proposes a learnable low-rank polynomial sketch to reduce approximation errors in linear attention mechanisms.

The paper introduces a speculative decoding framework that uses early recurrent states to draft tokens in looped transformers.

This paper fine-tunes a quantized decision model to classify diseases from patient symptom descriptions without free-form text generation.

Proposes stabilization modules to enable half-precision FP16 proximal policy optimization training on commodity GPUs without numerical divergence.

Replaces dense LoRA experts with block-diagonal factorizations to isolate gradients and prevent catastrophic forgetting during continual multimodal tuning.

The paper presents SparPR, a runtime key-value cache recovery mechanism that reduces memory bottlenecks during model inference on consumer hardware.

Distills multiple speech attributes from four teacher models into a single shared speech encoder.

The paper proposes Batch-Paced, a method that dynamically allocates decoding compute across batched requests to improve discrete diffusion language model inference.

The paper proposes a training approach that uses learned chunk routing to speed up long-context diffusion language model processing.

The paper compares direct prompting, model-distilled supervision, and larger reader datasets for predicting user text highlighting behavior on web pages.

Proposes a teacher-guided reinforcement learning method using bidirectional constraints to help small language models learn effective tool use.

This paper constructs joint token probability laws from physics models to improve speculative verification in parallel language model decoding.

This paper introduces LoongSage, an asynchronous reinforcement learning framework designed to run multi-turn agent tasks with reduced system overhead.

This paper proposes DeepThink, a conceptual architecture that decouples knowledge capacity, activation width, and recurrent depth in language models.

The paper introduces ASSET, a reinforcement learning method that adaptively compresses speech encoder outputs into continuous audio tokens.

The paper proposes a linear score correction method to recover retrieval quality in compressed late-interaction document representations.

Combines load-adaptive hierarchical key-value caching with speculative decoding to accelerate long-context language model serving.

Presents Outlier-Aware Attention Linearization, using quadratic approximations to retain outlier dimensions during efficient Transformer attention evaluation.

Introduces a counterfactual-guided few-shot distillation framework to train smaller question-answering language models using limited training samples.

Proposes spatial-semantic token compression and layer routing to accelerate 3D vision-language model inference across multi-view inputs.

Introduces Cluster-TNN, a mini-batch graph partitioning framework that enables scalable topological deep learning on large graphs.

Proposes a chunk-level credit assignment method for on-policy distillation across language models with different tokenizers.

Designs a most-significant-digit-first neural network accelerator supporting early exits and digit-level pruning for energy-efficient medical image segmentation.

Introduces speculative decoding using self-sparse attention to speed up rollout generation during reinforcement learning.

Identifies distribution shift during autoregressive decoding and proposes a decoding-aware pruning method to accelerate model inference.

Proposes a sequence modeling framework using dynamic nonlinear time delays based on semantic similarity between current and past inputs.

The paper introduces a dual-path parametric memory method that combines pooled history representations with sequential adapter updates.

The paper proposes a four-tensor attention architecture that jointly normalizes attention over semantic and temporal fibers for scene prediction.

The authors propose an optimizer for Monarch-factored weights that replaces dense matrix orthogonalization with smaller block updates to reduce memory and pretraining time.