All papers

The authors develop a reachability-guided dynamic sparse training method that preserves functional pathways during extreme network pruning and regrowth.

The paper analyzes the impact of teacher rationales before distilling medical reasoning capabilities into compact language models.

Proposes a knowledge distillation method that focuses learning signals on specific structural disagreements between teacher and student representations.

This paper presents a structured pruning framework that decouples data-free neuron selection from localized weight reconstruction to reduce model memory requirements.

The paper distills an LLM-based search relevance judge into smaller multimodal student models for efficient e-commerce search evaluation.

This paper introduces an aligned language model family trained with structured reinforcement learning for clinical reasoning and traditional Chinese medicine prescribing.

The authors introduce an on-policy distillation method to transfer knowledge from autoregressive language models into diffusion language models using autoregressive rollouts.

This paper presents an integer-only local learning system on edge FPGAs for energy-efficient convolutional neural network training.

The paper introduces structured random transform constructions to accelerate 1-bit quantized Johnson-Lindenstrauss compression for key-value caches.

The paper investigates methods for encoding context into model weights so that language models properly apply the temporary context during subsequent inference.

The paper proposes a generation-aware singular value decomposition method to compress large language models while preserving generative output quality.

Introduces bidirectional interleaved pipeline parallelism to improve throughput and activation memory scaling in long-sequence model training.

This paper introduces a scalar quantization method using analytic coding to preserve language model predictive fidelity at precisions down to two bits.

This paper proposes a framework that dynamically steers internal language model neurons based on query-induced activation patterns.

This paper presents an activation-pairing method and selection criterion to improve closed-form model merging without retraining.

This paper proposes a decomposed routing framework that distills large language model knowledge into lightweight graph neural networks.

This paper introduces a linear-time algorithm to place codec operators in computational graphs to compress operand traffic during on-device inference.

This paper proposes a few-shot meta-ensembling framework that combines zero-cost proxies to rank neural architectures efficiently.

This paper proposes a zeroth-order fine-tuning method for language models that optimizes within task-adaptive learnable low-rank subspaces.

This paper formulates sequential test-time memory updates as latent variables within a predictive coding framework without backpropagation through time.

The paper introduces a client-side reinforcement learning scheduler to optimize synchronization decisions in edge federated learning.

The paper proposes a dynamic stratified grid sampling approach to train implicit neural representations on large scientific data faster.

The paper introduces a lossless compression codec for model checkpoint trajectories conditioned on reference floating-point exponents.

The paper proposes a method to align and merge multiple Low-Rank Adaptations using a shared basis guided by pretrained model weights.

This paper introduces a training-free caching method that separates cache synchronization and computation repair to accelerate autoregressive video diffusion generation.

The paper introduces an online tokenization algorithm that processes text streams in linear time with minimal algorithmic delay.

The paper proposes a Transformer architecture that plans dynamic cross-layer connections globally across the model based on input tokens.

Presents an enhanced convolutional neural network architecture for real-time traffic sign classification in autonomous driving applications.

The paper introduces an orthogonal parameter alignment approach for low-rank adaptation to mitigate catastrophic forgetting in continual learning.

Proposes a hybrid architecture combining global state-space modeling with local transformer attention and lexical memory for efficient inference.

The paper presents a teacher-student framework using language models for weak annotation to classify cybersecurity tweets.

Applies adversarial training and distillation to remove unwanted gender bias in audio-enabled language models.

The authors present an image-token compression technique using variable-resolution merging for semantic regions and learned relevance pruning inside the model.

This technical report presents an updated summary-token attention architecture to optimize cache compression and computation across model layers.

This paper introduces a posterior learning approach that improves multi-sample reasoning coverage by updating models using single posterior draws.

This paper presents a reinforcement learning approach that uses in-context dynamic memory during rollouts and self-distillation to accelerate reasoning exploration.

ChampionKV adaptively allocates key-value cache memory budgets across attention heads using observation-window concentration proxies.

The paper presents a saliency-guided patch replay method to mitigate catastrophic forgetting in spiking neural networks under tight memory budgets.

The authors design an FPGA implementation of spiking neural networks that performs both inference and training on-chip without backpropagation through time.

The paper develops a plug-and-play module for convolutional neural networks that provides optimal rotation and translation invariance based on invariant theory.

The paper proposes a rubric-based reinforcement learning method combining selective teacher distillation with an adaptive hand-off to standard optimization.

The paper applies Hilbert geometry to constrain activation shifts in softmax layers for stable optimizer updates in language model heads.

Proposes Dynamic K-Head, a question-conditioned budget selector that sets continuous latent reasoning steps based on input difficulty.

The paper introduces a direction- and topology-aware graph transformer to predict neural network architecture attributes like accuracy and latency.

Introduces a calibration-free latent merging framework to compress Mixture-of-Experts models and reduce memory overhead.

Proposes a structured zeroth-order optimization framework using subspace estimation and spectral control for memory-efficient model fine-tuning.

Presents a loop-aware post-training quantization method to compress looped language models without severe accuracy loss.

Presents an activation quantization method that groups tokens via pre-scan features to improve low-bit post-training quantization for Mamba super-resolution models.

The paper proposes a reasoning framework that discretizes visual states into tokens to jointly optimize visual computation and chain-of-thought steps.

This paper proposes an optimal-transport bridge framework to translate audio representations directly between mismatched neural codecs at low bitrates.

The paper presents an approach using character-level tokens, larger models, and synthetic data for the multilingual BabyLM data-efficiency challenge.

Identifies translation circuits in multilingual models and optimizes fewer than five percent of parameters for low-resource translation.

This paper presents ReaDy, a speculative decoding method using hybrid linear attention recurrent states to provide position-specific context during drafting.

The paper evaluates how language exposure distribution and ordering affect the performance of multilingual models trained with limited data.

This paper proposes an attention-based noise-aware training method to improve the robustness of Transformer models deployed on analog compute-in-memory hardware.

The paper distils a power-grid foundation model into a compact student network for real-time edge inference on power systems.

Adds content-addressable associative memory to latent world models to prevent collapse and improve long-horizon planning.

The paper introduces a two-stage low-rank fine-tuning method that uses residual gradients to add complementary adaptation directions without expanding the rank.

The paper introduces a method to convert dense models into mixture-of-experts by dynamically allocating different numbers of active experts across layers.

The paper introduces a compact tabular foundation model combining distribution-aware representations with dataset-conditioned sparse mixture-of-experts computation.