The Archive

Closed-form analysis of cross-layer interactions under weight-space ablation in transformer models, validating on pretrained models.

LoopMTP combines looped transformers with multi-token prediction to improve parameter-efficient reasoning via intermediate supervision.

CMuon optimizer stabilizes Diffusion Transformer training by chunking momentum orthogonalization on fused tensor weights, improving late-stage convergence.

Role-decoupled attention residuals decouple matching and content retrieval in Transformer depth-routing, improving architectural efficiency.

Theoretical analysis of finite-precision Transformers with transcript management and pop-enabled context channels.

MixFrag uses KL-divergence-guided mixed-precision quantization to deploy Vision Transformers efficiently on resource-constrained devices.

Neuro-symbolic framework combining Swin Transformer vision with fuzzy rule-based reasoning for explainable sewer pipe severity classification.

CoMem exploits transformer depth division of labor to cache residuals at intermediate layers for unbounded-context memory on Qwen3-8B.

LM-GRASP reformulates combinatorial optimization as online imitation learning, training Transformer constructive policies from scratch per problem instance.

Hierarchical spatio-temporal transformer for multi-level emergency department demand forecasting; healthcare application, not core AI frontier.

Algebraic framework deriving transformer expressivity from finite-precision attention dynamics and memory constraints.

MDTransformer uses mode-division photonic hardware with inverse-designed crossbars to accelerate Transformer inference with reduced phase-shifter overhead.

Domain adaptation via DANN and CDAN improves CNN/transformer robustness across acoustic scene classification device shifts.

Controlled benchmark analyzing how LoRA adaptation site affects learning geometry across lexical, factual, causal, and procedural objectives in Transformers.

MMOE applies sparse-expert efficiency principles from LLMs to diffusion transformers, balancing generation quality against training and deployment cost.

Evaluation of multilingual NMT models (NLLB, mBART) on English-Tamil translation across four datasets with BLEU/chrF metrics.

Physics Transformer adapts attention for PDE solving via functional tokenization respecting infinite-dimensional field structure.

ADAPT-GQE uses generative AI and transformers to learn efficient quantum ground-state preparation circuits for electronic structure.

PRIMS: physics-aware multimodal Transformer for on-device fluid identification in microfluidic systems integrating physical knowledge into representation learning.

Interior interpretability framework for Transformers using attention rollout and contraction theory to characterize feature propagation across layers.

Theoretical analysis of information bottlenecks in RNNs, Transformers, and state-space models via indexing primitive and causal complexity.

IQ-JEPA joint-embedding architecture for sound speed estimation from ultrasound IQ data using masked prediction pretraining and Hermitian Vision Transformer.

Hopformer combines sparsity pattern aggregation and LoRA-tuned Transformer for multivariable time-series forecasting.

KroQuant: Kronecker-structured block transforms for W4A4 post-training quantization of diffusion transformers with efficient inference.

Multi-modal transformer jointly processes time-series, wavelet, and spectral representations for nanopore blockade signal classification in single-molecule sensing.

ELSAA: efficient attention mechanism combining low-rank and sparse approximations without decomposing projection matrices, extends Transformer context length.

Appearance Pointers enable spatial region control in Diffusion Transformers via compact tokens, improving controllable image generation.

Manifold-Constrained Hyper-Connections enable parameter-efficient finetuning of frozen Transformers via learned residual routing.

Technique to mitigate shortcut learning in transformer-based spoken English proficiency assessment systems.

Loopie: MoE Transformer models (20B and 6B parameters) that outperform vanilla scaling by efficiently looping under fixed compute budgets.