Search the full wire by company, model, lab, or keyword. Every story we have ever aggregated.
GTFormer replaces Transformer self-attention with hierarchical graph networks for O(N) complexity and 8192× length extrapolation.
Generalization bounds for multi-layer transformers show longer training sequences enable better extrapolation to longer sequences.
Triage: training-free method assigns video diffusion transformer attention tiles to skip, low-precision, or full-precision tiers based on error impact.
Evaluates transformer generalization for Bengali coreference resolution, finding limited cross-lingual transfer in low-resource settings.
Characterizes grokking via rank compression in MLPs and transformers, showing weight matrix compression follows generalization onset.
Theoretical analysis of KV-cache compression under key-query similarity bounds reveals transformer anisotropy contradicts worst-case alignment assumptions.
SVD-Attention diagnoses which transformer self-attention heads use full rank on real data, enabling sparsity-aware compression.
Theoretical framework for attention mechanisms through per-entity coordinate frames and parallel transport, constraining relational composition in transformers.
EEGBiFormer hybrid CNN-Transformer classifies attention states from 16-channel EEG in real-time for auditory protocol validation.
Supervised pairwise graph representations for Transformer error detection; post-hoc confounding analysis on graph topology.
Ariadne transfers task-specific fine-tuning behavior across differently-sized Transformers without parameter resizing, enabling efficient model adaptation across architectures.
Cross-modal probing reveals that vision transformer encoders develop layered orthographic, grammatical, and semantic text representations within vision-language models.
Transformer-based neural operator (RheOPT) pretrained on Newtonian flows for data-efficient few-shot prediction of non-Newtonian fluid dynamics.
Comparative analysis of position bias across Transformer, state-space, and Mamba architectures in long-context QA tasks.
Transformers with CoT reasoning learn heavy-ball gradient acceleration without explicit supervision, exceeding teacher-student gradient descent imitation.
Orbitol generates crystal structures via Crystalite diffusion Transformer with Wyckoff-orbit tokens, enabling direct control over space-group symmetry constraints.
Edge-aware attention layer for graph generation incorporates structural information while leveraging hardware-optimized Transformer implementations.
Minimal-pair probing reveals latent information exists in transformer models but remains unreadable to standard single-prompt probes.
AsymLoop separates context encoding from iterative refinement in looped Transformers to improve effective depth without scaling parameters.
Two-layer transformer with logarithmic precision copies arbitrary repeated strings via RoPE rotations, addressing failure modes in frontier LLM copying tasks.
MFlow proposes adaptive parameterization for pixel-space diffusion transformers, interpolating between x0 and velocity prediction to improve generation quality.
FuTransformer CUDA kernel for Hopper GPUs uses FP8 prefill, INT4 decode, and direct KV-cache placement for Llama inference.
Transformer-based variational autoencoder for sampling lattice systems, applying attention and physics-informed priors to the 2D XY model.
Mixture-Gated Theory of Anisotropic Representation Dynamics explains how Transformer self-attention layers grow or shrink representation differences via value transport and gating mechanisms.
LangDiT systematically optimizes Transformer architectures for continuous diffusion language models with improved normalization and conditioning.
LoopAhead accelerates looped Transformer decoding via speculative lookahead and branch pruning, reducing latency in parameter-efficient architectures.
Benchmark comparing Muon optimizer vs AdamW across data/time/tuning budgets on 17.7M–33.6M LLMs and vision transformers, showing 3.25–22.44% validation-loss gains.
Retraining-free elastic scaling of transformers enables single model to adapt across heterogeneous computational budgets via differentiable unit-ranking.
Analysis of simple relational baselines versus GNNs and Transformers for temporal heterogeneous graph prediction over relational databases.
Graph-enhanced framework discovers 99,527 disease trajectory motifs from MIMIC-IV with leakage-controlled splits and time-aware Transformer endpoint forecasting.