Extreme Adaptive Transformer for Time Series Forecasting
Adaptive Transformer for time series forecasting with extreme-event handling; specialized application to hydrologic streamflow prediction.
Search the full wire by company, model, lab, or keyword. Every story we have ever aggregated.
Adaptive Transformer for time series forecasting with extreme-event handling; specialized application to hydrologic streamflow prediction.
TGO-II framework analyzes geometric evolution of Vision Transformer representations during supervised training via representational similarity.
Self-gating attention mechanism reduces quadratic complexity of transformers in time series forecasting via redundant attention map pruning.
Layer-wise analysis shows single transformer layer recovers most RL post-training gains, challenging uniform parameter update assumptions.
State-prediction separation hypothesis splits transformer computation into distinct streams for token prediction and state storage, improving efficiency.
Survey chapter on LLM mechanisms: Transformer architecture, attention, emerging capabilities, and layer-wise implementation of reasoning.
KnowledgeDebugger: GUI tool for knowledge localization and editing in Transformers, integrating EasyEdit library with no-code interface.
Develops post-training pruning methods for Diffusion Transformers that account for their unique weight distribution and skip-connections.
LUNA proposes LBS-free neural animation model using transformer-based motion regression and 3D Gaussian deformations for photorealistic human avatars from monocular images.
Signed-permutation coordinate transport framework enables steering vector and SAE alignment for RMSNorm transformers via sign-marginalized Hungarian algorithm.
FlexViT: FPGA accelerator for efficient Vision Transformer inference on resource-constrained edge devices.
Review Residuals gate transformer layer updates conditioned on both state and proposed update, improving selective information flow.
SpikeLogBERT applies spiking neural networks for energy-efficient log parsing vs. dense transformer approaches.
DPPE rethinks camera-based positional encoding for scaling multi-view transformers in 3D vision and novel view synthesis.
Physics-aware Neural Operator Transformer for real-time temperature field reconstruction in EAST fusion divertor.
Proposes joint optimization for hybrid attention layer selection in Transformers, treating layer importance as interdependent for long-context efficiency.
Medication errors, particularly dosing errors in clinical trials (CT), can lead to patient harm, adverse drug events and worse patient outcomes. Dosing errors are preventable, and early identification can improve trial integrity and mitigate subsequent clinical and financial…
DNA Language Models benchmark compares transformers (DNABERT2) and CNNs (ConvNova) on genomic tasks, questioning pretraining costs.
PEHT transformer for cellular network traffic prediction via urban congestion features; domain-specific infrastructure application.
Challenges scaling assumption in time-series forecasting; shows Ridge regression with optimized preprocessing rivals large transformers at lower cost.
CARVE: content-aware recurrent architecture with improved value gating enabling WY-form chunk-parallel training competitive with Transformers.
Sparse voxel representation has emerged as a scalable foundation for image-to-3D Gaussian Splatting (3DGS) generation, yet current methods struggle to preserve high-frequency visual details of input images due to two structural bottlenecks. First, they adopt discriminative 2D…
The loss of plasticity - the ability of a network to learn new information after having already learned older information - is a fundamental challenge in creating artificial neural networks capable of continual learning. Although this phenomenon has been known for decades, it…
Physics-informed surrogate models can accelerate computational fluid dynamics simulations. However, many existing methods reproduce global flow patterns more reliably than localized multiscale structures. This study presents a physics-informed Fourier-wavelet transformer for…
Modern language models, including transformer, recurrent, and memory-based variants, share a common chassis: a stack of identical layers in which parameters are allocated uniformly across depth. This is a default inherited from the original transformer and largely unchanged…
Matrix-aware optimizers such as Muon and Muown have recently shown strong empirical performance for pre-training Transformers. In particular, Muown separates each weight matrix into row magnitudes and an un-normalized direction variable, updating the former with Adam and the…
Can representations learned for image generation also support the evaluation of generated images? We study text-to-image reward prediction as a downstream task of generative representation learning. To this end, we introduce DiT-Reward, which converts a pretrained text-to-image…