Sequential and temporal architectures are built around ordered information. The important question is not whether the input is text, audio, video, telemetry, or user events. The question is whether order changes meaning.
The Core Bet
This family assumes useful computation can be organized through time.
RNNs and LSTMs maintain state step by step. Temporal convolutional networks apply causal filters across windows. State space models compress long histories into efficient state updates. Mamba-style selective state space models add input-dependent control over what gets remembered and updated.
The shared idea is that the model should process a sequence without always paying the full cost of comparing every position to every other position.
When This Family Fits
Use this branch when streaming, latency, or long ordered context is a real constraint.
Speech, logs, market data, sensor streams, biological sequences, user sessions, and control systems often care about time. If the system must update continuously as new events arrive, temporal models can be a better engineering fit than full attention.
What To Watch
The tradeoff is expressiveness versus efficiency.
Attention is flexible because every token can directly inspect other tokens. Temporal and state-based models are often cheaper, but they must decide what to preserve. If the task needs arbitrary long-range comparisons, a purely temporal model may struggle. If the task needs fast streaming behavior, attention may be wasteful.
Where It Connects
Modern systems mix this family with attention, convolution, and generative objectives. A video model may need temporal reasoning across frames. A speech system may combine convolutional features, sequence encoders, and attention decoders.
Think of temporal architecture as the branch for state, order, and streaming pressure.