Seq2Synth is a taxonomy-guided benchmark for synthetic sequential tabular data — evaluating the time axis itself, not just the pooled rows.
Sequential tabular data — transaction logs, patient visits, loan performance records, sensor streams — is the dominant format in high-stakes domains. Yet single-table benchmarks treat records as independent samples, and relational benchmarks measure inter-table dependencies while staying agnostic to temporal ordering and population dynamics. Both collapse time into a static distribution.
The consequence is measurable. A state-of-the-art relational generator can reach near-perfect marginal fidelity while emitting duplicate timestamps, misordered records, and incomplete observation grids — properties that are fixed by construction in the real data, so there is no ambiguity about the correct value. Existing protocols report none of it.
Seq2Synth replaces the fixed metric suite with a taxonomy-guided protocol. Each dataset is classified along four axes — time representation, sampling regularity, trajectory dependence, and schema structure — and that classification determines which temporal metrics are meaningful to compute at all.
On top of the four fidelity dimensions, we extend standard utility and privacy protocols to the trajectory level: downstream prediction that consumes history, and leakage that shows up as reproduced sub-sequences rather than matching rows.
Is the generated time axis valid at all — before any repair or post-processing?
Does the population distribution hold at each aligned time slice, not only after pooling?
Do within-entity dynamics survive — step by step and across the whole trajectory?
Are relational cardinalities preserved as they evolve over time?
Given a real relational dataset 𝒯 and a synthetic counterpart 𝒯̂, the benchmark decides what is measurable before it measures anything. The order matters: timestamp metrics are computed on raw generator output so that validity violations cannot be hidden by repair.
Assign it along the four taxonomy axes: absolute or relative time, regular or irregular sampling, dependent or isolated trajectories, and schema type.
TS and LG apply broadly. CS requires absolute time and dependent trajectories. ST depends on the schema, with extra metrics for multi-child and multi-parent.
Timestamp metrics use raw output. Everything else runs after a post-processing pass: records sorted, timestamps aligned to the grid, validity corrections applied.
Each dimension score is the unweighted mean of its applicable sub-metrics, so no a priori importance ordering is imposed. Sub-metric profiles carry the diagnosis.
Score conventions: every metric lands in [0, 1] and higher is better, except TT-WassersteinDistance, DCR, and CS-DCR, which are reported as raw distances. Static-distribution (SD) and TS use raw outputs; CS and ST use the sparse variant; LG uses interpolation-fill where complete trajectories are required.
Seven core datasets, each supported by at least six of the seven broadly applicable baselines. Pick a dataset, then sort by any dimension — the rank column follows your sort. Higher is better throughout; “—” marks a dimension that does not apply to that dataset or a model that does not support it.
| # | Model | SD | TS | CS | LG | ST |
|---|
Across the seven core datasets, rankings induced by timestamp, longitudinal, and structural fidelity diverge sharply from the SD-based ranking — mean Spearman ρ of 0.36, 0.46, and 0.53, with 36%, 30%, and 27% of model pairs reversing order. Cross-sectional fidelity is the exception (pooled ρ = 0.82, 13% inversions), because it aggregates slice-wise scores over time and therefore preserves much of the global static ranking while still exposing temporally localized failures.
| Dataset | SD vs TS · ρ | Inv. | SD vs CS · ρ | Inv. | SD vs LG · ρ | Inv. | SD vs ST · ρ | Inv. |
|---|
Rossmann follows a daily grid with unique timestamps at one-day intervals, so TimestampUniqueness and RegularityConsistency equal 1.0 in the real data by construction. The joint-generation models score around 0.6 and 0.4 on those same properties — a violation read against a known reference value, not against another model's ranking. Autoregressive generators condition each event on the preceding sequence and preserve them; their strong SD scores show that plausible marginals can mask the failure entirely.
| Model | Order | Unique | Duration | Regularity | Grid |
|---|
TemporalOrderConsistency, TimestampUniqueness, TrajectoryDurationSimilarity, RegularityConsistency, and GridCompleteness. Note RTF's near-zero Order score alongside near-perfect Unique/Regularity/Grid: it generates a clean, regularly spaced axis whose global chronological direction is not anchored.| Model | KS-Comp | TV-Comp | StatSim | CatCov | ContSim |
|---|
| Model | FirstDiff KSComp | TransMat TVComp | AutoCorr Sim |
|---|
Structural sub-metrics are only distinguishing on multi-child schemas; among the core datasets that is Walmart alone. There, RTF's low ST is driven by SequenceLengthSimilarity and TemporalCardinalityShapeSimilarity — it does not reliably control how many child records each parent gets, or how that count evolves. Relational diffusion models (RDBDiff, ClavaDDPM) improve substantially under ST despite weaker SD rankings. On linear schemas, ST reduces to sequence-length similarity and saturates near 1.0, which is why those rows carry little signal.
Row-level nearest-neighbour metrics treat records as exchangeable across time, and can misread risk in both directions. Larger CS-DCR values than DCR mean the row-level nearest neighbours were coming from unrelated timestamps — incidental overlap in a bounded row vocabulary, not re-identification. In the other direction, single-row exposure cannot see copied consecutive sub-sequences: RTF and TabDiT look safe under DCR/NNDR and become the riskiest models under NGP(3). The autoregressive mechanism that improves within-trajectory fidelity also increases trajectory-level leakage.
| Model | DCR | NNDR | CS-DCR | CS-NNDR | NGP(1) | NGP(3) |
|---|
| Dataset | MLE-TS | MLE-Temporal | ||||
|---|---|---|---|---|---|---|
| ρSD | ρlong | ρall | ρSD | ρlong | ρall | |
The taxonomy is not descriptive metadata — it is what decides which metrics run. Cross-sectional fidelity, for instance, is undefined without absolute time and dependent trajectories, which is why AirBnB and PTB-XL show “—” in the CS column of every leaderboard. marks the seven core datasets used for cross-model comparison.
| Dataset | Domain | Time repr. | Periodicity | Trajectories | Schema |
|---|
Every metric declares its provenance and its applicability conditions. Adapted metrics reuse an established measure — usually an SDMetrics estimator — but change what it is computed over: inter-arrival times instead of feature values, a time slice instead of the pooled table, first differences instead of raw levels.
Scope: metrics are univariate (per feature, averaged over the applicable set), bivariate (per feature pair, averaged), or multivariate (joint over all features or a trajectory embedding). DCR and NNDR are time-agnostic baselines and are not counted in the provenance totals.
Because a model can be strong along one temporal axis and fail on another, and collapsing that into a leaderboard number would hide exactly what the benchmark exists to surface. Timestamp and longitudinal metrics test whether individual trajectories are valid; cross-sectional metrics test whether the population evolves correctly over absolute time; structural metrics test relational cardinalities. Use it as a diagnostic protocol and read the dimensions jointly.
Not on its own. Different metric families capture different properties, so disagreement only shows the dimensions are not interchangeable. The stronger evidence comes from properties with a fixed reference value in the real data: on a daily grid with unique timestamps, uniqueness and regularity must equal 1.0. A score of 0.4 there is a violation, not a preference.
Sorting and deduplication would repair precisely the failures we are trying to detect. Timestamp fidelity runs on raw generator output; every other dimension needs a well-defined time axis, so those run after records are sorted and aligned to the defined grid. Imputation ablations for that pipeline are in the online appendix.
NGP(n) = 1 − NgramExposure(n) measures whether discretized synthetic trajectory fragments reproduce real length-n token subsequences. DCR and NNDR compare single rows, so they cannot see a copied run of consecutive events. NGP(1) isolates the row-level case; NGP(3) catches sequential reproduction. Models can look safe at n = 1 and be the riskiest at n = 3.
Eight: ClavaDDPM, RCTGAN, RDBDiff, REaLTabFormer (RTF), RGCLD, RelDiff, SDV, and TabDiT. They split into autoregressive sequence models (RTF, TabDiT) and joint or relational generators (the rest) — a split that predicts their failure modes well. TabDiT covers only datasets whose structure it supports, hence the “—” rows.
Three things, none of which need a new model family. Model timestamps as ordered objects rather than ordinary continuous attributes. Give autoregressive generators explicit absolute-time anchoring — calendar embeddings, anchor tokens, or auxiliary timestamp losses — so token position is not confused with calendar time. Condition on sequence length and relational cardinality directly instead of letting them emerge, especially where parent–child counts evolve over time.
Yes. The evaluation code, taxonomy classification, and post-processing pipeline are in the GitHub repository. A new dataset needs its taxonomy assignment first, since that determines which metrics are applicable; a new generator only needs to emit the expected table format. Open an issue or a pull request.
Supported by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) — No. RS-2025-24803208, No. RS-2024-00354727 — and by IITP grants (No. RS-2020-II201336, AI Graduate School Program at UNIST; No. RS-2026-25616928; RS-2024-00436936; RS-2026-25528781), and by the Hankuk University of Foreign Studies Research Fund.