Reproducible Benchmark · NAB + ETT + Telecom Radio Network

Benchmarked Against Everything.
Orders of Magnitude More Efficient. Measured.

DriftMind was measured against adaptive ARIMA and Prophet, against OneNet, the leading deep architecture for online forecasting (NeurIPS 2023), and against the Chronos-Bolt and Google TimesFM foundation models. Same data, same protocol, and wherever possible the same physical hardware. DriftMind ran CPU-only, cold-start, with zero retraining; the same engine deploys unchanged from edge gateways to cloud. The results below presents a comprehensive and transparent evaluation.

78–350×

Faster than OneNet

Identical machine · CPU vs GPU · ~11× less memory

29–1,850×

Lower cost than foundation models

$ per 1M forecasts vs Chronos-Bolt & TimesFM, measured

3–6%

Accuracy gap to foundation models

Relative MAE on RAN telemetry · 42–740× more forecasts per CPU-second

Retraining cycles · cold-start

vs up to 16,870 for Prophet · no offline training

Benchmark 1 · NAB Multi-Dataset

Three-Way Direct Comparison

All three models were evaluated on six datasets from the Numenta Anomaly Benchmark (NAB), spanning machine sensors, cloud infrastructure, urban demand, ad-exchange pricing, and social-media volume, at three forecast horizons (h = 1, 6, 24). Same streaming protocol, same container, same hardware, no HTTP overhead, no artificial batching, and every engine receives the series' sampling interval and nothing else. Results are reported per horizon and per contender, with Prophet both on its default (automatic) seasonality and with seasonality switched off: DriftMind has the lower error in 16 of 18 cells against ARIMA and 18 of 18 against Prophet in either setting, at 16–91× the throughput of ARIMA at h = 24 and 56–655× at h = 1, with zero retraining.

NAB · Machine Temperature System Failure · 22,695 points · 5-min interval

Model	MAE (lower is better)	Throughput (pred/s)	Total Time	Retraining Cycles
DriftMind Accuracy Speed	0.7619	45,209	0.50 s	Continuous
Adaptive ARIMA	0.8090	300	1m 13s	1,055
Prophet (default seasonality)	3.7489	17	21m 14s	5,174
Prophet (no seasonality)	4.8704	24	15m 10s	5,285

Methodology. DriftMind runs one engine per horizon, with the look-back sized to the horizon (h = 1 → 15 steps, h = 6 → 30, h = 24 → 120), cluster retention 20 and similarity threshold 0.7; it learns continuously and never re-fits, so its MAE, throughput and time are reported per horizon engine. The baselines are adaptive, not static: ARIMA(5,1,0) runs in state-space form and applies its fitted coefficients through a Kalman filter at every step, with a full maximum-likelihood re-fit whenever accuracy degrades (sustained MASE above 3, or forecast-to-actual correlation below 0.2) and a cold re-fit if a warm start fails numerically; Prophet re-fits fully on the same triggers and extrapolates its latest fit between them, with its library defaults (automatic daily and weekly seasonality) in one setting and all seasonal components disabled in the other. Both baselines fit on the most recent max(200 observations, 3 days) window (864 points at 5-minute cadence, 200 at 30- and 60-minute) and produce one 24-step forecast vector per step scored at all three horizons, so their throughput, total time and re-fit count are engine-level (unchanged across the horizon tabs) while only their MAE varies. Every engine, DriftMind included, receives the series' sampling interval and nothing else; MAE is computed over the forecast origins common to all engines. All figures are from a single run on 27 October 2026 with DriftMind engine 6.0.0, every engine in the same container on the same machine.

Key finding: DriftMind has the lower error in 16 of the 18 dataset-horizon cells against ARIMA and in 18 of 18 against Prophet, with or without its seasonal components, and 16–655× the throughput of ARIMA in all 18. On the three series with a daily cycle (NYC Taxi, CPU Utilization, Twitter Volume) its error is 26–82% below ARIMA's. ARIMA keeps an edge on Machine Temperature, a slow drift with no daily cycle, at h = 6 (0.2%) and h = 24 (7.5%); those cells are shown unedited. DriftMind adapts continuously with zero re-fits, while ARIMA re-fits 6,259 times and Prophet 16,790 times on CPU Utilization alone, turning a seconds-long DriftMind run into minutes.

Mean Absolute Error

Throughput (predictions / second) · log scale

Total Execution Time (seconds) · log scale

Retraining Cycles

Benchmark 2 · ETTh2 & ETTm1 vs OneNet

DriftMind vs. State-of-the-Art Online Deep Learning

DriftMind was benchmarked against OneNet (NeurIPS 2023), the leading deep architecture for online forecasting under concept drift, which itself outperforms PatchTST, FEDformer, FSNet, and OnlineTCN. Protocol matched to OneNet's published setup exactly: all 7 channels (features=M), scaler fit on the training split only, identical evaluation windows and forecast-origin counts, MAE in the standardised space OneNet reports. Both models are evaluated under a strictly causal online update: no model is trained on any observation at or after the forecast origin. OneNet's published figures, produced under an evaluation that exposed future observations to the model, differ and are discussed in the conclusion. A naive-persistence floor is included: if a model cannot beat "tomorrow = today", the forecast has no value.

ETTh2 · Hourly Electricity Transformer Temperature · MAE (standardised space, lower is better)

Model	MAE	Hardware	Warm-up Required
DriftMind	0.313	CPU only	None (cold-start)
OneNet (NeurIPS 2023)	0.382	GPU	25% of dataset
Naive persistence (floor)	0.336	n/a	n/a

ETTm1 · 15-minute Interval · MAE (standardised space, lower is better)

Model	MAE	Hardware	Warm-up Required
DriftMind	0.181	CPU only	None (cold-start)
OneNet (NeurIPS 2023)	0.184	GPU	25% of dataset
Naive persistence (floor)	0.185	n/a	n/a

What this shows: evaluated causally, DriftMind has the lower error than OneNet at every horizon on both datasets, by 2–36% (ETTh2 0.313 vs 0.382 at h = 1, 0.436 vs 0.562 at h = 6; ETTm1 0.435 vs 0.683 at h = 24), and is below naive persistence in every cell, by 3–19%; OneNet has the higher error than persistence in 7 of the eight cells. On identical hardware the same DriftMind configurations run 78–350× faster, CPU against GPU (2.6–11.5 s per cell against 865–930 s), use ~11× less memory (0.6 GB vs ~6.8 GB, measured in a separate September run), and need no offline training phase.

MAE · ETTh2 (Horizon = 1, incl. persistence floor)

Runtime per evaluation cell · same machine (seconds, log scale)

Benchmark 3 · Chronos-Bolt & Google TimesFM

DriftMind vs. Time-Series Foundation Models

The same protocol, extended to the current generation of pre-trained foundation models: amazon/chronos-bolt-base (205M parameters) and google/timesfm-2.5-200m. All three engines ran on the same machine, CPU-only, identical windows and data. The foundation models arrived pre-trained on ~100 billion points; DriftMind cold-started on data it had never seen.

Accuracy · ETT · MAE (standardised space)

Cell	Persistence	DriftMind	Chronos-Bolt	TimesFM 2.5
ETTh2 H=1	0.336	0.313	0.308	0.304
ETTh2 H=6	0.504	0.436	0.391	0.393
ETTh2 H=24	0.688	0.566	0.515	0.529
ETTm1 H=1	0.185	0.181	0.175	0.171
ETTm1 H=6	0.296	0.288	0.241	0.243
ETTm1 H=24	0.525	0.435	0.358	0.361

The foundation models are more accurate on every ETT cell: by 2–5% at h = 1, 11–19% at h = 6 and 7–22% at h = 24. The rest of this section provides an explanation for why that is not the whole picture.

Resources · per ~75,500-forecast evaluation cell, same machine

Engine	Wall-clock per cell	Throughput (forecasts/s)	Peak RAM
DriftMind Speed Memory	2.6–11.5 s (incl. continuous learning)	~6,600–29,000	0.58 GB (September run)
Chronos-Bolt	336 s (inference only)	~225	1.79–1.83 GB
TimesFM 2.5	3,634–4,792 s (inference only)	~16–21	1.99–2.15 GB

Cost per 1M forecasts · measured CPU-time × list instance price (DigitalOcean 8 vCPU, $0.25/hr)

Engine	$ / 1M forecasts	vs DriftMind
DriftMind	$0.002–0.011	baseline
Chronos-Bolt	~$0.31	29–130×
TimesFM 2.5	$3.34–4.41	317–1,846×

The trade, stated plainly: foundation models buy their accuracy edge with two to three orders of magnitude more compute per forecast. At fleet scale the difference is structural: managed forecasting routes bill per predicted point: at list prices, a million forecast operations runs from a few hundred dollars (managed AutoML tiers) to thousands, against DriftMind's roughly one cent of commodity CPU. For a handful of series, use whatever you like. For thousands of streams, the economics decide.

Benchmark 4 · Synthetic RAN Telemetry

On Network Telemetry, the Gap Nearly Closes

The ETT datasets favour long-horizon pattern models. On telemetry closer to DriftMind's home domain, the open ranfst synthetic RAN dataset (PRB utilisation, RRC connections, user throughput; 5-minute cadence; stationary, regime-change and signalling-storm scenarios), the picture changes: the accuracy gap to both foundation models shrinks to 3–6% relative MAE, while the compute gap remains orders of magnitude. All engines evaluated on identical windows; DriftMind cold-started.

Scenario · Horizon	Persistence	DriftMind	Chronos-Bolt	TimesFM 2.5
Stationary · 5 min	0.3814	0.2928	0.2833	0.2832
Stationary · 1 h	0.3967	0.3097	0.2933	0.2942
Regime change · 5 min	0.3660	0.2843	0.2745	0.2743
Regime change · 1 h	0.3817	0.2995	0.2821	0.2825
Signalling storm · 5 min	0.3812	0.2956	0.2862	0.2854
Signalling storm · 1 h	0.4002	0.3140	0.2978	0.2974

MAE, standardised space, at the 5-minute (h = 1) and 1-hour (h = 12) horizons. The foundation models lead by 3–4% at 5 minutes and 5–6% at 1 hour; all three engines are far ahead of persistence, DriftMind by 22–23%. Same-machine throughput on this workload: DriftMind ~9,400–11,700 forecasts/s including continuous learning, vs ~219–224 (Chronos-Bolt) and ~16–20 (TimesFM).

Operator scale: at 15,000 sites → ~810,000 KPI streams → ~7 billion forecast operations per month, the measured compute bills are roughly $50/month for DriftMind vs ~$2,200 (Chronos-Bolt) and ~$24,000 (TimesFM 2.5); hardware cost only, identical workload. A 3–6% relative-MAE edge priced at 44–480× the compute is rarely the right trade for fleet telemetry.

Benchmark 5 · Every Cell, Every Contender

The Whole Table: 100 Cells, Seven Contenders

Everything above, collapsed into one view: 100 dataset-horizon-contender cells (NAB 54, ETT 28, RAN telemetry 18) at horizons 1, 6, 12 and 24, every contender DriftMind was measured against. The gap is the contender's MAE minus DriftMind's, divided by the contender's MAE: positive means DriftMind has the lower error, negative means the contender does.

Contender	Cells	DriftMind lower error	Mean gap where DriftMind is lower	Mean gap where DriftMind is higher
ARIMA	18	16 of 18	+41.9%	−3.8%
Prophet (default seasonality)	18	18 of 18	+61.8%	–
Prophet (no seasonality)	18	18 of 18	+60.0%	–
OneNet (causal protocol)	8	8 of 8	+19.6%	–
Persistence	14	14 of 14	+15.8%	–
Chronos-Bolt	12	0 of 12	–	−7.9%
TimesFM 2.5	12	0 of 12	–	−7.7%
All contenders	100	74 of 100	+43.8%	−7.5%

Mean relative error gap per contender · % of the contender's MAE

Every cell · one point per dataset, horizon and contender

The two halves are not symmetric. Where DriftMind is the more accurate model, 74 of the 100 cells, its error is on average 44% lower than the contender's (median 49%); where it is not, 26 cells, its error is on average 7% higher (median 6%). The asymmetry follows the contender, not the dataset: every cell DriftMind loses is against one of the two pre-trained foundation models, by 8% on average and at most 22%, or against ARIMA on the two machine-temperature cells, by 0.2% and 7.5%; every cell it wins is against an online or classical method, by 16–62% on average depending on the contender. Against the methods that share its operating conditions, a single CPU and no pre-training, the gains are large and almost universal (74 of 76 cells); against models pre-trained on ~10¹¹ points the loss is a few per cent, bought at two to three orders of magnitude more compute per forecast.

Benchmark Conclusion

What the Benchmark Shows

In this benchmark, traditional forecasting models, whether classical statistical approaches or deep learning-based architectures, struggle to adapt effectively to evolving data streams in real time, particularly when facing very different types of data sets and noise.

Why OneNet's published numbers differ: the accuracy figures reported in the OneNet paper differ from those published in this benchmark. This discrepancy stems from the evaluation methodology employed by the authors, which allowed future observations to be exposed to the model before predictions were requested. Consequently, the reported accuracy does not fully reflect performance under genuine online forecasting conditions.

Foundation Models for Time Series (FMTS) are among the few approaches capable of demonstrating meaningful adaptation while maintaining competitive accuracy. However, they face two significant limitations:

Computational requirements

Their substantial resource demands make deployment in constrained environments, particularly at the edge, challenging.

Potential data contamination

These models are pretrained on enormous datasets that may contain data similar to, or even overlapping with, commonly used benchmarks such as ETT. This raises questions about how well their reported accuracy generalises to genuinely unseen data.

Indeed, when evaluated on previously unseen workloads, such as our Telecom Radio dataset, their forecasting accuracy deteriorates significantly.

This is where DriftMind stands apart. It combines genuine online adaptability with a lightweight computational footprint, making it suitable for resource-constrained and edge environments. More importantly, it maintains competitive forecasting accuracy even when operating on previously unseen data, without requiring extensive pretraining or conventional model retraining.

DriftMind therefore addresses three fundamental requirements that existing forecasting approaches struggle to deliver simultaneously: online adaptability, computational efficiency, and robust generalisation to unseen workloads.

Online adaptability

Learning and inference are the same causal pass: every observation updates the model as it arrives, one point at a time, never looking ahead. No retraining cycle, no warm-up window.

Computational efficiency

Thousands to tens of thousands of forecasts per second on a single commodity CPU, in a sub-gigabyte footprint. No GPU, deployable from the edge to the cloud.

Robust generalisation

Because it learns each stream from scratch rather than recalling a pretraining corpus, its accuracy holds on genuinely unseen workloads, such as the Telecom Radio dataset, where pretrained models fall back.

Approach	Online adaptability	CPU / edge efficiency	Generalises to unseen
ARIMA / Prophet	Partialretrain lag	Noheavy retraining	n/a
OneNet	Yes*	NoGPU	n/a
Foundation models	Yes	No	Nocontamination
DriftMind	Yes	Yes	Yes

* Under the paper's non-causal online evaluation; corrected in Benchmark 2 above.

Benchmark Methodology

Setup & Reproducibility

All benchmarks were designed to be as fair and straightforward as possible: same data, same evaluation windows, and, for OneNet and the foundation models, the same physical machine. No cherry-picking of segments. Hyperparameters tuned on validation splits only, never on test data. Every engine receives the series' sampling interval and nothing else about the data. Where DriftMind loses, the loss is printed.

NAB Datasets

Six series from the Numenta Anomaly Benchmark (machine temperature, CPU utilisation, NYC taxi, ambient temperature, exchange CPC, Twitter volume), 1,624 to 22,695 points, 5- to 60-minute cadence. Continuous streaming simulation: no HTTP overhead, no artificial batching, all engines in one container.

ETTh2 & ETTm1 Datasets

Public electricity-transformer datasets, evaluated exactly as OneNet's published protocol: all 7 channels (features=M), one engine per channel, macro-averaged; the scaler is fit on the training split only; forecast-origin counts match OneNet's test set. MAE is reported in the standardised space OneNet reports.

Hardware · one machine

All same-platform comparisons ran on a single Apple M5 Max: DriftMind CPU-only, OneNet on the on-chip GPU (PyTorch MPS), Chronos-Bolt and TimesFM on CPU (torch, 6 threads). Identical data, identical windows; the speed and cost ratios are exact, not cross-hardware estimates.

Cold-Start Condition

DriftMind benchmarks were conducted with no offline training and no pre-loaded history: the engine starts empty and issues its first forecast once its initial buffer of L + H observations has filled. OneNet uses the first 25% of each dataset as its mandatory pre-training window.

OneNet Replication

The official OneNet implementation was cloned and run on the same machine under a delayed-update protocol that keeps every model update strictly before the forecast origin; its published accuracy was reproduced first, then re-measured causally. Published OneNet numbers are not used; both accuracy and timing come from our own causal run. The patch is supplied with the paper.

DriftMind Settings

Engine 6.0.0 everywhere. Look-back 15–60, max clusters 200, Gaussian smoothing; the ETT configurations were selected on the validation split only, and NAB uses one fixed configuration per horizon. The engine receives the series' sampling interval and nothing else.

Reproduce the NAB benchmark yourself: run the full comparison locally with a single Docker command, then open the Jupyter notebook:

docker run -p 8080:8080 -p 8888:8888 thngbk/driftmind-edge-lab

# Then open: http://localhost:8888/notebooks/dm_vs_arima_prophet_multi.ipynb

The academic paper detailing the full architecture and ETT benchmark methodology is available here: DriftMind: A Self-Adaptive, Cold-Start Framework for Time Series Forecasting.

See It Running on Your Data

Run DriftMind on your own time series: CSV upload, no setup, no GPU required. Results in seconds.

Try with Your CSV Read the Architecture

Benchmarked Against Everything.Orders of Magnitude More Efficient. Measured.