Yila
All use cases
Data Analysis

Time series forecasting: no peeking at the future

With 17,379 real hourly records from the UCI Bike Sharing dataset, Yila checks the time range and seasonality first, then trains the same model under a random split and a chronological split, putting MAE, RMSE, and R² side by side — showing why a random split is not real forecasting skill.

This is an hourly bike-sharing demand dataset. I want to forecast total rentals one hour ahead. First inspect the timestamps, frequency, target column, missing values, duplicates, and any suspicious columns — do not model yet.

Inspection complete: 17,379 hourly records spanning 2011-01 to 2012-12. One subtle risk caught — casual_users and registered_users sum exactly to bike_rentals on every row, so both are excluded from predictors; using them would reconstruct the target.

Inspection complete: 17,379 hourly records spanning 2011-01 to 2012-12. One subtle risk caught — casual_users and registered_users sum exactly to bike_rentals on every row, so both are excluded from predictors; using them would reconstruct the target.
Plot the overall bike-rental demand over time. Then show the average hourly pattern and compare working days with non-working days. Explain the trend and seasonality in plain language.

The series shows a long-term rise (mean hourly rentals grew from about 144 in 2011 to 235 in 2012), strong hourly peaks, warmer-month seasonality, and clear working-day effects — exactly the patterns a sound validation design must preserve.

The series shows a long-term rise (mean hourly rentals grew from about 144 in 2011 to 235 in 2012), strong hourly peaks, warmer-month seasonality, and clear working-day effects — exactly the patterns a sound validation design must preserve.
Before any complex model, evaluate two time-aware naive baselines — the previous hour and the same hour one week earlier — using exact prior timestamps. Report MAE and RMSE, and tell me which baseline is more reasonable.

Both baselines were evaluated on the same comparable samples with exact timestamp alignment: the same-hour-last-week baseline clearly wins, confirming that recurring calendar structure matters. A complex model only earns its place by beating this bar.

Now build a machine-learning regression model with lag features. First evaluate it with an ordinary random 80/20 train/test split, and explain clearly why that score may not represent forecasting skill.

Random 80/20 split: MAE 24.36, RMSE 41.00, R² 0.947 — but the split mixes timestamps from the entire period into both sets, so training sees later dates and matching seasonal conditions. This number should not be treated as forecasting performance.

Random 80/20 split: MAE 24.36, RMSE 41.00, R² 0.947 — but the split mixes timestamps from the entire period into both sets, so training sees later dates and matching seasonal conditions. This number should not be treated as forecasting performance.
Now use a chronological holdout: train on the earliest 80% of observations and test on the final continuous 20%. Use the same features, model, and metrics as before.

Chronological 80/20 holdout: MAE 45.48, RMSE 70.28, R² 0.898 — about 21 rentals higher MAE and 29 higher RMSE than the random split. Same model, same features; only the evaluation changed.

Chronological 80/20 holdout: MAE 45.48, RMSE 70.28, R² 0.898 — about 21 rentals higher MAE and 29 higher RMSE than the random split. Same model, same features; only the evaluation changed.
Using the same chronological training and test periods, fit a SARIMA model and compare it with the baselines and the machine-learning model. Use the same forecast horizon, MAE, and RMSE.

On the final future block, the chronological Random Forest outperformed both naive baselines — about 19 rentals lower MAE than the previous-hour baseline — and SARIMA was scored on the identical horizon, so every number shares one defensible protocol.

On the final future block, the chronological Random Forest outperformed both naive baselines — about 19 rentals lower MAE than the previous-hour baseline — and SARIMA was scored on the identical horizon, so every number shares one defensible protocol.
Compare the random split, chronological split, and baseline models in one table. Explain why the random-split result may be overly optimistic. Do not exaggerate the conclusion.

The gap comes from evaluation logic, not the model: random splitting lets later-date training observations sit beside test observations, inflating the score. No target leakage was detected — bike_rentals, casual_users, and registered_users were all absent from predictors. Even so, only the chronological holdout approximates live performance.

Summarize the correct workflow for time-series forecasting and list the three most common evaluation mistakes. Keep the conclusion concise and suitable for a beginner tutorial.

The right order: check the data → identify the cycles → split by time → build baselines → compare models. The three most common mistakes: using leaky columns like registered/casual as features; evaluating forecasting skill with a random split; and skipping naive baselines to jump straight to complex models.

unified-prediction-curves.pngUnified curves · PNG
unified-metrics.csvUnified metrics · CSV
split-comparison.csvSplit comparison · CSV
baseline-metrics.csvBaseline metrics · CSV
statistics-results.csvCycle statistics · CSV
leakage_curve_analysis.pyAnalysis script · Python
Task completed

Yila task replay complete.

Start with this method