Analyze with AI
Get AI-powered insights from this Mad Devs article:
A demand forecast can exist and still be unusable. Buyers may override it because stock records disagree with sales, promotion effects arrive late, or its horizon does not match ordering decisions. Demand forecasting estimates future demand over a defined horizon, but implementation succeeds only when that estimate fits the replenishment process. Praktiker’s move toward centralized forecasting and replenishment illustrates why the operating workflow matters alongside the forecast itself (Blue Sky Group). This guide helps retail data, engineering, supply chain, and merchandising leaders scope a first pilot, compare implementation options, and define evidence for a production decision.
Key takeaways
- Test machine learning against the current process and a simple statistical baseline on identical historical windows. Complexity is not evidence of improvement.
- Sales are an incomplete proxy for demand when stockouts, returns, substitutions, and assortment changes distort what customers could buy.
- A planning platform, managed ML stack, and custom pipeline assign different integration and ownership work – the right choice depends on the operating context.
- Validate more than forecast error. Backtest the model, simulate replenishment, and test data delivery, planner controls, monitoring, and fallback behavior before expansion.
Decide whether machine learning addresses your forecasting problem
Start with a falsifiable problem statement, not a model choice: “Weekly forecasts for store–SKU pairs underpredict promoted demand over the supplier lead-time horizon.” This gives a pilot a testable object, error, decision, and period.
Identify whether the problem is in the data, process, or model
Separate four failure modes. Data errors include missing stock positions, duplicate transactions, inconsistent identifiers, and late promotion records. Specification errors arise when a daily forecast feeds weekly ordering or ends before a supplier can deliver. Process constraints include late approvals and unusable files. Model limitations remain after these issues are controlled.
Consider a hypothetical retailer selling the same item in stores and online. Its model sees zero store sales for three days and predicts lower demand. The real cause was a stockout, but inventory snapshots were unavailable. A more complex algorithm cannot infer the missing availability reliably. The first pilot should therefore measure inventory-data coverage before it compares models.
Compare a statistical baseline with machine learning
Compare the current planner process, a transparent baseline such as a seasonal naive forecast, and an ML candidate. The baseline shows whether the new pipeline adds value beyond repeating the last comparable period.
ML is worth testing when demand contains recurring relationships that the baseline omits – for example, planned promotions, price changes, calendar effects, item attributes, and related behavior across many product series. RELEX describes promotions, product introductions, local events, and weather as relevant retail demand factors, while emphasizing planner input rather than model-only automation (RELEX).
Short history is not automatically disqualifying, and long history is not automatically useful. A new assortment may borrow signal from related items, while old data can mislead after channel changes. Nicolas Vandeput likewise treats expectations, data quality, metrics, and process as connected constraints (Towards Data Science). Test ML for a defined problem – do not presume it will win.
Define the forecast your replenishment process can use
A forecast specification tells engineering what to produce and operations how it affects a decision.
Set the forecast granularity, horizon, and update frequency
Define the object and unit first, then the SKU/location/channel hierarchy and time step. A store–SKU daily forecast supports different decisions from a category-level monthly forecast.
Set the horizon from the decision. If orders are reviewed weekly and delivery takes three weeks, a one-week forecast cannot cover the exposure. Include lead time and review cycle; update when new information can change the order. A buyer reviewing exceptions also needs a different interface from an automated service.
For the hypothetical retailer, a compact specification could be:
| Field | Pilot definition |
|---|---|
| Input | Daily sales, stock status, known promotions, prices, open orders, and item/location attributes |
| Output | Daily unit demand by SKU and fulfillment location, plus an uncertainty range |
| Cadence | Recomputed before the weekly order review and after validated material data changes |
| Owner | Supply chain product owner, with data engineering and ML owners named separately |
| Downstream action | Planner reviews suggested replenishment exceptions; forecast does not create orders directly |
Separate demand predictions from replenishment decisions
A forecast estimates demand. An order calculation also needs inventory, open orders, safety-stock policy, case packs, supplier calendars, and constraints. Treating forecast units as order units hides these rules.
RELEX likewise places demand forecasting inside a wider planning process in which planners assess exceptions and business context (RELEX). If the project scope extends from forecasting into transportation, warehouse, or order-management capabilities, use a separate framework for choosing logistics management software rather than expanding the forecast pilot into a full logistics replacement.
Reconstruct usable demand signals from sales and inventory data
Model readiness depends on whether the dataset represents what customers wanted and what the retailer knew at each decision point.
Account for stockouts, returns, promotions, and assortment changes
Build an event timeline for each item–location series. Join transactions to availability, returns, promotions, prices, calendars, assortment status, and identifier history. Reconcile units and time zones before aggregation.
Zero sales during a stockout do not prove zero demand. Mark the interval as censored or apply a documented estimation rule, then test sensitivity to that choice. Never label reconstructed demand as observed sales. Returns also need a policy because processing dates may differ from purchase dates.
Mad Devs encountered identifier, transfer, and quality work when moving retail purchase data from 1C into a warehouse for a recommender system. That is relevant experience with retail data and machine learning, but it was not a demand-forecasting implementation.
| Signal | Risk of error | Check | Action |
|---|---|---|---|
| Sales | Mistaken for unconstrained demand | Compare zero/low sales with availability records | Flag censored periods and retain observed sales separately |
| Inventory | Missing or stale snapshots | Reconcile movements and impossible negative balances | Set coverage thresholds; exclude or repair affected series |
| Returns | Assigned to processing rather than purchase period | Trace return timestamps to original orders where possible | Define gross, net, and exceptional-return treatment |
| Promotions | Dates, mechanics, or eligibility recorded incorrectly | Match campaign plans with price and transaction evidence | Use only validated, decision-time-known fields |
| Assortment | Launches and discontinuations look like demand shifts | Compare active-ranging dates with transactions | Split regimes and define cold-start handling |
Use only information available at prediction time
Build every training row as if the forecast were generated on that historical date. A promotion plan approved in advance is a valid future-known feature. Its eventual realized sales, end-of-week inventory, or a promotion label corrected afterward is not. Including those values creates leakage: the backtest solves a better-informed problem than production will face.
Store event time and availability time for changing inputs, then select the version that existed at each forecast cutoff. Timestamp manual overrides and their reasons too, or training can incorporate information that arrived after the prediction.
Choose between a planning platform and a custom forecasting pipeline
The implementation choice is an ownership decision. Compare candidates against the same required workflow and evidence, not feature-list breadth.
Compare a planning platform, managed ML, and custom development
| Option | Best fit | Main trade-off | Evidence to request |
|---|---|---|---|
| Planning platform | The retailer wants forecasting, planner workflows, exceptions, and replenishment features in one product | Process adaptation, data access, and configuration may be constrained by the product | Run a representative SKU/location workflow; export forecasts, overrides, and audit history |
| Managed ML | The team can build data and business integrations but wants managed training and inference components | Infrastructure work falls, but data contracts, evaluation, monitoring, and downstream integration remain | Reproduce a backtest and trace one batch from source data to consumed forecast |
| Custom pipeline | Requirements, controls, or integrations are materially specific and an internal or contracted team can own them | Maximum control brings engineering, operation, and lifecycle responsibility | Demonstrate reproducible training, versioned data, failure recovery, and total operating effort |
Evaluate process fit, data access, integrations, reproducible testing, support, and operating cost. Custom development is not the default winner; it is justified when critical requirements cannot be represented otherwise.
Define integration requirements and operational ownership
Map the minimum production flow:
Sources → validated historical snapshot → feature preparation → forecast generation → quality gate → planner or replenishment interface → monitoring
For every boundary, specify format, cadence, schema owner, late-data behavior, access, and service level. AWS’s current retail guidance shows uploaded datasets triggering SageMaker training and batch inference, with results stored and exposed to an application (AWS). It is an architecture example, not a mandatory stack.
Assign owners for the business decision, source quality, model behavior, and delivery. Record who handles incidents, approves retraining, and changes rules. Managed ML changes the infrastructure boundary; it does not remove responsibility.
Validate the pilot against a baseline and business constraints
The pilot must show whether the defined segment can be forecast and operated well enough to justify the next stage.
Test forecasts across consistent historical windows
Choose a bounded segment with meaningful replenishment pain – not only easy high-volume items. Freeze the data and specification. Compare all approaches on identical forecast origins, horizons, and item–location coverage.
Use rolling-origin backtesting: train before each cutoff, predict the required horizon, advance, and repeat. A random row split leaks future regimes and does not reproduce deployment. Include ordinary, promotional, and seasonal periods where history permits; report gaps.
Evaluate forecast error and bias across relevant segments
No aggregate metric is sufficient. Report absolute error and signed bias by product class, location, channel, horizon, and promotional status. Aggregation can hide systematic underforecasting in an important segment.
MAPE divides each error by its actual value, so zero actuals make individual terms undefined and near-zero actuals make them unstable. WAPE divides total absolute error by total actual volume; it is easier to aggregate but is undefined when that denominator is zero and can let high-volume series dominate. AWS’s metric documentation explicitly notes that WAPE is undefined when the sum of observed values is approximately zero in a backtest window (AWS). Select metrics from the decision cost and data shape, and inspect distributions rather than declaring a universal “90% accuracy” target.
Separate model performance from inventory outcomes
A backtest measures forecasts. A replenishment simulation adds ordering rules, lead times, inventory, and supply assumptions. A live pilot measures the whole system. Stockouts can persist when deliveries fail or order constraints prevent action.
Use a project-specific acceptance table before training begins:
| Criterion | How to verify | Approver |
|---|---|---|
| Data fitness | Coverage, reconciliation, cutoff-time, and leakage checks pass for the agreed segment | Data owner and supply chain owner |
| Forecast value | Candidate is compared with current process and baseline across identical windows and priority segments | ML owner and merchandising/supply chain lead |
| Bias control | Underforecast and overforecast patterns remain within agreed segment-specific limits | Business owner |
| Operational fit | Forecast arrives on time, can be reviewed, and reaches the downstream test interface | Product and integration owners |
| Safe operation | Monitoring, manual override, rollback, and fallback tests succeed | Engineering and operations owners |
Thresholds belong in the pilot charter because error costs differ by category. Mad Devs can help a team validate a forecasting proof of concept, from discovery and data assessment through integration planning.
Integrate forecasts gradually and keep a fallback
Production rollout should test the decision process while limiting the cost of failure.
Start in shadow mode and expand with planner approval
In shadow mode, generate forecasts on the production cadence without creating orders. Compare availability, timing, values, and exceptions with the existing process. Planners record whether they would accept or override each output and why.
Next, enable a bounded set of locations or categories with planner approval. Define the interface, forecast version, timestamp, unit, horizon, uncertainty, and status. Name an exception owner. Version manual adjustments so the team can distinguish them from model behavior.
Expand only after the same operating tests pass for the next segment. A category with intermittent demand or different supplier constraints may require a new acceptance decision rather than inheriting the first pilot’s result.
Monitor data freshness, forecast quality, and integration failures
Use three monitoring layers. Data monitoring checks arrival, schema, volume, missing values, identifier coverage, and promotions. Forecast monitoring compares observations with the correct forecast version and horizon. Integration monitoring confirms the downstream system received, parsed, and applied the approved output.
Drift is a diagnostic signal, not an automatic retraining command. A distribution change may reflect a promotion, assortment reset, broken feed, or genuine behavior change. Investigate the cause, then choose whether to repair data, update rules, retrain, or temporarily fall back. Preserve dataset, code, model, and forecast versions so the decision can be reproduced.
The same operational boundaries appear in AWS’s reference flow from source upload through orchestration, inference, storage, and application access (AWS). Roman Panarin’s publicly described work includes adjacent time-series analysis and anomaly detection at Enji, but that experience should not be interpreted as a retail demand-forecasting deployment (LinkedIn).
Define fallback behavior and criteria for expansion
Specify fallback by failure type. If a batch is missing, use the last valid forecast only within an agreed freshness limit; after that, revert to the approved baseline or existing planner process. If only some series fail quality checks, quarantine them rather than discarding the whole batch. If the interface fails, stop automated consumption and alert the integration owner. Never substitute zeros for unavailable forecasts.
The pilot output should be a decision package: confirmed scope and forecast contract, documented data limitations, reproducible comparison with baselines, integration design, ownership map, monitoring and fallback rules, and signed go/no-go criteria. This makes a negative result useful too – it can show that data or process work should precede further model investment.
Plan your first forecasting implementation
Choose one decision, a bounded item–location segment, and a replenishment horizon. Agree on the baseline, backtest windows, segment metrics, operational checks, and approvers before building. Reliability means a useful forecast and a traceable path from available data to a safe downstream action.
AI and machine learning development Plan your forecasting pilot Validate your data, model, and integrations before expanding a forecasting system. Discuss your project |
Latest articles here
