Fresh food forecasting

Demand forecasting is a barebone of every retailer's business: it is essential for managing supply chain, planning sales, and shaping customer loyalty. However, getting accurate and timely forecast is no easy task, with errors costing retailers billions of dollars. According to IHL Group, out-of-stocks account for $634 billion in lost sales worldwide each year, while overstocks result in $472 billion in lost revenues due to markdowns.

When it comes to fresh food forecasting, things get even more complicated. Fresh food is perishable and thus requires daily forecasting and daily replenishment. With fresh products accounting for up to 40 percent of grocers' revenue and shelf life of 1 to 7 days, perishables pose a forecasting challenge to every planning department.

For decades traditional planning and forecasting systems relied on a rule-based approach. This well-established technique proved to be useful when dealing with stable and predictable systems. Contemporary retailers' routine is anything but stable, however. Demand for fresh and ultra-fresh food fluctuates daily influenced by a wealth of internal and external parameters: advertising, price changes and promotions, product placement, public holidays, and even weather. In other words, historical sales data central to the rule-based approach are no longer enough to produce accurate demand forecasts for fresh food.

It is not only about the parameters. The classical approach also doesn't consider that overstocks and out-of-stocks affect business differently. In other words, the cost of forecasting error equal to 10 wholesale packages of milk is not the same for stockouts and overstocks. While the former results in lost sales, the latter consists of storage costs, cost of capital, and write-offs. For fresh-food assortments accounting for one-third of the cost of goods sold, this difference can be game-changing.

In a pursuit to overcome the limitations of the rule-based approach, retailers addressed the experience of other industries. Once the exclusive domain of internet giants, machine learning (ML) has spread far beyond the walls of IT companies to production lines, bank offices, warehouses, and even crop fields. Leading retailers also have adopted these smart predictive algorithms that analyze huge volumes of data. Unlike rule-based planning systems, ML algorithms can "learn" from data and make predictions based not only on historical sales record but a variety of parameters: from promotions and advertising to public holidays and weather.

What is more important, machine-learning technologies allow to fully automate demand-forecasting routine. Integrated into existing business processes, ML-based predictive solutions deliver daily forecasts for every SKU on a store level (to learn more about demand forecasting with ML technologies, download our latest case study on the project for Lenta, one of the biggest retail chains worldwide and the second-largest retail chain in Russia). To obtain highly accurate predictions, machine-learning algorithms that stem from gradient boosting over decision trees are used. At its core, the solution utilizes ensembles of models, every new of which improves the quality of the ensemble. These tools are set up to obtain better predictive performance than classical statistical approach and have proved to be equally suited for demand forecasting of basic stock items, promotions forecasting, and demand forecasting for new products when few historical data are available.

Fresh food is different though. Items with short shelf life, such as dairy, meat and produce are more difficult to forecast than basic stock items and thus require vastly different forecasting solutions. The good news is, machine learning is a set of various approaches that allow solving an exact pre-defined problem. What approach to use, depends on the problem. Having put to the test plenty of solutions, we seem to find an answer to the fresh-food forecasting challenge: Bayesian methods for machine learning.

Forecasting solutions for basic stock items usually deliver median or average forecasts. While such forecasts answer the question "How many items will be sold in the future?", they don't give much information when it comes to allocation of perishables. Instead of averages, Bayesian methods of machine-learning utilize interval estimations that allow computing quantile grids and approximating probability distributions for different levels of future demand. With fresh and ultra-fresh products losing their market value every day, interval estimations can help drive optimal inventory allocation decisions. Take, for example, a sour-milk product with a shelf life of 7 days. With interval estimations of Bayesian methods, probability distribution of different demand levels for this product may look as follows:

The left-hand column represents future demand in units, and the right-hand column stands for probability of selling the respective amount or less. In this example, the probability of selling 14 to 16 units of a sour-milk product is 12%, whilst the probability of selling 17 to 19 units is 53%. Instead of showing one averaged value, interval estimations allow introducing many possible outcomes to the analysis.

In addition, forecasting solutions based on Bayesian methods can be better aligned with business requirements. Instead of focusing on accuracy only, interval estimations reveal more information on future demand, allowing to assess the potential influence of overstocks and write-offs on key business metrics and thus giving an opportunity to make better decisions on the optimal quantity of each product to be ordered. After all, it is not the percentage of error but the cost of error that makes a difference.

Last but not least, Bayesian methods require fewer volumes of data to train the predictive model than traditional AI-based forecasting solutions. It allows forecasting demand for new products and recently opened stores when few historical data are available. It comes in handy for retail chains that expand their store network or regularly update assortments to keep their customers satisfied.

Proof-tested, utilizing Bayesian methods for fresh-food demand forecasting equals expectations. The forecasting system has been implemented by food delivery service with over 50 warehouses in Moscow and Saint Petersburg, demonstrating 14%-decrease in write-offs and 7.5%-decrease in out-of-stock days.

In a highly competitive retail industry, timely and reliable demand forecasting is essential. Order too little, and you lose sales; order too much, and you accumulate overstock and wastage. Bayesian methods for machine learning, however, give a fresh perspective on how to strike a balance between the two. Applied correctly, machine learning can help retailers reach new heights of operational efficiency, deepen customer loyalty and win a competitive advantage over industry rivals.