Case study
Café Inventory RL
Should a DQN run the kitchen? An honest audit
The question
Does added model complexity improve a campus café's daily fresh-meal order enough to justify the operational risk?
Q-learning, DQN and PPO against a business heuristic for daily replenishment, stress-tested through demand spikes, collapses, supplier disruption and spoilage shocks. The verdict was governance, not hype: keep the heuristic live and shadow the DQN for four weeks.
How it works
- 01
An auditable MDP: 28 daily decisions per episode — order 0, 4, 8 or 12 meals — with stochastic demand, overnight spoilage and shifting demand regimes.
- 02
Reward is sales minus purchasing, holding, spoilage, lost-demand and overflow costs.
- 03
A business-rule heuristic against tabular Q-learning, DQN and PPO, each trained for 100,000 steps on three seeds.
- 04
Every policy faces the same 200 seeds in six scenarios — normal, demand spike, demand collapse, forecast noise, supplier disruption, spoilage shock — with paired 95% bootstrap intervals.
- 05
A sales-only Q-agent as the reward-hacking control: it learns its proxy and lowers true profit from $782 to $749.
Results
95.2%
DQN service level — vs 92.0% heuristic
+$3.73
DQN profit edge, 95% CI −$17.51 to +$25.03
+$88.64
DQN edge in a demand spike
−$50.15
DQN edge in a demand collapse
Bottom line
Keep the heuristic live and shadow the DQN for four weeks — don't deploy it autonomously. It's non-inferior, not proven superior: it wastes more (8.9% vs 5.9%) and has a worse lower tail.
From the repo
Known limits
- —Demand is synthetic, and the simulator leaves out menu substitution, variable lead times, staffing and promotions.
- —Confidence intervals measure simulator uncertainty, not causal real-world impact.