Appendix C: Source Map¶
Course authority¶
- DDM slides: MDP notation, post-decision states, Bellman logic, ADP policy classes, RHO, CFA, look-ahead, VFA.
- DDM tutorials: ride-hailing, service-time PDF, pizza delivery state, Trucks & Barges, dynamic knapsack, MDP graph and VFA graph calculations.
DDM slide anchors used in the enrichment¶
| Book area | Course slide ideas used |
|---|---|
| Overview | course goals, business analytics framing, lecture roadmap, Trucks & Barges tutorial role |
| Information modeling | descriptive/predictive/prescriptive analytics, floating-car data, travel-time matrices, aggregation, normalization, clustering, spatial/temporal validation |
| Stochasticity | certainty/risk/uncertainty, stochastic vs quasi-stochastic modeling, Monte Carlo objective approximation, expected value-variance, constraint penalties, Hurwicz, minmax regret |
| Dynamism | static vs dynamic planning, exogenous information process, endogenous decision process, urban delivery and traffic-management case |
| MDPs | \(S_k\), \(x\in X(S_k)\), \(R(S_k,x)\), \(S_k^x\), \(\omega_{k+1}\), \(P\), Bellman recursion, parcel replenishment example, curses of dimensionality |
| ADP | policy classes from Powell framing: PFA, RHO, CFA, look-ahead, VFA |
| VFA/combined methods | lookup tables, aggregation to features, exploration/exploitation, rollout plus VFA, limited look-ahead horizon plus terminal value |
DDM tutorial anchors used in the enrichment¶
| Tutorial area | Book use |
|---|---|
| ride-hailing | information model categories, demand/resources/environment, spatial demand timing |
| service/repair time | nonnegative right-skew distribution, expected service time, scheduling buffers |
| pizza delivery | MDP state, decision, post-decision state, stochastic transition, objective |
| Trucks & Barges | MDP modeling, PFA filters/rules/conditions, policy evaluation language |
| dynamic knapsack | PFA threshold, CFA reserved capacity, MSA/look-ahead, VFA lookup-table learning |
| exam-style graph questions | Bellman recursion checklist, post-decision value calculation, optimal policy marking |
Kochenderfer, Wheeler, and Wray enrichment¶
Relevant chapters and ideas:
- probability representation and inference,
- maximum expected utility,
- Markov decision processes,
- value functions and policy iteration/value iteration,
- approximate value functions,
- receding horizon planning,
- lookahead with rollouts,
- sparse sampling and Monte Carlo tree search,
- policy validation,
- model-based and model-free learning.
Powell enrichment¶
Relevant ideas:
- universal modeling framework for sequential decisions,
- five model components: state, decision, exogenous information, transition, objective,
- policy classes,
- learning and optimization integration,
- lookup tables and aggregation,
- cost function approximations,
- direct lookahead policies,
- stochastic search and policy tuning.
Final exam reminder¶
When sources differ, use the DDM course notation:
\[
S_k,\quad x\in X(S_k),\quad R(S_k,x),\quad S_k^x,\quad
\omega_{k+1},\quad P,\quad V(S_k^x),\quad \hat V(S_k^x),\quad \pi
\]