11. Combined Methods

11.1 Why Combine Methods

Intuition

Real decision systems rarely use one pure method. They combine prediction, optimization, simulation, penalties, and learning.

Formal combined structure

prediction model -> information model -> decision model -> simulation/evaluation -> policy tuning

flowchart LR
    A["Supervised learning"] --> B["Information model"]
    B --> C["Optimization / ADP policy"]
    C --> D["Simulation"]
    D --> E["Performance evaluation"]
    E --> F["Tune parameters"]
    F --> C

Properties

• Combining methods can improve performance.
• More complex systems are harder to explain and maintain.
• Simulation helps compare policies before deployment.

11.2 Common Combinations

Combination	How it works	Merit	Risk
RHO + CFA	optimize current model with penalties/buffers	practical and interpretable	penalty tuning
Look-ahead + PFA	simulate future using simple base rule	more anticipatory than rule alone	base rule quality
Look-ahead + VFA	limited future plus terminal value	reduces horizon-end bias	needs learned values
Supervised learning + RHO	forecasts feed optimization	uses data and constraints	prediction bias
RL + simulation	learn from simulated experience	safer exploration	simulator mismatch

11.3 Worked Example: Trucks & Barges

Business problem:

Ship containers by truck, barge, train, or hold them.
Minimize expected cost over five days.

Combined method:

1. Use supervised learning or historical data to estimate random arrivals.
2. Use RHO to optimize daily shipping.
3. Add CFA penalties for containers close to due date.
4. Use simulation to tune urgency thresholds and penalties.
5. Optionally use VFA to value remaining yard states.

Policy:

If due date is close, prioritize shipment.
If many same-destination containers exist, use barge/train consolidation.
If future arrivals are likely, preserve flexible capacity when possible.

11.4 Worked Example: E-Grocery Delivery

Information model:

demand by zone and time, travel times, service times

Decision model:

accept/reject delivery slots, assign vehicles, route orders

Combined policy:

Use supervised learning to predict future demand.
Use RHO to plan current routes.
Use CFA to penalize overcommitting scarce time windows.
Use look-ahead to sample future orders.
Use VFA to estimate the value of remaining capacity.

11.5 Validation and Trade Analysis

Intuition

A policy is only useful if it performs well under realistic conditions. Validation means testing it.

Formal evaluation

For policy \(\pi\), simulate \(N\) sample paths:

\[ \widehat{J}(\pi) =\frac{1}{N}\sum_{n=1}^{N}J^n(\pi), \]

where \(J^n(\pi)\) is the total cost of policy \(\pi\) in sample path \(n\).

Properties

• Test on many scenarios.
• Include common and rare cases.
• Compare against simple baselines.
• Track tradeoffs, not just one metric.

Worked comparison

Policy	Avg delay	Avg cost	Rejections
closest driver	18	1000	12
RHO	15	1100	8
RHO + CFA	13	1160	6

Business interpretation:

RHO + CFA improves service but costs more. The manager must decide whether the service improvement is worth the extra cost.

11.6 Designing a Practical Decision Architecture

Intuition

A practical DDM system is usually a pipeline. Data predicts the world, optimization chooses feasible decisions, simulation tests policies, and learning improves parameters.

flowchart TB
    subgraph Info["Information layer"]
        A["Historical data"]:::data
        B["Forecasts / samples"]:::forecast
    end
    subgraph Decision["Decision layer"]
        C["State S_k"]:::state
        D["Policy pi<br/>PFA/RHO/CFA/Look-ahead/VFA"]:::policy
        E["Decision x"]:::decision
    end
    subgraph Eval["Evaluation layer"]
        F["Simulation sample paths"]:::sample
        G["Metrics<br/>cost, delay, service"]:::metric
        H["Tune parameters"]:::learn
    end
    A --> B --> C --> D --> E --> F --> G --> H --> D
    classDef data fill:#e0f2fe,stroke:#0284c7,stroke-width:2px,color:#0f172a;
    classDef forecast fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#0f172a;
    classDef state fill:#eff6ff,stroke:#2563eb,stroke-width:2px,color:#0f172a;
    classDef policy fill:#ede9fe,stroke:#7c3aed,stroke-width:2px,color:#0f172a;
    classDef decision fill:#ffedd5,stroke:#ea580c,stroke-width:2px,color:#0f172a;
    classDef sample fill:#fef9c3,stroke:#ca8a04,stroke-width:2px,color:#0f172a;
    classDef metric fill:#f8fafc,stroke:#334155,stroke-width:2px,color:#0f172a;
    classDef learn fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#0f172a;

Worked example: choosing a combined method for e-grocery

Problem:

Customers book delivery windows.
Future orders are uncertain.
Vehicles and drivers are limited.
Late deliveries are costly.

Combined method:

Supervised learning predicts demand by zone and time.
RHO builds current routes.
CFA penalizes overfilling scarce evening capacity.
Look-ahead samples future orders before accepting a new slot.
VFA estimates the value of remaining capacity by zone/time.
Simulation validates service and cost tradeoffs.

Policy validation checklist

Check	Question
average performance	Does the policy reduce expected cost/delay?
variability	Is performance stable across days?
rare events	What happens on extreme demand days?
tradeoff	What cost increase buys better service?
robustness	Does the policy still work when forecasts are wrong?

11.7 Slide Enrichment: Why Combining Methods Is Normal

Mental model

The combination slides show that real systems rarely use one pure method. A policy may optimize the current decision, sample a limited future, and add a learned value at the horizon. Combining methods is a way to balance anticipation, runtime, and model accuracy.

flowchart LR
    A["Current state S_k"]:::state --> B["RHO / current optimizer"]:::method
    A --> C["Scenario sampler"]:::sample
    C --> D["Rollout or MSA"]:::method
    D --> E["Terminal VFA value"]:::value
    B --> F["Candidate decisions"]:::decision
    E --> F
    F --> G["Choose and implement first x"]:::decision
    G --> H["Observe omega_{k+1}"]:::random
    H --> A
    classDef state fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#0f172a;
    classDef method fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#0f172a;
    classDef sample fill:#fee2e2,stroke:#dc2626,stroke-width:2px,color:#0f172a;
    classDef value fill:#ede9fe,stroke:#7c3aed,stroke-width:2px,color:#0f172a;
    classDef decision fill:#ffedd5,stroke:#f97316,stroke-width:2px,color:#0f172a;
    classDef random fill:#fef9c3,stroke:#ca8a04,stroke-width:2px,color:#0f172a;

Course combination patterns

Combination	How it works	Merit	Weakness
RHO + CFA	optimize current model with penalties/buffers	practical and interpretable	penalty tuning
Rollout + PFA	simulate future using a simple rule	improves a baseline	base policy may be weak
Look-ahead + VFA	simulate limited horizon, add learned terminal value	balances horizon and runtime	needs both scenarios and learned values
Supervised learning + RHO	predict travel/demand, then optimize	easy analytics pipeline	forecast errors can mislead

11.8 Worked Example: Limited Look-Ahead Horizon Plus VFA

Suppose a delivery platform can look ahead only two decision points because long scenario trees are too expensive. At the end of the two-step look-ahead, it adds an approximate value of the remaining state:

\[ \text{score}(x)=R(S_k,x)+\text{sampled two-step future cost}+\hat V(S_{k+2}^x). \]

Business translation: the system explicitly simulates the near future and uses a learned value function to avoid pretending the world ends after two steps.

Numeric illustration

Current decision	Immediate cost	Two-step sampled future	Terminal \(\hat{V}\)	Total
Use central vehicle	4	18	20	42
Use outer vehicle	7	20	8	35

The combined method chooses the outer vehicle because it leaves a more valuable future position.

Chapter source note

Course basis: DDM combined-methods lecture and tutorials on Trucks & Barges, knapsack, and VFA. Textbook enrichment: Kochenderfer et al. on policy validation and trade analysis; Powell on mixing policy classes and blending learning with optimization. Combined methods should still be explainable using the DDM loop.

---