Concept 04: Expected Value & Decision Making

▶ Interactive Demo: Endgame Strategy Monte Carlo Simulator

Open the interactive demo below to simulate 1,000 matches comparing Safe vs. Risky autonomous strategies and watch the empirical match scores converge to their theoretical Expected Values.

1. The Real-World Problem: The Endgame Dilemma

With 20 seconds remaining in an intense playoff match, your alliance is deciding its final autonomous action:

• Strategy A (Safe Low Score):
  • 100% chance of success ➔ +2 points (Guaranteed).
• Strategy B (Risky Trap Climb):
  • 65% chance of success ➔ +5 points.
  • 35% chance of failure ➔ 0 points.

Which decision yields the higher average point output in the long run? How does an autonomous robot evaluate risk vs. reward dynamically during a match?

2. Solving It in Code (Java & WPILib)

First-Principles Java: Expected Value & Strategy Decision

// Strategy A: Reef High Goal (5 points, 70% success, 0 points on miss)
double evA = 0.70 * 5.0 + 0.30 * 0.0; // 3.50 points

// Strategy B: Reef Low Goal (2 points, 99% guaranteed)
double evB = 0.99 * 2.0 + 0.01 * 0.0; // 1.98 points

System.out.printf("Expected Value Strategy A (High): %.2f pts%n", evA);
System.out.printf("Expected Value Strategy B (Low):  %.2f pts%n", evB);

if (evA > evB) {
    System.out.println("Autonomous Decision: Attempt High Goal (Higher Long-Term Score)");
}

3. Bridge to Machine Learning: Reinforcement Learning & MCTS

In modern game-playing and autonomous AI:

• Reinforcement Learning (Q-Learning): The AI selects actions that maximize the Expected Future Reward (Q(s, a) = E[r + \γ \max Q(s’, a’)]).
• Monte Carlo Tree Search (MCTS in AlphaZero & Game Bots): The AI simulates thousands of random rollouts from the current board state to estimate the win probability of every legal move!

4. Review Checkpoints

Checkpoint 1

An autonomous shooting routine has a 40% chance to score a 3-point goal, and a 60% chance to score a 1-point ball. What is the expected point value per shot?

Solution: E[X] = (3)(0.40) + (1)(0.60) = 1.20 + 0.60 = 1.80 points.

Checkpoint 2

Why do autonomous robots use Monte Carlo simulations rather than just trusting single worst-case or best-case scenarios?

Solution: Because reality is probabilistic. Monte Carlo simulations reveal the entire distribution of possible outcomes (both median payoff and variance), allowing the software to make optimal risk-adjusted decisions.