Axon 07: Reinforcement Learning & Agentic Decision Systems

Welcome to the Reinforcement Learning & Agentic Decision Systems Axon. This track synthesizes mathematics, physics models, and deep learning into autonomous decision agents capable of mastering high-stakes games, strategic alliance coordination, and real-time robot control policies.

Modules in this Axon

1. Markov Decision Processes & Reward Engineering

• The Real-World Problem: How do we frame a complex physical robotics task as an optimization problem of states, actions, and rewards?

• Key Concepts: States S, Actions A, Transition probabilities P(s’

  s, a), Discount factor γ, Sparse vs. Dense reward shaping, and reward hacking pitfalls.

2. Value Functions & Deep Q-Learning

• The Real-World Problem: How does an agent learn the long-term value of taking an action before seeing the final match outcome?
• Key Concepts: State-Value V(s), Action-Value Q(s, a), Bellman Optimality equations, Deep Q-Networks (DQN), Experience Replay, and Target Networks.

3. Policy Gradients & Actor-Critic Methods

• The Real-World Problem: How do we train an agent to output continuous voltage and steering velocities directly without discretizing actions?

• Key Concepts: Policy function pi(a

  s), REINFORCE algorithm, Advantage Actor-Critic (A2C), and Proximal Policy Optimization (PPO).

4. Monte Carlo Tree Search & Match Strategists

• The Real-World Problem: How can an autonomous multi-robot alliance plan optimal endgame scoring cycles, defense counter-strategies, and cooperative climbing paths?
• Key Concepts: Tree traversal (Selection, Expansion, Simulation, Backpropagation), Upper Confidence Bounds for Trees (UCT), AlphaZero hybrid policy-value search, and real-time alliance strategists.