Module 2: Neural Layers & Activations

Welcome to Module 2: Neural Layers & Activations. In this module, we construct the building blocks of deep neural networks—from linear matrix dot-product transformations to non-linear activation functions that allow networks to bend decision boundaries and learn complex physical dynamics.

Concepts in this Module

• Concept 01: Linear Layers (Weights, Biases & Dot Products)
  • The Everyday Problem: How does a neural network combine multiple robot sensor inputs into a unified prediction?
  • Code & Math: Linear equation y = W @ x + b, weights as feature importance, biases as decision thresholds.
• Concept 02: Non-Linear Activation Functions (ReLU, GELU, Sigmoid)
  • The Everyday Problem: Why is a deep stack of purely linear layers no more powerful than a single linear layer? How do activations introduce non-linear curvature?
  • Code & Math: Rectified Linear Unit (ReLU), Gaussian Error Linear Unit (GELU), Sigmoid, and solving non-linear boundaries (XOR).