3D Optimization: Multiple Regression

Swimming Through Parameter Space

With 3 parameters, the likelihood becomes a 4D object: three dimensions for the parameters (β₀, β₁, β₂) and a fourth for the likelihood value itself.

• Full 3D view: Shows nested "shells" of likelihood — dark red = high likelihood, blue = low. The algorithm "swims" toward the darkest red region.
• 2D slice view: Fixes one parameter to show a 2D contour of the other two — easier to interpret but loses information.
• The MLE sits at the "hottest" point in the volume — the center of the red region where all gradients are zero.
• Real GLMs have dozens or hundreds of parameters — imagine navigating a 100-dimensional space!

This is the curse of dimensionality: we can't visualize high-dimensional spaces, so we rely on algorithms to navigate them mathematically. The gradient tells us "which way is up" even when we can't see the landscape.