Projected gradient descent

\(\mathbf y_{t+1}=\mathbf x_t-\eta_t\nabla f(\mathbf x_t)\)
\(\mathbf x_{t+1}=\Pi_{\mathcal X}(\mathbf y_{t+1})\)
Iteration t
Current loss f(xₜ)

Red cross: current xₜ
Orange: gradient proposal yₜ₊₁
Blue: projection to xₜ₊₁
Gray: completed iterates

Each iteration reveals the proposal, then its projection, then the new iterate.

ηₜ = 1/L = 1/70. Euclidean box projection clips each coordinate to its bounds.

Same rotated quadratic as DIRECT and Bayesian optimization: f(x) = z₁² + 35z₂², z = Rᵀx, rotation 0.65. Gray shading marks points outside the feasible box. The boundary start makes projection active; choose the box center for the same start as DIRECT / BO.