Frank–Wolfe

\(\mathbf s_t\in\arg\min_{\mathbf u\in\mathcal X}\langle\nabla f(\mathbf x_t),\mathbf u\rangle\)
\(\mathbf x_{t+1}=(1-\eta_t)\mathbf x_t+\eta_t\mathbf s_t\)
Iteration t
Current loss f(xₜ)

Red cross: current xₜ
Orange: linear-oracle solution sₜ
Blue: convex-combination update
Gray: completed iterates

Each iteration reveals the linear minimizer, then the feasible update, then the new iterate.

The box oracle chooses a corner by the signs of the gradient coordinates.

Same rotated quadratic as DIRECT and Bayesian optimization: f(x) = z₁² + 35z₂², z = Rᵀx, rotation 0.65. Gray shading marks points outside the feasible box. Choose the box center for the same start as DIRECT / BO. Updates stay on a feasible segment; no projection is needed.