Quasi-Newton Methods

If the Hessian is unavailable or if the resulting linear equation system in Newton's method is too expensive to solve $({\cal O}(n^3))$,we can use Quasi-Newton Methods. These methods only use gradient information, but are more sophisticated, since we approximate second order derivatives. The idea is to approximate the inverse of the Hessian, $[\nabla^2 f(x)]^{-1}$, using gradient information.