Pytorch implementation of a Trust Region Newton Conjugate Gradient method.
To install the model please follow the next steps in the specified order:
- Clone this repository and install it using the setup.py script:
git clone https://github.com/vchoutas/torch-trust-ncg.git
- If you do not wish to modify the optimizer then run:
python setup.py install
- If you want to be able to modify the optimizer then run:
python setup.py build develop
To create the optimizer simply run:
optimizer = TrustRegionNewtonCG(parameter_list)
where paremeter_list is the list of parameters you wish to optimize. To perform one optimization step simply call the step function and pass a closure that computes the loss and the gradients. Note the the closure should have a boolean argument named backward, so that the optimizer avoids unnecessary backward passes.
For a simple example see the main.py function. To run it for the rosenbrock function execute the following command:
python -m torchtrustncg
For more details see chapter 7.2 of "Numerical Optimization, Nocedal and Wright":
@Book{NoceWrig06,
Title = {Numerical Optimization},
Author = {Jorge Nocedal and Stephen J. Wright},
Publisher = {Springer},
Year = {2006},
Address = {New York, NY, USA},
Edition = {second}
}