From ea35161a94316202308adbecc2b3786e6c695c72 Mon Sep 17 00:00:00 2001 From: Davis King Date: Mon, 7 Sep 2009 04:09:51 +0000 Subject: [PATCH] updated the docs --HG-- extra : convert_revision : svn%3Afdd8eb12-d10e-0410-9acb-85c331704f74/trunk%403191 --- docs/docs/algorithms.xml | 166 ++++++++++++++++++++++++++++++++------- docs/docs/index.xml | 4 +- docs/docs/term_index.xml | 13 ++- 3 files changed, 151 insertions(+), 32 deletions(-) diff --git a/docs/docs/algorithms.xml b/docs/docs/algorithms.xml index e3c15b461..a7b334185 100644 --- a/docs/docs/algorithms.xml +++ b/docs/docs/algorithms.xml @@ -32,13 +32,19 @@ Optimization derivative + negate_function make_line_search_function poly_min_extrap line_search - find_min_quasi_newton - find_min_conjugate_gradient - find_min_quasi_newton2 - find_min_conjugate_gradient2 + find_min + find_min_using_approximate_derivatives + find_max + find_max_using_approximate_derivatives + + cg_search_strategy + bfgs_search_strategy + lbfgs_search_strategy + objective_delta_stop_strategy @@ -129,12 +135,28 @@ - make_line_search_function + negate_function dlib/optimization.h dlib/optimization/optimization_abstract.h + + This is a function that takes another function as input and returns + a function object that computes the negation of the input function. + + + + + + + + + make_line_search_function + dlib/optimization.h + dlib/optimization/optimization_line_search_abstract.h This is a function that takes another function f(x) as input and returns - a function object l(z) = f(start + z*direction). + a function object l(z) = f(start + z*direction). It is useful for + turning multi-variable functions into single-variable functions for + use with the line_search routine. @@ -145,7 +167,7 @@ poly_min_extrap dlib/optimization.h - dlib/optimization/optimization_abstract.h + dlib/optimization/optimization_line_search_abstract.h This function finds the 3rd degree polynomial that interpolates a set of points and returns you the minimum of that polynomial. @@ -158,7 +180,7 @@ line_search dlib/optimization.h - dlib/optimization/optimization_abstract.h + dlib/optimization/optimization_line_search_abstract.h Performs a line search on a given function and returns the input that makes the function significantly smaller. @@ -168,14 +190,24 @@ - + - find_min_quasi_newton + cg_search_strategy dlib/optimization.h - dlib/optimization/optimization_abstract.h + dlib/optimization/optimization_search_strategies_abstract.h - Performs an unconstrained minimization of the potentially nonlinear function f() using the - BFGS quasi newton method. + This object represents a strategy for determining which direction + a line search should be carried out along. This particular object + is an implementation of the Polak-Ribiere conjugate gradient method + for determining this direction. + +

+ This method uses an amount of memory that is linear in the number + of variables to be optimized. So it is capable of handling problems + with a very large number of variables. However, it is generally + not as good as the L-BFGS algorithm (see the + lbfgs_search_strategy class). +

optimization_ex.cpp.html @@ -186,12 +218,21 @@ - find_min_conjugate_gradient + bfgs_search_strategy dlib/optimization.h - dlib/optimization/optimization_abstract.h + dlib/optimization/optimization_search_strategies_abstract.h - Performs an unconstrained minimization of the potentially nonlinear function f() using a - conjugate gradient method. + This object represents a strategy for determining which direction + a line search should be carried out along. This particular object + is an implementation of the BFGS quasi-newton method for determining + this direction. + +

+ This method uses an amount of memory that is quadratic in the number + of variables to be optimized. It is generally very effective but + if your problem has a very large number of variables then it isn't + appropriate. Instead You should try the lbfgs_search_strategy. +

optimization_ex.cpp.html @@ -202,13 +243,20 @@ - find_min_quasi_newton2 + lbfgs_search_strategy dlib/optimization.h - dlib/optimization/optimization_abstract.h + dlib/optimization/optimization_search_strategies_abstract.h - Performs an unconstrained minimization of the potentially nonlinear function f() using the - BFGS quasi newton method. This version doesn't take a gradient function of f() - but instead numerically approximates the gradient. + This object represents a strategy for determining which direction + a line search should be carried out along. This particular object + is an implementation of the L-BFGS quasi-newton method for determining + this direction. + +

+ This method uses an amount of memory that is linear in the number + of variables to be optimized. This makes it an excellent method + to use when an optimization problem has a large number of variables. +

optimization_ex.cpp.html @@ -219,13 +267,15 @@ - find_min_conjugate_gradient2 + objective_delta_stop_strategy dlib/optimization.h - dlib/optimization/optimization_abstract.h + dlib/optimization/optimization_stop_strategies_abstract.h - Performs an unconstrained minimization of the potentially nonlinear function f() using a - conjugate gradient method. This version doesn't take a gradient function of f() - but instead numerically approximates the gradient. + This object represents a strategy for deciding if an optimization + algorithm should terminate. This particular object looks at the + change in the objective function from one iteration to the next and + bases its decision on how large this change is. If the change + is below a user given threshold then the search stops. optimization_ex.cpp.html @@ -233,6 +283,68 @@ + + + + find_min + dlib/optimization.h + dlib/optimization/optimization_abstract.h + + Performs an unconstrained minimization of a nonlinear function using + some search strategy (e.g. bfgs_search_strategy). + + + optimization_ex.cpp.html + + + + + + + + find_min_using_approximate_derivatives + dlib/optimization.h + dlib/optimization/optimization_abstract.h + + Performs an unconstrained minimization of a nonlinear function using + some search strategy (e.g. bfgs_search_strategy). + This version doesn't take a gradient function but instead numerically approximates + the gradient. + + + optimization_ex.cpp.html + + + + + + + + find_max + dlib/optimization.h + dlib/optimization/optimization_abstract.h + + Performs an unconstrained maximization of a nonlinear function using + some search strategy (e.g. bfgs_search_strategy). + + + + + + + + find_max_using_approximate_derivatives + dlib/optimization.h + dlib/optimization/optimization_abstract.h + + Performs an unconstrained maximization of a nonlinear function using + some search strategy (e.g. bfgs_search_strategy). + This version doesn't take a gradient function but instead numerically approximates + the gradient. + + + + diff --git a/docs/docs/index.xml b/docs/docs/index.xml index ee93dc489..bb28f3d57 100644 --- a/docs/docs/index.xml +++ b/docs/docs/index.xml @@ -109,7 +109,9 @@ transpose, trig functions, etc...
  • Unconstrained non-linear optimization algorithms such as - conjugate gradient and quasi newton techniques
    • + conjugate gradient, + BFGS, and + L-BFGS techniques
  • A big integer object
  • A random number object
    • diff --git a/docs/docs/term_index.xml b/docs/docs/term_index.xml index b21fea102..60dee61bd 100644 --- a/docs/docs/term_index.xml +++ b/docs/docs/term_index.xml @@ -33,10 +33,15 @@ - - - - + + + + + + + + +