updated docs

docs/docs/optimization.xml

@@ -37,6 +37,7 @@
          <item>find_max_bobyqa</item> 
          <item>find_max_trust_region</item> 
          <item>find_min_trust_region</item> 
+         <item>find_optimal_parameters</item> 
       </section>
 
       <section>
@@ -58,6 +59,7 @@
          <item>find_max_factor_graph_potts</item> 
          <item>find_max_parse_cky</item> 
          <item>min_cut</item> 
+         <item>elastic_net</item> 
       </section>
 
       <section>
@@ -877,6 +879,65 @@ Or it can use the elastic net regularizer:
                                  
       </component>
 
+   <!-- ************************************************************************* -->
+      
+      <component cpp11="true">
+         <name>find_optimal_parameters</name>
+         <file>dlib/optimization/find_optimal_parameters.h</file>
+         <spec_file link="true">dlib/optimization/find_optimal_parameters_abstract.h</spec_file>
+         <description>
+            Performs a constrained minimization of a function and doesn't require derivatives from the user. 
+            This function is similar to <a href="#find_min_bobyqa">find_min_bobyqa</a> and
+            <a href="#find_min_single_variable">find_min_single_variable</a> except that it
+            allows any number of variables and never throws exceptions when the max iteration
+            limit is reached (even if it didn't converge).
+         </description>
+      </component>
+
+   <!-- ************************************************************************* -->
+      
+      <component cpp11="true">
+         <name>elastic_net</name>
+         <file>dlib/optimization/elastic_net.h</file>
+         <spec_file link="true">dlib/optimization/elastic_net_abstract.h</spec_file>
+         <description>
+                This object is a tool for solving the following optimization problem:
+
+<pre>
+   min_w:      length_squared(X*w - Y) + ridge_lambda*length_squared(w)
+   such that:  sum(abs(w)) &lt;= lasso_budget
+</pre>
+
+                <p>
+                That is, it solves the elastic net optimization problem.  This object also
+                has the special property that you can quickly obtain different solutions
+                for different settings of ridge_lambda, lasso_budget, and target Y values.
+                </p>
+
+                <p>
+                This is because a large amount of work is precomputed in the constructor.
+                The solver will also remember the previous solution and will use that to
+                warm start subsequent invocations.  Therefore, you can efficiently get
+                solutions for a wide range of regularization parameters.
+                </p>
+                
+                
+                The particular algorithm used to solve it is described in the paper:
+                <blockquote>
+                    Zhou, Quan, et al. "A reduction of the elastic net to support vector
+                    machines with an application to gpu computing." arXiv preprint
+                    arXiv:1409.1976 (2014).  APA 
+                </blockquote>
+
+                And for the SVM solver sub-component we use the algorithm from:
+                <blockquote>
+                    Hsieh, Cho-Jui, et al. "A dual coordinate descent method for large-scale
+                    linear SVM." Proceedings of the 25th international conference on Machine
+                    learning. ACM, 2008. 
+                </blockquote>
+         </description>
+      </component>
+
    <!-- ************************************************************************* -->
       
       <component>

docs/docs/release_notes.xml

@@ -12,6 +12,8 @@
 
 <current>
 New Features:
+   - Added find_optimal_parameters()
+   - Added elastic_net
    - Added random_color_transform and disturb_colors().
    - Added a constructor for seeding rand with a time_t.
    - Added apply_random_color_offset()
@@ -46,15 +48,20 @@ New Features:
       
 Non-Backwards Compatible Changes:
    - Everything in dlib/matlab/call_matlab.h is now in the dlib namespace.
+   - DLIB_TEST() and DLIB_TEST_MSG() macros now require you to terminate them with ;
 
 Bug fixes:
    - Fixed bug in 10 argument version of call_matlab() and also cleaned up a few
    minor things.
    - setup.py and cmake scripts work in a few more contexts.
+   - Compiler errors in visual studio 2015
+   - Fixed a bug in gaussian_blur() that caused messed up outputs when big sigma values were used on some pixel types.
+   - Fixed minor bugs in join_rows() and join_cols(). They didn't work when one of the matrices was empty.
 
 
 Other:
    - Made cmake scripts uniformly require cmake version 2.8.4.
+   - faster fHOG feature extraction / face detection
    - C++11
       - CMake scripts now enable C++11 by default
       - Gave array2d and matrix move constructors and move assignment operators.

docs/docs/term_index.xml

@@ -261,6 +261,8 @@
          <term file="optimization.html" name="find_min_single_variable"       include="dlib/optimization.h"/>
          <term file="optimization.html" name="find_min_using_approximate_derivatives" include="dlib/optimization.h"/>
          <term file="optimization.html" name="find_min_bobyqa"                include="dlib/optimization.h"/>
+         <term file="optimization.html" name="find_optimal_parameters"        include="dlib/optimization/find_optimal_parameters.h"/>
+         <term file="optimization.html" name="elastic_net"                    include="dlib/optimization/elastic_net.h"/>
          <term file="optimization.html" name="solve_qp_box_constrained"       include="dlib/optimization.h"/>
          <term file="optimization.html" name="solve_qp_using_smo"             include="dlib/optimization.h"/>
          <term file="optimization.html" name="solve_qp2_using_smo"            include="dlib/optimization.h"/>