Linear Algebra

This page documents the core linear algebra tools included in dlib. In particular, the three most important objects in this part of the library are the matrix, vector, and rectangle. All the other tools on this page are functions for manipulating these three objects. A good example and introduction can be found in the matrix example program.

Most of the linear algebra tools deal with dense matrices. However, there is also a limited amount of support for working with sparse matrices and vectors. In particular, the dlib tools represent sparse vectors using the containers in the C++ STL. For details, see the notes at the top of dlib/svm/sparse_vector_abstract.h.

Finally, note that all the dense matrix tools can be obtained by #including <dlib/matrix.h> while the sparse vector tools can be obtained by #including <dlib/sparse_vector.h>. The geometry tools can be used by #including <dlib/geometry.h>.

Dense Matrix Tools matrix Basic_Math_Operators exp dlib/matrix/matrix_math_functions_abstract.h.html#exp log10 dlib/matrix/matrix_math_functions_abstract.h.html#log10 log dlib/matrix/matrix_math_functions_abstract.h.html#log sqrt dlib/matrix/matrix_math_functions_abstract.h.html#sqrt pow dlib/matrix/matrix_math_functions_abstract.h.html#pow squared dlib/matrix/matrix_math_functions_abstract.h.html#squared cubed dlib/matrix/matrix_math_functions_abstract.h.html#cubed sigmoid dlib/matrix/matrix_math_functions_abstract.h.html#sigmoid abs dlib/matrix/matrix_math_functions_abstract.h.html#abs reciprocal dlib/matrix/matrix_math_functions_abstract.h.html#reciprocal reciprocal_max dlib/matrix/matrix_math_functions_abstract.h.html#reciprocal_max normalize dlib/matrix/matrix_math_functions_abstract.h.html#normalize round dlib/matrix/matrix_math_functions_abstract.h.html#round ceil dlib/matrix/matrix_math_functions_abstract.h.html#ceil floor dlib/matrix/matrix_math_functions_abstract.h.html#floor round_zeros dlib/matrix/matrix_math_functions_abstract.h.html#round_zeros conj dlib/matrix/matrix_math_functions_abstract.h.html#conj norm dlib/matrix/matrix_math_functions_abstract.h.html#norm imag dlib/matrix/matrix_math_functions_abstract.h.html#imag real dlib/matrix/matrix_math_functions_abstract.h.html#real complex_matrix dlib/matrix/matrix_math_functions_abstract.h.html#complex_matrix sin dlib/matrix/matrix_math_functions_abstract.h.html#sin cos dlib/matrix/matrix_math_functions_abstract.h.html#cos tan dlib/matrix/matrix_math_functions_abstract.h.html#tan asin dlib/matrix/matrix_math_functions_abstract.h.html#asin acos dlib/matrix/matrix_math_functions_abstract.h.html#acos atan dlib/matrix/matrix_math_functions_abstract.h.html#atan sinh dlib/matrix/matrix_math_functions_abstract.h.html#sinh cosh dlib/matrix/matrix_math_functions_abstract.h.html#cosh tanh dlib/matrix/matrix_math_functions_abstract.h.html#tanh Linear_Algebra inv dlib/matrix/matrix_la_abstract.h.html#inv pinv dlib/matrix/matrix_la_abstract.h.html#pinv svd dlib/matrix/matrix_la_abstract.h.html#svd svd2 dlib/matrix/matrix_la_abstract.h.html#svd2 svd3 dlib/matrix/matrix_la_abstract.h.html#svd3 svd_fast dlib/matrix/matrix_la_abstract.h.html#svd_fast orthogonalize dlib/matrix/matrix_la_abstract.h.html#orthogonalize det dlib/matrix/matrix_la_abstract.h.html#det trace dlib/matrix/matrix_la_abstract.h.html#trace dot dlib/matrix/matrix_utilities_abstract.h.html#dot length dlib/matrix/matrix_utilities_abstract.h.html#length length_squared dlib/matrix/matrix_utilities_abstract.h.html#length_squared trans dlib/matrix/matrix_utilities_abstract.h.html#trans diag dlib/matrix/matrix_utilities_abstract.h.html#diag diagm dlib/matrix/matrix_utilities_abstract.h.html#diagm lowerm dlib/matrix/matrix_utilities_abstract.h.html#lowerm upperm dlib/matrix/matrix_utilities_abstract.h.html#upperm chol dlib/matrix/matrix_la_abstract.h.html#chol inv_lower_triangular dlib/matrix/matrix_la_abstract.h.html#inv_lower_triangular inv_upper_triangular dlib/matrix/matrix_la_abstract.h.html#inv_upper_triangular lu_decomposition dlib/matrix/matrix_la_abstract.h.html#lu_decomposition qr_decomposition dlib/matrix/matrix_la_abstract.h.html#qr_decomposition cholesky_decomposition dlib/matrix/matrix_la_abstract.h.html#cholesky_decomposition eigenvalue_decomposition dlib/matrix/matrix_la_abstract.h.html#eigenvalue_decomposition real_eigenvalues dlib/matrix/matrix_la_abstract.h.html#real_eigenvalues Conversions mat matrix_cast dlib/matrix/matrix_utilities_abstract.h.html#matrix_cast pixel_to_vector dlib/matrix/matrix_utilities_abstract.h.html#pixel_to_vector vector_to_pixel dlib/matrix/matrix_utilities_abstract.h.html#vector_to_pixel Sub_Matrix_Expressions range dlib/matrix/matrix_subexp_abstract.h.html#range subm dlib/matrix/matrix_subexp_abstract.h.html#subm rowm dlib/matrix/matrix_subexp_abstract.h.html#rowm colm dlib/matrix/matrix_subexp_abstract.h.html#colm set_subm dlib/matrix/matrix_subexp_abstract.h.html#set_subm set_colm dlib/matrix/matrix_subexp_abstract.h.html#set_colm set_rowm dlib/matrix/matrix_subexp_abstract.h.html#set_rowm Statistics sum dlib/matrix/matrix_utilities_abstract.h.html#sum sum_rows dlib/matrix/matrix_utilities_abstract.h.html#sum_rows sum_cols dlib/matrix/matrix_utilities_abstract.h.html#sum_cols prod dlib/matrix/matrix_utilities_abstract.h.html#prod mean dlib/matrix/matrix_utilities_abstract.h.html#mean max dlib/matrix/matrix_utilities_abstract.h.html#max min dlib/matrix/matrix_utilities_abstract.h.html#min find_min_and_max dlib/matrix/matrix_utilities_abstract.h.html#find_min_and_max index_of_min dlib/matrix/matrix_utilities_abstract.h.html#index_of_min index_of_max dlib/matrix/matrix_utilities_abstract.h.html#index_of_max variance dlib/matrix/matrix_utilities_abstract.h.html#variance stddev dlib/matrix/matrix_utilities_abstract.h.html#stddev covariance dlib/matrix/matrix_utilities_abstract.h.html#covariance randm dlib/matrix/matrix_utilities_abstract.h.html#randm gaussian_randm dlib/matrix/matrix_utilities_abstract.h.html#gaussian_randm Other_Utilities fft dlib/matrix/matrix_fft_abstract.h.html#fft ifft dlib/matrix/matrix_fft_abstract.h.html#ifft is_col_vector dlib/matrix/matrix_utilities_abstract.h.html#is_col_vector is_row_vector dlib/matrix/matrix_utilities_abstract.h.html#is_row_vector is_vector dlib/matrix/matrix_utilities_abstract.h.html#is_vector is_finite dlib/matrix/matrix_utilities_abstract.h.html#is_finite const_temp_matrix dlib/matrix/matrix_abstract.h.html#const_temp_matrix symmetric_matrix_cache dlib/matrix/symmetric_matrix_cache_abstract.h.html conv dlib/matrix/matrix_conv_abstract.h.html#conv conv_same dlib/matrix/matrix_conv_abstract.h.html#conv_same conv_valid dlib/matrix/matrix_conv_abstract.h.html#conv_valid xcorr dlib/matrix/matrix_conv_abstract.h.html#xcorr xcorr_same dlib/matrix/matrix_conv_abstract.h.html#xcorr_same xcorr_valid dlib/matrix/matrix_conv_abstract.h.html#xcorr_valid flip dlib/matrix/matrix_utilities_abstract.h.html#flip flipud dlib/matrix/matrix_utilities_abstract.h.html#flipud fliplr dlib/matrix/matrix_utilities_abstract.h.html#fliplr make_symmetric dlib/matrix/matrix_utilities_abstract.h.html#make_symmetric ones_matrix dlib/matrix/matrix_utilities_abstract.h.html#ones_matrix zeros_matrix dlib/matrix/matrix_utilities_abstract.h.html#zeros_matrix uniform_matrix dlib/matrix/matrix_utilities_abstract.h.html#uniform_matrix identity_matrix dlib/matrix/matrix_utilities_abstract.h.html#identity_matrix rotate dlib/matrix/matrix_utilities_abstract.h.html#rotate reshape_to_column_vector dlib/matrix/matrix_utilities_abstract.h.html#reshape_to_column_vector reshape dlib/matrix/matrix_utilities_abstract.h.html#reshape removerc dlib/matrix/matrix_utilities_abstract.h.html#removerc remove_row dlib/matrix/matrix_utilities_abstract.h.html#remove_row remove_col dlib/matrix/matrix_utilities_abstract.h.html#remove_col set_all_elements dlib/matrix/matrix_utilities_abstract.h.html#set_all_elements hash dlib/matrix/matrix_utilities_abstract.h.html#hash tmp dlib/matrix/matrix_utilities_abstract.h.html#tmp equal dlib/matrix/matrix_utilities_abstract.h.html#equal pointwise_multiply dlib/matrix/matrix_utilities_abstract.h.html#pointwise_multiply join_rows dlib/matrix/matrix_utilities_abstract.h.html#join_rows join_cols dlib/matrix/matrix_utilities_abstract.h.html#join_cols tensor_product dlib/matrix/matrix_utilities_abstract.h.html#tensor_product scale_columns dlib/matrix/matrix_utilities_abstract.h.html#scale_columns scale_rows dlib/matrix/matrix_utilities_abstract.h.html#scale_rows sort_columns dlib/matrix/matrix_utilities_abstract.h.html#sort_columns rsort_columns dlib/matrix/matrix_utilities_abstract.h.html#rsort_columns clamp dlib/matrix/matrix_utilities_abstract.h.html#clamp lowerbound dlib/matrix/matrix_utilities_abstract.h.html#lowerbound upperbound dlib/matrix/matrix_utilities_abstract.h.html#upperbound linspace dlib/matrix/matrix_utilities_abstract.h.html#linspace linpiece dlib/matrix/matrix_utilities_abstract.h.html#linpiece logspace dlib/matrix/matrix_utilities_abstract.h.html#logspace cartesian_product dlib/matrix/matrix_utilities_abstract.h.html#cartesian_product
2D/3D Geometry border_enumerator rectangle vector point rotate_point point_rotator point_transform point_transform_affine find_affine_transform point_transform_projective find_projective_transform rotation_matrix get_rect centered_rect center dcenter shrink_rect grow_rect translate_rect resize_rect resize_rect_width resize_rect_height move_rect nearest_point distance_to_rect_edge
Sparse Vector Tools sparse_to_dense dot dlib/svm/sparse_vector_abstract.h.html#dot distance_squared dlib/svm/sparse_vector_abstract.h.html#distance_squared distance dlib/svm/sparse_vector_abstract.h.html#distance assign dlib/svm/sparse_vector_abstract.h.html#assign length_squared dlib/svm/sparse_vector_abstract.h.html#length_squared length dlib/svm/sparse_vector_abstract.h.html#length scale_by dlib/svm/sparse_vector_abstract.h.html#scale_by add dlib/svm/sparse_vector_abstract.h.html#add subtract dlib/svm/sparse_vector_abstract.h.html#subtract max_index_plus_one dlib/svm/sparse_vector_abstract.h.html#max_index_plus_one add_to dlib/svm/sparse_vector_abstract.h.html#add_to subtract_from dlib/svm/sparse_vector_abstract.h.html#subtract_from min dlib/svm/sparse_vector_abstract.h.html#min max dlib/svm/sparse_vector_abstract.h.html#max make_sparse_vector dlib/svm/sparse_vector_abstract.h.html#make_sparse_vector make_sparse_vector_inplace dlib/svm/sparse_vector_abstract.h.html#make_sparse_vector_inplace sparse_matrix_vector_multiply dlib/svm/sparse_vector_abstract.h.html#sparse_matrix_vector_multiply
sparse_to_dense dlib/sparse_vector.h dlib/svm/sparse_vector_abstract.h This is a set of simple functions that take sparse vectors and converts them into equivalent dense vectors. mat dlib/matrix.h dlib/matrix/matrix_mat_abstract.h This is a set of simple functions that take objects like std::vector or array2d and convert them into matrix objects. Note that the conversion is done using template expressions so there is no runtime cost associated with calling mat(). matrix dlib/matrix.h dlib/matrix/matrix_abstract.h This is a 2D matrix object that enables you to write code that deals with matrices using a simple syntax similar to what can be written in MATLAB. It is implemented using the expression templates technique which allows it to eliminate the temporary matrix objects that would normally be returned from expressions such as M = A+B+C+D; Normally each invocation of the + operator would construct and return a temporary matrix object but using this technique we can avoid creating all these temporary objects and receive a large speed boost.

This object is also capable of using BLAS and LAPACK libraries such as ATLAS or the Intel MKL when available. To enable BLAS support all you have to do is #define DLIB_USE_BLAS and then make sure you link your application with your BLAS library. Similarly, to enable LAPACK support just #define DLIB_USE_LAPACK and link to your LAPACK library. Finally, the use of BLAS and LAPACK is transparent to the user, that is, the dlib matrix object uses BLAS and LAPACK internally to optimize various operations while still allowing the user to use a simple MATLAB like syntax.

Note that the cmake files that come with dlib will automatically link with ATLAS or the Intel MKL if they are installed. So using cmake makes this easy, but by no means are you required to use cmake or the dlib cmake files.

It is also worth noting that all the preconditions of every function related to the matrix object are checked by DLIB_ASSERT statements and thus can be enabled by #defining ENABLE_ASSERTS or DEBUG. Doing this will cause your program to run slower but should catch any usage errors.

matrix_ex.cpp.html matrix_expressions_ex.cpp.html matrix_utilities dlib/matrix/matrix_utilities_abstract.h This extension contains miscellaneous utility functions for manipulating matrix objects. matrix_la dlib/matrix/matrix_la_abstract.h This extension contains linear algebra functions to calculate QR, LU, Cholesky, eigenvalue, and singular value decompositions. It also contains a few other miscellaneous functions that solve systems of equations or calculate values derived from the above decompositions. matrix_math_functions dlib/matrix/matrix_math_functions_abstract.h This extension contains mathematical functions that operate on each element of a matrix independently. matrix_sub_expressions dlib/matrix/matrix_subexp_abstract.h This extension contains a number of functions for dealing with sub-matrices. border_enumerator dlib/geometry.h dlib/geometry/border_enumerator_abstract.h This object is an enumerator over the border points of a rectangle. nearest_point dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and a point and returns the point in the given rectangle that is nearest to the given point. distance_to_rect_edge dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and a point and returns the Manhattan distance between the rectangle's edge and the point. move_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and moves it so that it's upper left corner occupies the given location. resize_rect_height dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and returns a new rectangle with the given height but otherwise with the same edge points as the original rectangle. resize_rect_width dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and returns a new rectangle with the given width but otherwise with the same edge points as the original rectangle. resize_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and returns a new rectangle with the given size but with the same upper left corner as the original rectangle. translate_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle and moves it by a given number of units along the x and y axis relative to where it was before the move. center dlib/geometry.h dlib/geometry/rectangle_abstract.h Returns the center point of a rectangle. dcenter dlib/geometry.h dlib/geometry/rectangle_abstract.h Returns the center point of a rectangle. This is a version of center() which returns a double version of the point rather than one which uses integers to represent the result. Therefore, it is slightly more accurate. centered_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h There are various overloads of this function but the basic idea is that it returns a rectangle with a given width and height and centered about a given point. shrink_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle object, shrinks its borders by a given amount, and returns the result. grow_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h This function takes a rectangle object, grows its borders by a given amount, and returns the result. rotate_point dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is a function that rotates a 2D vector or point object about a given point. point_rotator dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is an object that rotates a 2D vector or point object about the origin. point_transform dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is an object that rotates a 2D vector or point object about the origin and then adds a displacement vector. point_transform_affine dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is an object that applies an affine transformation to a vector or point. Note that you can use find_affine_transform to easily create affine transforms from sets of point correspondences. find_affine_transform dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is a routine that takes in two sets of points and finds the best affine transformation that maps between them. point_transform_projective dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is an object that applies a projective transformation to a vector or point. Note that you can use find_projective_transform to easily create projective transforms from sets of point correspondences. find_projective_transform dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is a routine that takes in two sets of points and finds the best projective transformation that maps between them. rotation_matrix dlib/geometry.h dlib/geometry/point_transforms_abstract.h This is a method for creating 2D rotation matrices. get_rect dlib/geometry.h dlib/geometry/rectangle_abstract.h This is a simple template function that returns a rectangle representing the size of a 2D container (e.g. matrix or array2d). rectangle dlib/geometry.h dlib/geometry/rectangle_abstract.h This object represents a rectangular region inside a Cartesian coordinate system. It allows you to easily represent and manipulate rectangles. vector dlib/geometry.h dlib/geometry/vector_abstract.h This object represents a two or three dimensional vector.

If you want to work with general N-dimensional column vectors then you should the matrix object. In particular, you should usually use a matrix with this type: dlib::matrix<double,0,1>.

 point dlib/geometry.h dlib/geometry/vector_abstract.h This object represents a point inside a Cartesian coordinate system. Note that a point is simply a typedef for a vector that is 2D and uses longs to represent coordinate values.