This is causing us to run out of travis-ci credits, making tests not run
at all. I deleted the duplicative tests and then disabled two additonal
ones by commenting them out that would be nice to run but I think are
not essential. In particular, the OSX one eats up a ton of credits. So
I disabled that. Maybe we can turn it back on later if we end up well
under the credit budget (or switch to github actions which appears to
have higher limits)
* wip: loss goes down when training without a dnn_trainer
if I use a dnn_trainer, it segfaults (also with bigger batch sizes...)
* remove commented code
* fix gradient computation (hopefully)
* fix loss computation
* fix crash in input_rgb_image_pair::to_tensor
* fix alias tensor offset
* refactor loss and input layers and complete the example
* add more data augmentation
* add documentation
* add documentation
* small fix in the gradient computation and reuse terms
* fix warning in comment
* use tensor_tools instead of matrix to compute the gradients
* complete the example program
* add support for mult-gpu
* Update dlib/dnn/input_abstract.h
* Update dlib/dnn/input_abstract.h
* Update dlib/dnn/loss_abstract.h
* Update examples/dnn_self_supervised_learning_ex.cpp
* Update examples/dnn_self_supervised_learning_ex.cpp
* Update examples/dnn_self_supervised_learning_ex.cpp
* Update examples/dnn_self_supervised_learning_ex.cpp
* [TYPE_SAFE_UNION] upgrade (#2443)
* [TYPE_SAFE_UNION] upgrade
* MSVC doesn't like keyword not
* MSVC doesn't like keyword and
* added tests for emplate(), copy semantics, move semantics, swap, overloaded and apply_to_contents with non void return types
* - didn't need is_void anymore
- added result_of_t
- didn't really need ostream_helper or istream_helper
- split apply_to_contents into apply_to_contents (return void) and visit (return anything so long as visitor is publicly accessible)
* - updated abstract file
* - added get_type_t
- removed deserialize_helper dupplicate
- don't use std::decay_t, that's c++14
* - removed white spaces
- don't need a return-statement when calling apply_to_contents_impl()
- use unchecked_get() whenever possible to minimise explicit use of pointer casting. lets keep that to a minimum
* - added type_safe_union_size
- added type_safe_union_size_v if C++14 is available
- added tests for above
* - test type_safe_union_size_v
* testing nested unions with visitors.
* re-added comment
* added index() in abstract file
* - refactored reset() to clear()
- added comment about clear() in abstract file
- in deserialize(), only reset the object if necessary
* - removed unecessary comment about exceptions
- removed unecessary // -------------
- struct is_valid is not mentioned in abstract. Instead rather requiring T to be a valid type, it is ensured!
- get_type and get_type_t are private. Client code shouldn't need this.
- shuffled some functions around
- type_safe_union_size and type_safe_union_size_v are removed. not needed
- reset() -> clear()
- bug fix in deserialize() index counts from 1, not 0
- improved the abstract file
* refactored index() to get_current_type_id() as per suggestion
* maybe slightly improved docs
* - HURRAY, don't need std::result_of or std::invoke_result for visit() to work. Just privately define your own type trait, in this case called return_type and return_type_t. it works!
- apply_to_contents() now always calls visit()
* example with private visitor using friendship with non-void return types.
* Fix up contracts
It can't be a post condition that T is a valid type, since the choice of T is up to the caller, it's not something these functions decide. Making it a precondition.
* Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
* Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
* Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
* - added more tests for copy constructors/assignments, move constructors/assignments, and converting constructors/assignments
- helper_copy -> helper_forward
- added validate_type<T> in a couple of places
* - helper_move only takes non-const lvalue references. So we are not using std::move with universal references !
- use enable_if<is_valid<T>> in favor of validate_type<T>()
* - use enable_if<is_valid<T>> in favor of validate_type<T>()
* - added is_valid_check<>. This wraps enable_if<is_valid<T>,bool> and makes use of SFINAE more robust
Co-authored-by: pfeatherstone <peter@me>
Co-authored-by: pf <pf@me>
Co-authored-by: Davis E. King <davis685@gmail.com>
* Just minor cleanup of docs and renamed some stuff, tweaked formatting.
* fix spelling error
* fix most vexing parse error
Co-authored-by: Davis E. King <davis@dlib.net>
Co-authored-by: pfeatherstone <45853521+pfeatherstone@users.noreply.github.com>
Co-authored-by: pfeatherstone <peter@me>
Co-authored-by: pf <pf@me>
Co-authored-by: Davis E. King <davis685@gmail.com>
* [TYPE_SAFE_UNION] upgrade
* MSVC doesn't like keyword not
* MSVC doesn't like keyword and
* added tests for emplate(), copy semantics, move semantics, swap, overloaded and apply_to_contents with non void return types
* - didn't need is_void anymore
- added result_of_t
- didn't really need ostream_helper or istream_helper
- split apply_to_contents into apply_to_contents (return void) and visit (return anything so long as visitor is publicly accessible)
* - updated abstract file
* - added get_type_t
- removed deserialize_helper dupplicate
- don't use std::decay_t, that's c++14
* - removed white spaces
- don't need a return-statement when calling apply_to_contents_impl()
- use unchecked_get() whenever possible to minimise explicit use of pointer casting. lets keep that to a minimum
* - added type_safe_union_size
- added type_safe_union_size_v if C++14 is available
- added tests for above
* - test type_safe_union_size_v
* testing nested unions with visitors.
* re-added comment
* added index() in abstract file
* - refactored reset() to clear()
- added comment about clear() in abstract file
- in deserialize(), only reset the object if necessary
* - removed unecessary comment about exceptions
- removed unecessary // -------------
- struct is_valid is not mentioned in abstract. Instead rather requiring T to be a valid type, it is ensured!
- get_type and get_type_t are private. Client code shouldn't need this.
- shuffled some functions around
- type_safe_union_size and type_safe_union_size_v are removed. not needed
- reset() -> clear()
- bug fix in deserialize() index counts from 1, not 0
- improved the abstract file
* refactored index() to get_current_type_id() as per suggestion
* maybe slightly improved docs
* - HURRAY, don't need std::result_of or std::invoke_result for visit() to work. Just privately define your own type trait, in this case called return_type and return_type_t. it works!
- apply_to_contents() now always calls visit()
* example with private visitor using friendship with non-void return types.
* Fix up contracts
It can't be a post condition that T is a valid type, since the choice of T is up to the caller, it's not something these functions decide. Making it a precondition.
* Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
* Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
* Update dlib/type_safe_union/type_safe_union_kernel_abstract.h
* - added more tests for copy constructors/assignments, move constructors/assignments, and converting constructors/assignments
- helper_copy -> helper_forward
- added validate_type<T> in a couple of places
* - helper_move only takes non-const lvalue references. So we are not using std::move with universal references !
- use enable_if<is_valid<T>> in favor of validate_type<T>()
* - use enable_if<is_valid<T>> in favor of validate_type<T>()
* - added is_valid_check<>. This wraps enable_if<is_valid<T>,bool> and makes use of SFINAE more robust
Co-authored-by: pfeatherstone <peter@me>
Co-authored-by: pf <pf@me>
Co-authored-by: Davis E. King <davis685@gmail.com>
* add helper methods to implement fused convolutions
* fix grammar
* add method to disable affine layer and updated serialization
* add documentation for .disable()
* add fuse_convolutions visitor and documentation
* update docs: net is not constant
* fix xml formatting and use std::boolalpha
* fix warning and updated net requirement for visitor
* fix segfault in fuse_convolutions visitor
* copy unconditionally
* make the visitor class a friend of the con_ class
* setup the biases alias tensor after enabling bias
* simplify visitor a bit
* fix comment
* setup the biases size, somehow this got lost
* copy the parameters before resizing
* remove enable_bias() method, since the visitor is now a friend
* Revert "remove enable_bias() method, since the visitor is now a friend"
This reverts commit 35b92b1631.
* update the visitor to remove the friend requirement
* improve behavior of enable_bias
* better describe the behavior of enable_bias
* wip: use cudnncudnnConvolutionBiasActivationForward when activation has bias
* wip: fix cpu compilation
* WIP: not working fused ReLU
* WIP: forgot do disable ReLU in visitor (does not change the fact that it does not work)
* WIP: more general set of 4d tensor (still not working)
* fused convolutions seem to be working now, more testing needed
* move visitor to the bottom of the file
* fix CPU-side and code clean up
* Do not try to fuse the activation layers
Fusing the activation layers in one cuDNN call is only supported when using
the cuDNN ones (ReLU, Sigmoid, TanH...) which might lead to suprising
behavior. So, let's just fuse the batch norm and the convolution into one
cuDNN call using the IDENTITY activation function.
* Set the correct forward algorithm for the identity activation
Ref: https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnConvolutionBiasActivationForward
* move the affine alias template to its original position
* wip
* remove unused param in relu and simplify example (I will delete it before merge)
* simplify conv bias logic and fix deserialization issue
* fix enabling bias on convolutions
* remove test example
* fix typo
* update documentation
* update documentation
* remove ccache leftovers from CMakeLists.txt
* Re-add new line
* fix enable/disable bias on unallocated networks
* update comment to mention cudnnConvolutionBiasActivationForward
* fix typo
Co-authored-by: Davis E. King <davis@dlib.net>
* Apply documentation suggestions from code review
Co-authored-by: Davis E. King <davis@dlib.net>
* update affine docs to talk in terms of gamma and beta
* simplify tensor_conv interface
* fix tensor_conv operator() with biases
* add fuse_layers test
* add an example on how to use the fuse_layers function
* fix typo
Co-authored-by: Davis E. King <davis@dlib.net>
* Don't try to use labels in unsupervised losses
I hope that is the right way of fixing this...
* fix it by duplicating most code in send_job (works on my machine)
I will probably need to find a way to reuse the code
* try to fix it reusing the code... not sure though
* Revert "try to fix it reusing the code... not sure though"
This reverts commit f308cac6df.
* check the type of the training label to fix the issue instead
* Missing include for `dlib::loss_multiclass_log_per_pixel_::label_to_ignore`
I was trying to compile the examples and encountered this issue after moving `rgb_label_image_to_index_label_image` to cpp file. Headers should include all symbols they mention.
* Update pascal_voc_2012.h
Should use the official entrypoint for including dnn stuff.
Co-authored-by: Davis E. King <davis685@gmail.com>
* add progress information (current/total and percent)
* print a new line instead of overwritting with spaces
* check if target_val is an integer with std::trunc
* add function to compute string dimensions in pixels
* use custom struct as a return value, remove first and last params
* Update dlib/image_transforms/draw_abstract.h
Co-authored-by: Davis E. King <davis@dlib.net>
* added support for std::optional if using C++
* oops, bug fix + check if item already holds a type
* oops, another bug fix
* remove warnings about unused parameters
Co-authored-by: pf <pf@me>
* [SERIALIZATION] addes support for std::variant
* [SERIALIZATION] bug fix + added tests
* support immutable types
* put an immutable type in std::variant
Co-authored-by: pf <pf@me>
Also fix this error from cmake 3.5.1:
```
CMake Error at CMakeLists.txt:62 (if):
if given arguments:
"CMAKE_COMPILER_IS_GNUCXX" "AND" "CMAKE_CXX_COMPILER_VERSION" "VERSION_LESS_EQUAL" "4.8.5"
Unknown arguments specified
```
* [TYPE_SAFE_UNION] use std::aligned_union instead of stack_based_memory_block. std::aligned_union was specifically designed to do this kind of stuff and we are better off trusting the standard library deciding what the correct storage type should be and what the appropriate alignment should be
* [TYPE_SAFE_UNION] as per Davis' suggestion, std::aligned_union can take Len parameter 0. Also, the content's of validate_type() has been bugging me for ages, so i created is_any which is based on std::is_same. I've also replaced is_same_type with std::is_same
Co-authored-by: Peter Featherstone <peter@grampus-server.com>
