Commit Graph

10 Commits

Author SHA1 Message Date
Null Chilly
18927f93e2
Fix typos (#2811) 2023-06-11 20:21:05 -04:00
Adrià Arrufat
a7627cbd07
Rename function to disable_duplicative_biases (#2246)
* Rename function to disable_duplicative_biases

* rename also the functions in the tests... oops
2020-11-24 22:07:04 -05:00
Adrià Arrufat
3c82c2259c
Add Layer Normalization (#2213)
* wip: layer normalization on cpu

* wip: add cuda implementation, nor working yet

* wip: try to fix cuda implementation

* swap grid_strid_range and grid_strid_range_y: does not work yet

* fix CUDA implementation

* implement cuda gradient

* add documentation, move layer_norm, update bn_visitor

* add tests

* use stddev instead of variance in test (they are both 1, anyway)

* add test for means and invstds on CPU and CUDA

* rename visitor to disable_duplicative_bias

* handle more cases in the visitor_disable_input_bias

* Add tests for visitor_disable_input_bias
2020-10-20 07:56:55 -04:00
Adrià Arrufat
5ec60a91c4
Show how to use the new visitors with lambdas (#2162) 2020-09-06 09:27:50 -04:00
Adrià Arrufat
e7ec6b7777
Add visitor to remove bias from bn_ layer inputs (#closes 2155) (#2156)
* add visitor to remove bias from bn_ inputs (#closes 2155)

* remove unused parameter and make documentation more clear

* remove bias from bn_ layers too and use better name

* let the batch norm keep their bias, use even better name

* be more consistent with impl naming

* remove default constructor

* do not use method to prevent some errors

* add disable bias method to pertinent layers

* update dcgan example

- grammar
- print number of network parameters to be able to check bias is not allocated
- at the end, give feedback to the user about what the discriminator thinks about each generated sample

* fix fc_ logic

* add documentation

* add bias_is_disabled methods and update to_xml

* print use_bias=false when bias is disabled
2020-09-02 21:59:19 -04:00
Adrià Arrufat
5a715fe24d
Remove outdated comment from DCGAN example (#2048)
* Remove outdated comment

That comment was there from when I was using a dnn_trainer to train
the discriminator network.

* Fix case
2020-04-02 07:14:42 -04:00
Adrià Arrufat
e9c56fb21a
Fix warnings while running the tests (#2046)
* fix some warnings when running tests

* rever changes in CMakeLists.txt

* update example make use of newly promoted method

* update tests to make use of newly promoted methods
2020-03-31 19:35:23 -04:00
Adrià Arrufat
57bb5eb58d
use running stats to track losses (#2041) 2020-03-30 20:20:50 -04:00
Davis King
0057461a62 Promote some of the sub-network methods into the add_loss_layer interface so users don't have to write .subnet() so often. 2020-03-29 12:17:56 -04:00
Adrià Arrufat
f42f100d0f
Add DCGAN example (#2035)
* wip: dcgan-example

* wip: dcgan-example

* update example to use leaky_relu and remove bias from net

* wip

* it works!

* add more comments

* add visualization code

* add example documentation

* rename example

* fix comment

* better comment format

* fix the noise generator seed

* add message to hit enter for image generation

* fix srand, too

* add std::vector overload to update_parameters

* improve training stability

* better naming of variables

make sure it is clear we update the generator with the discriminator's
gradient using fake samples and true labels

* fix comment: generator -> discriminator

* update leaky_relu docs to match the relu ones

* replace not with !

* add Davis' suggestions to make training more stable

* use tensor instead of resizable_tensor

* do not use dnn_trainer for discriminator
2020-03-29 11:07:38 -04:00