查看2019凯风研读营的源代码

==== 张江 ====

* Learning Discrete Operators by Gradient Back Propagation
The triumph of deep learning technology has enabled us to learn very complex functions in various backgrounds via image recognition to natural language processing. The platforms like Tensorflow and pytorch further accelerate the progress of deep learning programming by automating the computation of gradient on complex structures. However, the requirement of the differentialability blocks the wide applications of automatic gradient computation because non-differential operators, such as sampling, permutation, selection, indexing, and so forth widely exists in programs. Although the conventional reinforcement learning algorithm like policy gradient my alleviates the suffering of non-differentialability in some sense, the slow speed of computation and the requirement of complicated skills made it difficult to apply. While, recently, researchers have invented several tricks like gumbel softmax, sinkhorn gumbel, and soft indexing to tackle the problem of non-differentialability, by carefully devising the computation process, we can do sampling, permutation, and indexing various objects with gradients. In this talk, I would like to introduce these methods and some applications. 

* 参考资料
- Categorical Reparameterization with Gumbel-Softmax, arXiv:1611.01144
- LEARNING LATENT PERMUTATIONS WITH GUMBEL- SINKHORN NETWORKS, arXiv:1802.08665v1
- Learning sparse transformations through backpropagation, arXiv:1810.09184v1
-