Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
random_projections [2018/04/20 15:38]
admin
random_projections [2018/11/03 16:49] (current)
admin
Line 356: Line 356:
 We propose an integer approximation of Echo State Networks (ESN) based on the mathematics of hyperdimensional computing. The reservoir of the proposed Integer Echo State Network (intESN) contains only n-bits integers and replaces the recurrent matrix multiply with an efficient cyclic shift operation. Such an architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. Our architecture naturally supports the usage of the trained reservoir in symbolic processing tasks of analogy making and logical inference. We propose an integer approximation of Echo State Networks (ESN) based on the mathematics of hyperdimensional computing. The reservoir of the proposed Integer Echo State Network (intESN) contains only n-bits integers and replaces the recurrent matrix multiply with an efficient cyclic shift operation. Such an architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. Our architecture naturally supports the usage of the trained reservoir in symbolic processing tasks of analogy making and logical inference.
  
 +https://​arxiv.org/​abs/​1412.7026v2 Language Recognition using Random Indexing
 +
 +https://​arxiv.org/​abs/​1808.07172 Fisher Information and Natural Gradient Learning of Random Deep Networks
 +
 +We obtain the inverse of Fisher information explicitly. We then have an explicit form of the natural gradient, without relying on the numerical matrix inversion, which drastically speeds up stochastic gradient learning.
 +
 +https://​arxiv.org/​abs/​1412.6616v2 Outperforming Word2Vec on Analogy Tasks with Random Projections
 +
 +https://​arxiv.org/​abs/​1712.04323v2 Deep Echo State Network (DeepESN): A Brief Survey
 +
 +https://​arxiv.org/​abs/​1803.07125v2 Local Binary Pattern Networks
 +
 +In this paper, we tackle the problem using
 +a strategy different from the existing literature by proposing local
 +binary pattern networks or LBPNet, that is able to learn and perform
 +binary operations in an end-to-end fashion. LBPNet1 uses local binary
 +comparisons and random projection in place of conventional convolution
 +(or approximation of convolution) operations. These operations can
 +be implemented efficiently on different platforms including direct hardware
 +implementation