# Learning to Compose

http://openreview.net/pdf?id=S1c2cvqee DESIGNING NEURAL NETWORK ARCHITECTURES USING REINFORCEMENT LEARNING https://bowenbaker.github.io/metaqnn/

We propose a meta-modelling approach based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using Q-learning with an - greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task.

http://openreview.net/pdf?id=r1Ue8Hcxg Neural Architecture Search with Reinforcement Learning

Despite their success, neural networks are still hard to design. In this paper, we use a recurrent network to generate the model descriptions of neural networks and train this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set.

http://arxiv.org/abs/1511.02799v3 Neural Module Networks

In this paper, we have introduced neural module networks, which provide a general-purpose framework for learning collections of neural modules which can be dynamically assembled into arbitrary deep networks.

http://openreview.net/pdf?id=BJK3Xasel NONPARAMETRIC NEURAL NETWORKS

Determining the optimal size of a neural network for a given task is a challenging problem that is often addressed through a combination of expert tuning, random search or black-box bayesian optimization. These methods have two drawbacks: (A) They are expensive because training has to be started anew for each network size considered and (B) They cannot seize upon performance improvements that may arise by altering the network architecture during a single training cycle. In this paper, we present a framework for adapting the network size during training so as to find a good architecture automatically while possibly squeezing additional performance from that architecture. We do this by continuously adding new units to the network during training while removing the least useful units via an `2 penalty. We train the network with a novel algorithm, which we term “Adaptive Radial-Angular Gradient Descent” or AdaRad.

https://arxiv.org/pdf/1601.01705v1.pdf Learning to Compose Neural Networks for Question Answering

We describe a question answering model that applies to both images and structured knowledge bases.The model uses natural language strings to automatically assemble neural networks from a collection of composable modules. Parameters for these modules are learned jointly with network-assembly parameters via reinforcement learning, with only (world, question, answer) triples as supervision. Our approach, which we term a dynamic neural module network, achieves state-of-the-art results on benchmark datasets in both visual and structured domains.

A learned syntactic analysis (a) is used to assemble a collection of neural modules (b) into a deep neural network ©, and applied to a world representation (d) to produce an answer.

https://arxiv.org/abs/1611.09100 LEARNING TO COMPOSE WORDS INTO SENTENCES WITH REINFORCEMENT LEARNING

We use reinforcement learning to learn tree-structured neural networks for computing representations of natural language sentences. In contrast with prior work on tree-structured models in which the trees are either provided as input or predicted using supervision from explicit treebank annotations, the tree structures in this work are optimized to improve performance on a downstream task. Experiments demonstrate the benefit of learning task-specific composition orders, outperforming both sequential encoders and recursive encoders based on treebank annotations. We analyze the induced trees and show that while they discover some linguistically intuitive structures (e.g., noun phrases, simple verb phrases), they are different than conventional English syntactic structures.

https://arxiv.org/pdf/1703.01925v1.pdf Grammar Variational Autoencoder

However, generative modeling of discrete data such as arithmetic expressions and molecular structures still poses significant challenges. Crucially, state-of-the-art methods often produce outputs that are not valid. We make the key observation that frequently, discrete data can be represented as a parse tree from a context-free grammar. We propose a variational autoencoder which encodes and decodes directly to and from these parse trees, ensuring the generated outputs are always valid. Surprisingly, we show that not only does our model more often generate valid outputs, it also learns a more coherent latent space in which nearby points decode to similar discrete outputs. We demonstrate the effectiveness of our learned models by showing their improved performance in Bayesian optimization for symbolic regression and molecular synthesis.

https://arxiv.org/pdf/1606.06361v1.pdf A Probabilistic Generative Grammar for Semantic Parsing

We present experimental results showing, for a simple grammar, that our parser outperforms a state-of-theart CCG semantic parser and scales to knowledge bases with millions of beliefs.

https://arxiv.org/abs/1611.09100 Learning to Compose Words into Sentences with Reinforcement Learning

https://arxiv.org/pdf/1705.04153v1.pdf Dynamic Compositional Neural Networks over Tree Structure

Tree-structured neural networks have proven to be effective in learning semantic representations by exploiting syntactic information. In spite of their success, most existing models suffer from the underfitting problem: they recursively use the same shared compositional function throughout the whole compositional process and lack expressive power due to inability to capture the richness of compositionality. In this paper, we address this issue by introducing the dynamic compositional neural networks over tree structure (DC-TreeNN), in which the compositional function is dynamically generated by a meta network. The role of metanetwork is to capture the metaknowledge across the different compositional rules and formulate them. Experimental results on two typical tasks show the effectiveness of the proposed models.

http://bair.berkeley.edu/blog/2017/06/20/learning-to-reason-with-neural-module-networks/ Learning to Reason with Neural Module Networks

http://egrefen.com/docs/acl17tutorial.pdf Deep Learning for Semantic Composition