Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
context [2017/12/21 11:14]
admin
context [2019/01/16 17:11] (current)
admin
Line 33: Line 33:
  
 https://​arxiv.org/​pdf/​1703.06408v1.pdf Multilevel Context Representation for Improving Object Recognition https://​arxiv.org/​pdf/​1703.06408v1.pdf Multilevel Context Representation for Improving Object Recognition
 +
  
 This paper This paper
Line 166: Line 167:
 selective to object types, their shapes and colors. The second stream has minicolumns with position and selective to object types, their shapes and colors. The second stream has minicolumns with position and
 orientation selectivity. ​ orientation selectivity. ​
 +
 +https://​openreview.net/​forum?​id=rkfOvGbCW Memory-based Parameter Adaptation
 +
 +Our method, Memory-based Parameter Adaptation, stores examples in memory and then uses a context-based lookup to directly modify the weights of a neural network. Much higher learning rates can be used for this local adaptation, reneging the need for many iterations over similar data before good predictions can be made. As our method is memory-based,​ it alleviates several shortcomings of neural networks, such as catastrophic forgetting, fast, stable acquisition of new knowledge, learning with an imbalanced class labels, and fast learning during evaluation. We demonstrate this on a range of supervised tasks: large-scale image classification and language modelling.
 +
 +
 +https://​arxiv.org/​abs/​1802.05365 Deep contextualized word representations
 +
 +We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state of the art across six challenging NLP problems, including question answering, textual entailment and sentiment analysis. We also present an analysis showing that exposing the deep internals of the pre-trained network is crucial, allowing downstream models to mix different types of semi-supervision signals.
 +
 +
 +https://​arxiv.org/​pdf/​1803.05897v1.pdf Contrasting information theoretic decompositions of
 +modulatory and arithmetic interactions in neural information
 +processing systems
 +
 +The decompositions that we report here show that contextual modulation
 +has information processing properties that contrast with those of all four simple arithmetic
 +operators, that it can take various forms, and that the form used in our previous studies of artificial
 +nets composed of local processors with both driving and contextual inputs is particularly
 +well-suited to provide the distinctive capabilities of contextual modulation under a wide range
 +of conditions. We argue that the decompositions reported here could be compared with those
 +obtained from empirical neurobiological and psychophysical data under conditions thought to
 +reflect contextual modulation. That would then shed new light on the underlying processes involved.
 +Finally, we suggest that such decompositions could aid the design of context-sensitive
 +machine learning algorithms.
 +
 +https://​arxiv.org/​abs/​1804.01622v1 Image Generation from Scene Graphs
 +
 +To truly understand the visual world our models should be able not only to recognize images but also generate them. To this end, there has been exciting recent progress on generating images from natural language descriptions. These methods give stunning results on limited domains such as descriptions of birds or flowers, but struggle to faithfully reproduce complex sentences with many objects and relationships. To overcome this limitation we propose a method for generating images from scene graphs, enabling explicitly reasoning about objects and their relationships. Our model uses graph convolution to process input graphs, computes a scene layout by predicting bounding boxes and segmentation masks for objects, and converts the layout to an image with a cascaded refinement network. The network is trained adversarially against a pair of discriminators to ensure realistic outputs. We validate our approach on Visual Genome and COCO-Stuff, where qualitative results, ablations, and user studies demonstrate our method'​s ability to generate complex images with multiple objects.
 +
 +https://​arxiv.org/​abs/​1803.08493v5 Context is Everything: Finding Meaning Statistically in Semantic Spaces
 +
 +This paper introduces Contextual Salience (CoSal), a simple and explicit measure of a word's importance in context which is a more theoretically natural, practically simpler, and more accurate replacement to tf-idf. CoSal supports very small contexts (20 or more sentences), out-of context words, and is easy to calculate. A word vector space generated with both bigram phrases and unigram tokens reveals that contextually significant words disproportionately define phrases. This relationship is applied to produce simple weighted bag-of-words sentence embeddings. This model outperforms SkipThought and the best models trained on unordered sentences in most tests in Facebook'​s SentEval, beats tf-idf on all available tests, and is generally comparable to the state of the art. This paper also applies CoSal to sentence and document summarization and an improved and context-aware cosine distance. Applying the premise that unexpected words are important, CoSal is presented as a replacement for tf-idf and an intuitive measure of contextual word importance.
 +
 +https://​arxiv.org/​abs/​1804.07983v1 Context-Attentive Embeddings for Improved Sentence Representations
 +
 +While one of the first steps in many NLP systems is selecting what embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce a novel, straightforward yet highly effective method for combining multiple types of word embeddings in a single model, leading to state-of-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new insight into the usage of word embeddings in NLP systems.
 +
 +https://​arxiv.org/​abs/​1712.09926 Rapid Adaptation with Conditionally Shifted Neurons
 +
 +We describe a mechanism by which artificial neural networks can learn rapid adaptation - the ability to adapt on the fly, with little data, to new tasks - that we call conditionally shifted neurons. We apply this mechanism in the framework of metalearning,​ where the aim is to replicate some of the flexibility of human learning in machines. Conditionally shifted neurons modify their activation values with task-specific shifts retrieved from a memory module, which is populated rapidly based on limited task experience. On metalearning benchmarks from the vision and language domains, models augmented with conditionally shifted neurons achieve state-of-the-art results.
 +
 +https://​arxiv.org/​abs/​1612.08083v3 Language Modeling with Gated Convolutional Networks
 +
 +https://​arxiv.org/​abs/​1711.06640v2 Neural Motifs: Scene Graph Parsing with Global Context
 +
 +Our analysis motivates a new baseline: given object detections, predict the most frequent relation between object pairs with the given labels, as seen in the training set. This baseline improves on the previous state-of-the-art by an average of 3.6% relative improvement across evaluation settings. We then introduce Stacked Motif Networks, a new architecture designed to capture higher order motifs in scene graphs that further improves over our strong baseline by an average 7.1% relative gain. Our code is available at github.com/​rowanz/​neural-motifs.
 +
 +https://​arxiv.org/​abs/​1808.08493 Contextual Parameter Generation for Universal Neural Machine Translation
 +
 +https://​arxiv.org/​abs/​1809.01997 Dual Ask-Answer Network for Machine Reading Comprehension
 +
 +https://​openreview.net/​forum?​id=BylBfnRqFm CAML: Fast Context Adaptation via Meta-Learning
 +
 +https://​arxiv.org/​pdf/​1810.03642v1.pdf CAML: Fast Context Adaptation via Meta-Learning
 +
 +
 +CAML: Fast Context Adaptation via Meta-Learning
 +Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, Shimon Whiteson
 +(Submitted on 8 Oct 2018 (this version), latest version 12 Oct 2018 (v2))
 +We propose CAML, a meta-learning method for fast adaptation that partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared parameters that are meta-trained and shared across tasks.
 +
 +https://​arxiv.org/​abs/​1810.08135 Contextual Topic Modeling For Dialog Systems
 +
 +Our work for detecting conversation topics and keywords can be used to guide chatbots towards coherent dialog.
 +
 +https://​www.nature.com/​articles/​s41467-018-06781-2 Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences
 +
 +https://​arxiv.org/​abs/​1901.03415v1 Context Aware Machine Learning
 +
 +The embedding of an observation can also be decomposed into a weighted sum of two vectors, representing its context-free and context-sensitive parts
 +
 + new architecture for modeling attention in deep neural networks. More surprisingly,​ our new principle provides a novel understanding of the gates and equations defined by the long short term memory model, which also leads to a new model that is able to converge significantly faster and achieve much lower prediction errors. Furthermore,​ our principle also inspires a new type of generic neural network layer that better resembles real biological neurons than the traditional linear mapping plus nonlinear activation based architecture. Its multi-layer extension provides a new principle for deep neural networks which subsumes residual network (ResNet) as its special case, and its extension to convolutional neutral network model accounts for irrelevant input (e.g., background in an image) in addition to filtering.
 +
 +