An alternative that can achieve the same goal is a feedback based approach in which the representation is formed in an iterative manner based on a feedback received from previous iteration’s output. We establish that a feedback based approach has several fundamental advantages over feedforward: it enables making early predictions at the query time, its output naturally conforms to a hierarchical structure in the label space (e.g. a taxonomy), and it provides a new basis for Curriculum Learning. Deeply Supervised Networks Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback Unsupervised Learning from Continuous Video in a Scalable Predictive Recurrent Network