LAUSR

The process of learning good representations for machine learning tasks can be very computationally expensive. Typically, the model learned on training set is leveraged to infer the labels of testing data. Interestingly, this learning and inference paradigm, however, is quite different from the typical inference scheme of human biological visual systems. Essentially, neuroscience studies have shown that the right hemisphere of the human brain predominantly makes a fast processing of low-frequency spatial signals, while the left hemisphere more focuses on analyzing high-frequency information in a slower way. And the low-pass analysis helps facilitate the high-pass analysis via feedback. Inspired by this biological vision mechanism, this article explores the possibility of learning a layer-skippable inference network. Specifically, we propose a layer-skippable network that dynamically carries out coarse-to-fine object categorization. Such a network has two branches to jointly deal with both coarse and fine-grained classification tasks. The layer-skipping mechanism is proposed to learn a gating network by generating dynamic inference graphs, and reducing the computational cost by detouring the inference path from some layers. This adaptive path inference strategy endows the deep networks with dynamic structures, making the networks enjoy greater flexibility and larger capacity. To efficiently train the gating network, a novel ranking-based loss function is adopted. Furthermore, the learned representations are enhanced by the proposed top-down feedback mechanism and feature-wise affine transformation, individually. The former one employs features of a coarse branch to help the fine-grained object recognition task, while the latter one encodes the selected path to enhance the final feature representations. Extensive experiments are conducted on several widely used coarse-to-fine object categorization benchmarks, and promising results are achieved by our proposed model. Quite surprisingly, our layer-skipping mechanism improves the network robustness to adversarial attacks.