Texture bias in primate ventral visual cortex and deep neural network models of vision

Abstract

To accurately recognize objects despite variation in their appearance, humans rely on shape more than other low-level features. This is in contrast to leading deep neural network (DNN) models of visual recognition, which are texture biased, meaning they rely more on local texture information than global shape for categorization. Does the finding of texture bias in DNN models suggest that object representations in biological and artificial neural networks encode different types of information? Here, we addressed this question by recording neural responses from inferior temporal (IT) cortex of rhesus macaque monkeys in response to a novel object stimulus set, where we independently vary shape, texture, and pose. We observed reliable tuning for both object shape and texture in IT cortex, but texture information was more accurately decodable. We tested IT neural responses and DNN models in a two-alternative match-to-sample behavioral task. We found, to our surprise that IT neural responses consistently grouped images with matching texture over images with matching shape, demonstrating a bias towards texture information, on par with DNN models. Thus, our results suggest that the ventral visual cortex, like DNN models, provides a texture-like basis set of features, and that further neural computations, perhaps downstream of IT, are necessary to account for the shape selectivity of visual perception.