The perception of color and material in naturalistic tasks

Perceived object color and material help us to select and interact with objects. Because there is no simple mapping between the pattern of an object’s image on the retina and its physical reflectance, our perceptions of color and material are the result of sophisticated visual computations. A long-standing goal in vision science is to describe how these computations work, particularly as they act to stabilize perceived color and material against variation in scene factors extrinsic to object surface properties, such as the illumination. If we take seriously the notion that perceived color and material are useful because they help guide behavior in natural tasks, then we need experiments that measure and models that describe how they are used in such tasks. To this end, we have developed selection-based methods and accompanying perceptual models for studying perceived object color and material. This focused review highlights key aspects of our work. It includes a discussion of future directions and challenges, as well as an outline of a computational observer model that incorporates early, known, stages of visual processing and that clarifies how early vision shapes selection performance. Media Summary Perceived object color and material help us to select and interact with objects. There is no simple mapping between the pattern of an object’s image on the retina and its physical reflectance; our perceptions of color and material are the result of sophisticated visual computations. A long-standing goal in vision science is to describe how these computations work. We have developed selection-based methods and accompanying perceptual models for studying perceived object color and material. This focused review highlights key aspects of our work and includes a discussion of future directions and challenges.