Neural basis of perceptual surface qualities: Evidence from EEG decoding

Abstract

The human visual system can easily recognize object material categories and estimate surface properties such as glossiness and smoothness. A number of psychophysical and computational studies suggest that the material perception depends on global feature statistics of the entire image at multiple processing levels. Neural representations of such global features, which is independent of precise retinotopy, may be captured even by EEG that have low spatial resolution. To test this possibility, here we measured visual evoked potentials (VEPs) for 191 natural images consisting of 20 categories of materials. We then sought to classify material categories and surface properties from the VEPs, and to reconstruct the rich phenomenal appearance of materials themselves via neural representations of global features as estimated from the VEPs. As a result, we found that material categories were correctly classified by the VEPs even at latencies of ∼150 ms. The apparent surface properties were also significantly classified at ∼175 ms (lightness, chromaticity, and smoothness) and >200 ms (glossiness, hardness, and heaviness). In a subsequent reverse-correlation analysis, we further found that the VEPs at these latencies are highly correlated with low- and high-level global features statistics of the surface images; Portilla-Simoncelli texture statistics and style information in deep convolutional neural network (dCNN), indicating that neural activities about such global features are reflected in the VEPs that enabled successful classification of materials. To demonstrate this idea more directly, we trained deep generative models (MVAE models) that reconstruct the surface image itself from the VEPs via style information (gram matrix of the dCNN output). The model successfully reconstructed realistic surface images, a part of which were nearly indistinguishable from the original images.