SNShin'Ya Nishida

Shin'Ya Nishida

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Shin'Ya Nishida has a total of 36 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Perturbation Tolerance of Deep Neural Networks and Humans in Material Recognition

    by Xing Liu, Masataka Sawayama, Ryusuke Hayashi, Mete Ozay, Takayuki Okatani, Shin'Ya Nishida

    We can visually recognize a variety of materials, such as fabric, glass, and metal. A promising approach to elucidate complex visual mechanisms underlying material perception is to construct and analyze artificial neural networks which recognize materials as humans do. We previously (Liu et al., VSS2016) showed that a convolutional neural network (CNN) could recognize material categories of Flickr Material Database (FMD; Sharan et al., 2013) as accurately as humans. The network was developed from a CNN model for object recognition (Simonyan & Zisserman, 2015), and fine-tuned for material recognition with FMD. To compare the performance of humans and CNNs in a broader context, the present study investigated the perturbation tolerance. Specifically, we distorted the material images of FMD by adding Gaussian noise with several levels of standard deviation, and let human participants and a CNN classify images into ten material categories. The CNN was fine-tuned for material categorization, but only with noiseless FMD images (Liu et al., VSS2016). Results showed that the categorization accuracy dropped with the noise level more steeply for the CNN than for humans, implying lower perturbation tolerance for the CNN than for humans. Furthermore, we examined whether the noise tolerance of the CNN could be improved by adding a de-noising network to the original material-recognition CNN in a cascaded manner. We obtained a major improvement to the level of human performance when using a type of de-noising network that learned to match features represented by the following CNN between the original and noisy images of FMD (Johnson et al., 2016), while we obtained only a minor improvement when using another type of de-noising network without CNN feature matching mechanisms (Isola et al., 2016). Although behavioral similarity does not imply computational similarity, these findings provide novel insights into human visual mechanism for material recognition. Meeting abstract presented at VSS 2018

  • Open Access

    Object constancy from view-based models of the face.

    by Jevgenija Beridze, Shin'Ya Nishida, Alan Johnston

    Recognising a familiar face in various poses is a challenging computational task that the human visual system can solve with remarkable ease. Current theory favours a view-based approach to object constancy over an object-centred approach but are different views of an object only related through association? We investigated whether it is possible to reconstruct a view of the face from another viewpoint. We recorded high quality, high frame rate videos of the moving human face, simultaneously from six cameras separated by 19 degrees in a horizontal arc around the face. The videos sequences from each perspective view were converted into separate image files and concatenated to form a sampled panoramic view of the face for each time point. We then expressed each multi-view image in terms of the vector field required warp the multi-view frame onto a reference multi-view frame and the resulting warped multi-view image. Principal components analysis (PCA) was used model the variation in these vectors. A PCA model can be cast as a content addressable memory. From the multi-view vector we removed the data from all but one view, setting the values for those other views to zero. We projected this partial representation back into the principal component model, regenerating the other views. We found that the reconstruction was effective, although there were some differences from the ground truth (reconstruction using all views). We found that summing up loadings for the partial representations projected into the space gave the loading for the full view representation (ground truth). We were able to substantially improve the single view reconstruction by multiplying the resulting loadings by a scale factor. The scale factor differed over views but was consistent over frames. Object constancy for the face may therefore be achieved through the mapping of relative information between view based-models. Meeting abstract presented at VSS 2015

  • Open Access

    Adaptation to the spatial smoothness of visual motion flow.

    by Kazushi Maruya, Takahiro Kawabe, Shin'Ya Nishida

    The spatiotemporal pattern of local motion vectors is a useful information source for visual recognition of moving objects, as demonstrated by our recent finding that human observers can recognize dynamic liquids and their viscosity only from the visual motion flow information (Kawabe et al., in press, Vision Research). Furthermore, the same study shows that the critical image parameter for the liquid-from-motion perception is the spatial smoothness of motion flow. However, it remains poorly understood how the brain processes the spatiotemporal pattern of motion flow, including the spatial smoothness. We have been analyzing the motion flow processing using the adaptation paradigm. Last year (Maruya & Nishida, 2014, VSS), we showed that the adaptation to a non-uniform motion field (a checkerboard pattern defined by two motion directions) significantly elevated the threshold (in terms of minimum direction difference) to detect a test motion-defined pattern. This time, we examined whether the adaptation to a specific range of spatial smoothness of motion flow would alter the apparent smoothness of the subsequently seen motion flows. The stimulus was an array of moving noise patches, each windowed by a stationary Gaussian. By rearranging patch positions of a motion field of a liquid flow, we generated motion flow fields with various levels of spatial smoothness — smooth and non-smooth fields for adaptation, and intermediate ones for test. To avoid adaptation to local motions and specific arrangements of motion flow, we updated the adaptation field to a new one every 1s. Observers’ task was to score the perceived smoothness of the motion flow in the test pattern, which was presented briefly (1s) after adaptation. The results show that perceived smoothness with test patterns is higher after adaptation to non-smooth flows than after adaptation to smooth flows. This indicates the existence of adaptable encoder of the spatial smoothness of motion flow. Meeting presented at VSS 2015

  • Open Access

    Effect of the luminance signal on adaptation-based time compression

    by Inci Ayhan, Aurelio Bruno, Shin'Ya Nishida, Alan Johnston

    Traditionally, time perception has been considered the product of a central, generic, cognitive mechanism. Recent evidence, however, has shown that high temporal frequency adaptation induces local reductions in the apparent duration of brief intervals suggesting a distributive system with modality-specific sensory components. Here, we examine the effect of the luminance signal on these adaptation-based temporal distortions. Our results show that the luminance signal is crucial to generate duration compression as the effect disappears at isoluminance and that low visibility and task difficulty at isoluminance cannot explain the discrepancy. We also demonstrate that the effects of adaptation on perceived duration are dissociable from those on apparent temporal frequency. These results provide further evidence for the involvement of the magnocellular system in the neural encoding and representation of visual time.


  • Open Access

    Spatial pooling of one-dimensional second-order motion signals

    by Kazushi Maruya, Shin'Ya Nishida

    We can detect visual movements not only from luminance motion signals (first-order motion) but also from non-luminance motion signals (second-order motion). It has been established for first-order motions that the visual system pools local one-dimensional motion signals across space and orientation to solve the aperture problem and to estimate two-dimensional object motion. In this study, we investigated (i) whether local one-dimensional second-order motion signals are also pooled across space and orientation into a global 2D motion, and if so, (ii) whether the second-order motion signals are pooled independently of, or in cooperation with, first-order motion signals. We measured the direction-discrimination performance and the rating of a global circular translation of four oscillating bars, each defined either by luminance or by a non-luminance attribute, such as flicker and binocular depth. The results showed evidence of motion pooling both when the stimulus consisted only of second-order bars and when it consisted of first-order and second-order bars. We observed global motion pooling across first-order motion and second-order motions even when the first-order motion was not accompanied by trackable position changes. These results suggest the presence of a universal pooling system for first- and second-order one-dimensional motion signals.


  • Position-variant perception of a novel ambiguous motion field

    by Andrew Rider, Alan Johnston, Shin'Ya Nishida

    Observers can extract the translational motion of Gabor arrays (static Gaussian-windowed drifting sine gratings) when the velocities of the individual Gabors are consistent with a global solution (Amano et al., 2009; doi:10.1167/9.3.4). The ambiguity in the motion of the Gabors (aperture problem) is overcome by pooling over space and orientation. We have shown that observers can perform a similar disambiguation for rotating and expanding stimuli where a large-field pooling algorithm for computing global translation would be uninformative (Rider and Johnston, 2008, ECVP). Models of global complex motion encoding typically involve three stages: local motion extraction, pooling to provide unambiguous 2D estimates of local motion and a third stage that uses these estimates to calculate the global complex motion percept. We developed a novel stimulus that is theoretically ambiguous at all three stages. The orientations of an array of Gabors are chosen to be orthogonal to their position vector relative to the centre of the array and hence form concentric ring patterns. The drift speeds are then set to be consistent with a rigid translation, but this means the arrays are also consistent with an infinite number of rotations. Subjects were shown these arrays in a number of positions in the visual field and adjusted the motion of a surrounding array of plaid patches to match the perceived motion of the Gabor array. We found that the stimuli were perceived as translating, rotating clockwise or rotating anticlockwise depending on their position in the visual field, although conventional models predict translation only. We propose an explanation in which local 1D motion estimates are used directly in computing the global rotation without being locally disambiguated. This implies a novel mechanism for the aperture problem solution that uses global rotation templates. Acknowledgement: Supported by the EPSRC and BBSRC.

  • The time marker account of cross-channel temporal judgments

    by Shin'Ya Nishida, Alan Johnston

    Summary The human sensory system, at least in its early stages, consists of multiple channels for different modalities (e.g., vision, audition) and for different attributes in each modality (color, motion). Temporal congruency is a critical factor in the binding of signals across channels, but little is known about what representations and algorithms are used for matching. We first analyze this mechanism from a general theoretical point of view and then address the specific mechanisms underlying the perception of color–motion synchrony and audiovisual simultaneity. We hypothesize that judgments about cross-channel temporal relations are based on the comparison of time markers by a mid-level perceptual process. The time markers are amodal tokens that reference salient, figural features extracted from early-level sensory signals. A temporal marker should reference the time a specific event occurs in the world rather than the time the processing of the event completes in the brain. Introduction The human sensory system has a complex architecture. It consists of multiple parallel channels for different sensory modalities (e.g., vision, audition). The channel for each sensory modality is subdivided into multiple parallel channels, each specialized for processing of different attributes (e.g., color, motion). Furthermore each channel consists of multiple serial processing stages. The transmission and processing of sensory information by neural mechanisms takes time, and the amount of time taken varies significantly across channels. For example, in monkey visual cortex, the latency of stimulus onset evoked response is about 40–100 msec in V1, 50–100 msec in MT, 70–160 msec in V4, and 90–180 msec in IT (Bullier 2001).


  • Open Access

    The sliding window of audio-visual simultaneity

    by Warrick Roseboom, Shin'Ya Nishida, Derek H. Arnold

    Humans exist in an environment wherein many unrelated events occur in close spatial and temporal proximity. Audio–visual timing experiments, however, have often examined only isolated pairs of sensory events. We therefore decided to assess how audio–visual timing perception would be shaped by the presence of an additional audio or visual event. We found that the point of subjective synchrony for a sensory event can be shifted away from the presence of other temporally proximate events. These interactions made audio–visual pairs seem unrelated, or asynchronous, at timings at which they had seemed synchronous when presented in isolation. This shows that the interval across which humans are insensitive to audio–visual asynchrony is not fixed, but dynamic, shaped by interactions between multiple sensory events. Importantly, we establish that these interactions can enhance the sensitivity of timing judgments. These interactions could therefore help to segregate unrelated sensory events across time. Such effects are likely to be common in the cluttered environments in which humans exist.


  • Open Access

    The spatial tuning of adaptation-based time compression

    by Inci Ayhan, Aurelio Bruno, Shin'Ya Nishida, Alan Johnston

    Temporal processing is traditionally dissociated from spatial vision. Recent evidence, however, has shown that adaptation to high temporal frequency (D. Burr, A.Tozzi, & M. C. Morrone, 2007; A. Johnston, D. H. Arnold, & S. Nishida, 2006; A. Johnston et al., 2008) induces spatially specific reductions in the apparent duration of subsecond intervals containing medium frequency drift or flicker. Here we examine the spatial tuning of these temporal adaptation effects. Our results show that duration compression is tightly tuned to the spatial location of the adaptor and can be induced by very narrow adaptors. We also demonstrate that the effects of adaptation on perceived duration are dissociable from those on apparent temporal frequency, which suggests early but separate influences of temporal frequency adaptation on time and speed perception.