Possible significance of spatial heterogeneities of local visual features for face perception

Introduction

The issue of visual image formation has a long history. Until recently, there hasn’t been proposed a theory that could explain everything on that matter. In visual neuroscience 3 points of view on image formation have prevailed. According to the first one, an image is a holistic description in which the most typical object of its class is taken as a standard. This is called ‘a template theory’. The second point of view also came from a holistic image description, but takes an average description of an object of its class as standard. This is called ‘a prototype theory’. The third theory assumes that every image can be described using the summation of its features. This is called ‘a feature theory’. Until now it’s remained unknown how could the invariance of holistic descriptions be provided and what could be presented as separation features.

No matter what point of view is closer to the truth, it is now stated for sure that the initial visual processing is a parallel local description of an input, which results in breaking a scene into a quantity of fragments, which are known as primitives. These are the gradients of luminance of various localization, orientation and spacial frequency. The operation begins in retina and ends in the visual cortex.

But this is only the start of visual processing. Image formation inevitably includes grouping of primitives, attributed to a single object. At first, the theory of the integration of features, according to which the mechanism of bounding is selective attention¹, was popular. Lately though, the number of tasks had been described to solve which spacial grouping implements preattentively. It is, for example, perceiving of second order movement² and texture separation³. These operations can be done by the so-called ‘second-order mechanisms’ that preattentively (following a certain rule) bind outputs of first-order mechanisms (the linear striate neurons)^3–5. The following studies proved an existence of such mechanisms and determined its properties^6–9. Considering the second order movement to be a laboratory phenomenon, the ability to quickly divide textures is very important in everyday life.

Is the role of the second order mechanisms limited by the task of texture separation? Considering that these mechanisms can distinguish special modulations of local features, the attempts were made to establish the considerable role of this information in perceiving complex scenes and objects. Analysis of the natural pictures showed that notably the first and the second order features spatially overlap^10,11. As a result the second order features were determined to delete the ambiguities from interpretation of the change of luminance (the first order features)^12,13.

Meanwhile, it’s quite rational to assume that these modulations could contain important information concerning an object’s forms and their details. Considering this, the goal of our study – to determine whether information concerning spacial heterogeneity – could be useful in identifying all the images and faces among the ‘not faces’ in particular.

Initiating the task ahead, we cannot ignore the fact that the early visual processing is operated by the system of parallel paths that are set to different spacial frequencies^14–17. It is known that, when it comes to tasks of identifying faces, these frequencies are not the same. There’s also the probability that the results of the processing by particular spacial channel are united in certain combinations.

Preparing the test images, we followed the assumptions about the organization of the second order mechanisms, which are displayed in the ‘filter-rectify-filter’ model¹⁸. According to it, the outputs from adjacent linear filters (the first-order filters) with the same frequency and orientation set are united by the certain algorithm in the second-order filters. In other words, in case of the second-order mechanisms, those filters that differ only in localization in field of view unite. Such filters with different resolutions pass those regions of image that differ in heterogeneity to contrast, orientation and spacial frequency. We followed the assumption that these regions, due to their heterogeneity, contain the important information and could be viewed as the ‘regions of interest’.

Methods

Results

The information was allowed to pass only through one ‘window’, centered relative to the face (Figure 1A,C) when the initial (real) images were processed by the SOFs tuned to the lowest frequency of the carrier (0.5 cpd) (we denote these filters as F1). The result of the processing is shown in Figure 1B,D. Looking at the presented images the observers determined the gender in 87.9% and gave a more general response “a face” only in 11.4%.

Figure 1. The face processing at the carrier frequency 0.5 cpd.

A, C – the initial images. The circles are the windows through which the filtered image is allowed to pass. B, D – the test (processed) images. There are only ‘the regions of interest’ at the frequency of filtering.

If the initial images were processed by the SOFs tuned to a higher frequency of the carrier (1 cpd) (F2) the information from only a part of a face could be transmitted through one window (Figure 2A,C). The information was transmitted through 2 windows because there were 2 local maximums at the F2 outputs. The result of the processing may be seen in Figure 2B,D. We should mention that the observers’ results were a little worse than the previous ones. Now the gender was defined in 75.7% and the response ‘a face’ was given in 20%.

Figure 2. The face processing at the carrier frequency 1 cpd.

The SOFs spatially integrating the higher frequency signals (2 cpd) (F3) passed information through the windows which size was about 0.25 of the face (Figure 3A,C). As a result, the test images were formed, shown in Figure 3B,D. In this case the performance was again improved. The observers determined the gender in 94.3%.

Figure 3. The face processing at the carrier frequency 2 cpd.

A further reducing of the SOFs‘ size while increasing the carrier frequency led only to deterioration of the performance (Figure 4, Figure 5, Figure 6).

Figure 4. The face processing at the carrier frequency 4 cpd.

Figure 5. The face processing at the carrier frequency 8 cpd.

Figure 6. The face processing at the carrier frequency 16 cpd.

All obtained results are summarized in the Table 1.

Integration of the information which was extracted by three SOFs from the low-frequency half of the spectrum (F1+F2+F3) (Figure 7A) did not improve the performance comparing with using only F3 (92.1% versus 94.3% respectively).

But this operation (F1+F2+F3) makes it to identify the person if the initial image is a familiar face²⁵ (Figure 7B).

Figure 7. The results of windows‘ combination at the frequencies 0.5, 1, and 2 cpd.

A – the unfamiliar face from our experiment, B – the familiar face.

Discussion

The issue of ‘face perceiving’ can be divided conditionally into two parts: the allocation of the useful information (the reduction of redundancy) and the building a sizer of the selected information (the recognition). Our research concerns first part of this issue.

Among all the known algorithms of finding the ‘regions of interest’ only the small part could be viewed as neural^26–32. These algorithms can be divided into the modular and the net. In case of the first ones the weight is given, in case of the second ones it is formed during the training of the net. The approach used by us in our work is based on the modular architecture of the earlier levels of processing, finishing with the automatic allocation of the useful information from an incoming image.

In our research the first order filters formed six copies of an incoming image with different definition, and the second order filters were used as windows, which were at the maximum level of difference from surroundings by contrast, orientation or spatial-frequency.

The received results show that the identification of a face is more effective on the carrier frequency 2 cpd. This conforms with the other authors’ data that showed that the identification of faces is faster and more precise if the frequencies of the middle range are used^{16,17,33–37}. So what is new in our information?

We’ve shown that not the whole face is informative, but only its regions with spatial heterogeneity. Meaning, in task of detection of a human face and the definition of the gender information of a whole face is significantly redundant. It does not contradict with the data that processing of a human face is holistic^38,39. It’s just that the integrated information concerning its most informative areas could be enough for the holistic description of it.

If 100% would be the sum of all second-order filters activated by our images we can presume that at frequency of 2pcd the volume of selected information would be 1%. Reminder, this amount of information is enough to determine gender confidently.

To identify a familiar face, determination of regions of interest on one of the frequencies is not enough. The summation of the low-frequency half of spectrum is necessary at least (Figure 8). High-frequency information is not crucial for the gender determination and identification, but useful in perceiving details and delicate differentiation.

Figure 8. The result of the familiar faces‘ processing at the carrier frequency 2 cpd.

At the left – Einstein, at the right – princess Diana.

Thus been said, we chose areas of image that differ most from the surroundings in contrast, orientation and spacial frequency to be the most informative. The second order filters that we used form maps of convexity for every carrier frequency. As a result we have the “embedded maps”. At the lowest of the used frequencies one of the filters selects face as a whole. The following maps select areas of a face that are smaller and smaller. If the object approaches or retreats, so that its size changes in a certain range, the embedded maps stay the same. The difference would be only if the object approaches the same regions of interest would be allocate by second-order filters, which are tuned at lower carrier frequency, and if it retreats – at the higher one.

The smaller the window, which transmits the information, the higher its definition is. As a result, the same portions of information are transmitted through every window. If size of an object or it’s turns is changing, general capacity and nature of information transmitted by second-order filters stays the same.

Conclusions

Note that the information allocated with the given algorithm is useful for perceiving faces, the following hypothetical model of second-order filters image formating can be proposed. The face describing is simultaneous in a number of definition levels. At the relatively low level a face is described as a whole. With the higher definition transmits information concerning large objects of a face. Every higher level describes even smaller details. Wherein the given information allocates and transmits with parallel frequency channels. As a result, a hierarchical description of a face formed parallel, according to automatic algorithm. Wherein, the system of decision making can not use all available information. Elaborations will be made until specific visual task will be resolved.

The obtained results allow to assume that second-order filters are suitable candidates to the role of mechanism of convexity map formating, and information they allocate can be used to form a face image.

Data availability

F1000Research: Dataset 1. Frequencies of different types of the responses, 10.5256/f1000research.5975.d41499⁴⁰