Article summary of Untangling invariant object recognition by DiCarlo & Cox - Chapter

This article provides a graphical perspective on the arithmetic challenges of object recognition. It also looks at which neuronal population is responsible for the representation of objects. Our daily activities are accompanied by quick and accurate recognition of visual stimuli; this way we can recognize thousands of objects within seconds. However, which brain mechanisms are involved in this process is still unknown.

Object recognition is defined as being able to accurately distinguish objects or categories from all kinds of possible stimuli. This is done via the retina through an identification-prescribing transformation. Object recognition is difficult for a variety of reasons. The main reason is that every object can produce an infinite number of different images on the retina, while this image is still recognized from every angle. This is also called the invariance problem: the fact that we never see exactly the same thing twice and can nevertheless recognize things.

Arithmetic processes

When solving a recognition task, someone must use internal neuronal representations. These internal neural representations come from the visual vision and from this a choice is made by the brain. The brain must use a selection function to distinguish between which neurons are fired when object A is presented and which is not. Somewhere in the brain there are the right neurons that react to something and you have to filter it out. The central question in this process remains: which format of neurons represents the choice and which decision functions belong to that representation?

You can see the above problem from two sides. On the one hand, object recognition is a problem for finding complex decision functions and on the other it is a problem for finding operations that progressively transform from the retinal representation in the form of a new representation, followed by decision functions. The latter vision can be well used when investigating the architecture of the visual system (especially the ventral path).

Object recognition is difficult

Our eyes fixate an average of 300 ms on the world and then move on. During every glimpse, a visual image is already made and stored using at least 100 million cells. Such a representation can be seen as high-dimensional. An example of a low dimensional representation is a face.

Then it is about a fixed object that can be seen in many different ways. How you can see an object is called a manifold. Different objects have different manifolds.

The manifolds of all different objects are crumpled together in the brain. This means that the retina does not immediately recognize what we see, but it does pass on the information that we need to make a choice of what we see.

You can see the brain recognition mechanism as a transformation of the incoming visual representation that is easy to build into recognition. However, it is not possible to decode how recognition works.

The ventral visual path

This path translates the manifolds into objects. The ventral path is as follows: V1 to V2 to V4 to IT. Gross's studies have shown that the IT has the most specific complex neurons. The neurons there are likely to cause object recognition, because these neurons respond specifically to certain forms and are reasonably insensitive to changes in object position.

Recognition is not a result of performance, but of how strong the visual representation is in the IT cortex. This also means that the manifolds are less mixed up in the IT cortex. This also means that the V1 cortex is still very mixed up when it comes to the manifolds (just like in the retinal representation). In short: the ventral path ensures that objects are recognized by unraveling the manifolds. It is not yet known how this happens.

Meanwhile, there are already several ideas about and investigations into the process of the ventral path. Some neurophysiologists have focused on characterizing tolerance in IT neurons towards some objects. This is the same as object tangling. Other research is aimed at understanding the characteristics of shape dimensions. These studies are important for defining complex characteristics of the ventral visual pathway for neuronal tuning, which is related to manifold untangling.

The perspective of the tangling object leads to a different approach. Individual IT neurons are not expected to be responsible for the recognition, but for population representations. In addition, this perspective assumes that the immediate preparation of a goal determines how well the ventral visual path causes detangling of manifolds. This perspective offers a better way to make computer models, because populations can be more meaningful than individual neurons.

In addition, the perspective states that a focus on the cause of the tangling is better than focusing on the characteristics or forms where something reacts. Finally, with this perspective, hypotheses can be tested, which can lead to new biological hypotheses.

Flattened manifolds

By flattening manifolds one can perhaps see what happens. We are looking for transformations that cause a manifold to be flattened without interfering with others. This allows the correct neurons to be identified. At IT-level, the detangling of object manifolds results in the folding of each manifold into one point. This suggests that detangled IT-representations not only directly provide object recognition, but also recognize other tasks such as position, location and size. IT-neurons therefore have large limited receptive fields. Here the limitation works to the advantage. The detangling of manifolds can be viewed with neuronal images. Yet this is very difficult to see.

Further analysis shows that the V1 cortex sees the world through a narrow hole and that the V2 can do the same. After that the recognition gets better and better. There are three consistent mathematical ideas that allow the detangling of the above physiology:

• Idea 1: the visual system projects incoming information to higher dimensional places so that the data spreads more in the space.

• Idea 2: neuronal sources are present at every stage that correspond to the distribution of visual information from the real world.

• Idea 3: time implicitly ensures the supervision of manifold flattering.