in an interview with lee gomes, michael jordan (no, not the basketball player, the ieee fellow) stated that there are still many challenging, unsolved problems in computer vision, even though deep learning has given us a great tool in terms of certain types of image classification. although michael jordan had a thing or two to say about the interview that was published, it doesn’t appear as if he rebutted any of the statements made regarding computer vision.
as mentioned in the interview, even though deep learning can be used to classify the objects in some images, a method has not yet been found to enable computers to establish the full relation between objects. computers can also not fully comprehend the use of objects. for example, humans can easily find a place to sit even though there might not be traditional chairs in most situations. computers, however, find this considerably more difficult. (there has been some research into this, such as the article “what makes a chair a chair?” by h. grabner, j. gall and l. van gool)
what we often don’t consider is how the human brain is trained to interpret what is seen. even something as simple as recognising objects in a 2d image seems to be a learnt skill. in other words, even if we knew what objects looked like, and could recognise them all around us, the ability to recognise them on a photo seems to be a learnt skill. this is the conclusion that the author of “smart moves”, dr c. hannaford, a neurophysiologist and educator, has come to. in her book, she explains how she had visited a tribe that had never before been exposed to 2d images. when showing them pictures of a mountain range, they were completely unable to ‘see’ the mountain range (even though they were very familiar with mountain ranges). how their brains process visual information was not adapted to recognising objects in a 2d picture. just as gestalt theory also reveals, therefore, what we see is not necessarily what is present.
an example of this is the blue-black vs white-gold dress that made its rounds on social media ‘recently’, and can be seen in figure 1.
not everyone sees this dress the same. some people immediately see it as a black-blue dress, whilst others see it as a white-gold dress. in essence, taken from this article that explains it in detail, what happens is that the sharp outside light may cause your brain to think that you are looking at the shadowed version of the dress, and so compensates, and you see a white-gold dress. however, when people see it as a black-blue dress, the brain realises that the dress itself is also well-lit, and that the bright light isn’t to be taken into account when looking at the dress itself.
from this can be seen that our brains consider the surrounds when processing colour. whether that surrounds is something specific as the actual colour that surrounds the one in question, or whether it is from more of an idea such as a light-source or shadow.
consider the objects in figure 2.
it appears in figure 2 as if the middle rectangle in the bigger rectangles are different colours. however, they are actually the same colour, as can be seen in figure 3. this effect is known as simultaneous contrast, and refers to how colours from different objects affect each other .
the simultaneous contrast effect is most intense when the colours that interact with each other are complementary colours. complementary colours are the colours that are directly opposite one another on the colour wheel (as shown in figure 4).
this effect, together with colour theory and the colour wheel is used in many different applications. obvious places where it is used is in areas such as art, logo design and in the design of websites. a less obvious place it is used in is in choosing the colour of life rafts. as can be seen in the colour wheel in figure 4, orange is the complementary colour to blue. as the oceans appear blue in general, life rafts’ colour has been chosen to be orange, since it is the colour that will stand out the most on blue.
as mentioned before, the idea of a shadow can also have an effect on how we perceive colours. as an example of this, consider the images in figure 5.
in the image to the left, it appears as if the blocks labelled ‘a’ and ‘b’ are of different colours. however, in the image to the right, it can be seen that the two blocks are actually the same colour. part of the reason for this ‘trick’ is due to our brains seeing the shadow thrown by the green cylinder and in so doing, adapting our perception of the colour of block ‘b’. for the full reasoning behind this example, see this article.
the human brain is a complex system with extremely complex interactions. above are some simple examples of how our brains adapt to the information that is perceived. it is also adaptions like these that makes teaching a computer to gather the same information from of a scene as a human, more difficult.