Image Recognition Methods

Breaking out Features

Sliding Window: A common way of breaking up features in an image, such as characters or words, is to use a Sliding Window to "cut out" parts of the picture, which are then compared against previously trained sample data of good places to split characters vs bad places. The window is then moved by some step size and that is repeated all over the image. Then that is repeated again with a slightly larger window. This is very costly, but works well.

Outline Tracing: Joseph Watson shared

"an algorithm used by a simple system intended to read books. In this system, the idea was to explore an image consisting of only black and white pixels to determine what printed letters might be found there. Needless to say, the pixels were much smaller than the printed letters.

The first part of the algorithm consists simply of scanning around in the white image until a black pixel is found. Then the following rules are applied with each successive pixel check. Each move is made by moving only in X or in Y, never along a diagonal.
1. If you have turned the same way three times in a row, turn the other way.
2. If the sampled pixel is black, turn right.
3. If the sampled pixel is white, turn left.

What this algorithm accomplishes is that it traces around the perimeter of the black character shape. When the trace action around a character begins to retrace the same pixels, it is terminated, the character was then recognized by the maximum and minimum points reached in the X and Y axes and the order in which they are found."

Objectness Measures: Attempts to find general areas where an object of any type is likely vs areas of background, noise, etc... This then guides the use of sliding windows and systems trained to recognize a specific type of object.

• http://groups.inf.ed.ac.uk/calvin/objectness/
• http://groups.inf.ed.ac.uk/calvin/Publications/alexe12pami.pdf "While object detectors are specialized for one object class, such as cars or swans, in this paper we define and train a measure of objectness generic over classes. It quantifies how likely it is for an image window to cover an object of any class. Objects are standalone things with a well-defined boundary and center, such as cows, cars, and telephones, as opposed to amorphous background stuff, such as sky, grass, and road"

This is done through a series of test, taking cues of different types from the image, which are combined in a Bayesian framework

• Multi-scale Saliency (MS): "For every scale s  {16, 24, 32, 48, 64} and channel c we rescale the image to s × s and then compute MS(w, ¸s MS) using one integral image [10] (indicating the sum over the saliency of pixels in a rectangle)."
• Color Contrast (CC): "We convert the image to the quantized LAB space 4×8×8 and then compute CC(w, ¸CC ) using one integral image per quantized color."
• Edge Density (ED): "We rescale the image to 200 × 200 pixels and then compute ED(w, ¸ED) using one integral image (indicating the number of edgels in a rectangle)."
• Superpixels Straddling (SS): "
• Location and Size (LS)

See also:

• http://www.qelzal.com/ is doing impressive work with seperating out objects in a complex realtime image using motion to prevent collisions.
• https://www.coursera.org/lecture/convolutional-neural-networks/yolo-algorithm-fF3O0 How YOLO: You Only Look Once works
• https://plus.google.com/u/0/+JamesNewton/posts/Wwq7k3CYMxx Extract moving objects from successive frames, build a 3D model to predict future frames, focus on the different between predictions and actual frames to improve the model.
• https://www.ted.com/talks/pawan_sinha_on_how_brains_learn_to_see/transcript?language=en Motion is critical to the development of the ability to separate objects in humans.
• https://www.youtube.com/watch?v=uihBwtPIBxM How edge detection filters work. Including finding the angle of the edge, which can help to find objects in the picture.