Using image processing to automatically detect labels (part 3)

Finally getting to the meat of using image processing to detect labels automatically. For background information you should first read part 1 and part 2 before reading this.

Just computing a histogram won't get me very far. For it to be useful I need to compare it to something. As said what I will do is to compare one part of the image (where I think the label is) with the rest of the image, minus the center hole as that isn't part of the record. The idea is that the overlap between the two will be minimal, but possibly not in the cases where the vinyl and the label have the same colour, or a similar colour.

With OpenCV there are built in functions to compare these and basically what I did is follow this blog post at PyImageSearch about comparing histograms and used it on my test images.

After computing the histograms (32 buckets) I computed the four types of histogram comparisons from the PyImageSearch post for the red, green and blue channels, as well as the full image. It is much better explained there than what I do, but the four types are:

1. correlation
2. chi squared distance
3. intersection
4. Bhattacharyya distance

The numbers are interpreted in a different way (and I don't know the fine details), but the important thing to keep in mind for correlation and intersection: the lower the score, the more different the histograms are. For chi squared and Bhattacharyya distance it is the opposite: the higher the score, the more different the histograms are.

Sleeve histogram comparison

For the sleeve I am expecting a high correlation, as the colours of the two parts that are compared are very similar.

Red channel

1. correlation: 0.8926749724715063
2. chi squared: 120660.19493757954
3. intersection: 74872.0
4. Bhattycharyya distance 0.29158852379442657

Green channel

1. correlation: 0.6275938864980654
2. chi squared: 181519.2086741701
3. intersection: 63567.0
4. Bhattycharyya distance: 0.35726567943462395

Blue channel

1. correlation: 0.7437035346944266
2. chi squared:190204.78252395047
3. intersection: 73194.0
4. Bhattycharyya distance: 0.32126113326366795

Full image

1. correlation: 0.5584962418370338
2. chi squared: 32.475552958421424
3. intersection: 2.4023326691039983
4. Bhattycharyya distance: 0.5647389844577926

Those numbers only make sense when comparing with the results of the label histogram comparisons.

Label histogram comparison

For the label I am expecting a much lower correlation, although the label itself is quite dark as well, which might make things more difficult. Again: the results are for the red, green and blue channels, as well as the full image.

Red channel

1. correlation: 0.04524780052542104
2. chi squared: 435578.4556929312
3. intersection: 17567.0
4. Bhattycharyya distance: 0.825618002673244

Green channel

1. correlation: 0.028391995259706947
2. chi squared: 437614.5409754197
3. intersection: 17194.0
4. Bhattycharyya distance: 0.8394928259148619

Blue channel

1. correlation: 0.04220587964200443
2. chi squared: 490151.69460585836
3. intersection: 15318.0
4. Bhattycharyya distance: 0.8624890522162065

Full image

1. correlation: 0.12186543341765493
2. chi squared: 11.862898455687839
3. intersection: 0.15000400247754442
4. Bhattycharyya distance: 0.8706885323226409

As you can see all comparisons say that the label I cut from the image is showing very little overlap with the rest of the image, meaning that this method seems to work and I can tell from just these numbers that the latter one (with the label) is most likely the image with the label (as expected).

The test image I used is of course the ideal situation, so I wondered how well my method works for other images that I downloaded from Discogs. I will dig into that on my next post.