Vinyl & Data

About 1.5 year ago I discovered that some people use the wrong character set in the "Rights Society" field, because characters in various character sets (Latin, Cyrillic, Greek) look alike (the so called "homoglyphs"). Some people even mixed characters from different character sets. You can read the articles I wrote here and here and I would recommend you read those first as I will be making comparisons with the data presented there. I was wondering if this situation had actually changed, or if things got worse, so I ran my scripts and found 1146 releases where I know that the wrong character set(s) were used. The distribution across the data set is as follows: Distribution of releases with using the wrong character set(s) for Rights Society Comparing it to the data from 1.5 years ago it seems to have gotten a little bit worse (though not much). As long as there is no search functionality for rights societies on the Discogs website it won't matter muc...

I decided to look into character sets again, because the last time I only looked at a few instances of "homoglyphs" (characters that look like characters in other languages, but which are different). It is recommended to first read the first article about this. I looked into rights societies again. Last time I mostly focused on Greek characters, this time I looked at where Latin characters were used where Cyrillic characters should have been used. The rights society in Russia is called RAO which in Russian is spelled РАО, which looks a lot like PAO, but it is different (first one is using the Cyrillic alphabet, second one the Latin alphabet). They are in the data as follows: Distribution of wrong entries for Russian rights society РАО What is interesting is the peak for recent releases. I don't know whether this is because more Russian releases have been added recently, or if this error has mostly been corrected for earlier releases.

A challenge in many data sets is how to properly deal with internationalization and characters. Discogs is no exception. In quite a few character sets there are characters that look exactly the same as characters in other character sets, but they aren't. Take for example the following two instances of a Rights Society name that I found in Discogs: BIEM ΒΙΕΜ To the naked eye they might look the same (depending on the font the website is using), but they actually are not the same. The first one is all ASCII, but in the second one all the characters are from the Greek alphabet. From a data point of view these characters are very different! Characters that look a lot like other characters are called homoglyphs . I wrote about this before when looking at misspellings in Czechoslovak and Czech releases (which was essentially the same problem). There are many examples of homoglyphs in the Discogs database and it can get confusing. For example, let's look at entries for a Gree...