Skip to main content

Finding promising releases for fixing (part 1)

So far these blogs have been about detecting wrong information in releases that people entered into the database, but there are also other errors that are much harder to find, namely detecting if a release is missing information and are incomplete.

Many releases in Discogs are missing information. One of the things we did for a while was track new Spanish that were added, looking at the pictures (if any) and then adding the missing information, such as the depósito legal, or a rights society. But this does not scale, and it is also quite labour intensive, and it is not easy to remember by heart which releases have already been checked. So I want to optimize to success: which releases will likely be the easiest to fix?

I ran a test, where I confined my search to:
  1. releases from Spain
  2. 7" releases (vinyl, flexi-disc)
with the specific goal to fill in the depósito legal field(s) and other things like rights society fields (if not already there).

The following was done: first I looked at which releases are from Spain. In the dump covering March 1 - 31 2019 there are 298,468 releases from Spain. I then looked at the XML from the dump and discarded all the releases that have no images. Whether or not there is an image is stored in the image as in the following example:

<release id="7146474" status="Accepted">
  <images>
    <image height="602" type="primary" uri="" uri150="" width="600"/>
    <image height="606" type="secondary" uri="" uri150="" width="600"/>
    <image height="603" type="secondary" uri="" uri150="" width="600"/>
    <image height="601" type="secondary" uri="" uri150="" width="600"/>
  </images>



If the images element is empty the entry can be skipped, as there won't be any useful information to extract from it.

I also only looked at the 7" releases, that did not have a Depósito Legal field and also could not be found in the list of smells (indicating a possible or faulty Depósito Legal, although this is not fool proof). The result: 36,008 releases (and remember, this is just 7" releases, no LPs, CDs, or other releases that also have this number), distributed like this:


I then sorted these in reverse order, with the releases with the most images first and started working my way through. As it turns out, this is a very effective method to prioritize releases that need to be fixed, but only as long as there are enough images: when going through just a handful of releases with 4 images most of the releases (my guess: 3 out of 5) had a depósito legal value that could be added. For releases that only come with a single image (a bit less than 18,000) the success rate plummets drastically, as most of the images are simply the front sleeve (and this is why Discogs should start labeling the uploaded images as well).

That being said, not every release I looked at was suitable, because:

  • no depósito legal present on the release, because it was either too old, actually from another country, or simply didn't have it
  • the pictures did not have enough detail or were very fuzzy
  • stickers on labels blocking the depósito legal
  • impossible to read data, even with a high enough detailed scan, such as this Randy Newman release
I am very sure that this method can be expanded to other information as well such as Czechoslovak license numbers or Greek license numbers. But before I do that there are 36,000 potential depósito legal numbers to add!

Comments

Popular posts from this blog

SID codes (part 1)

One thing that I only learned about after using Discogs is the so called Source Identification Code, or SID. These codes were introduced in 1994 to combat piracy and to find out on which machines a CD was made. It was introduced by Philips and adopted by IFPI, and specifications are publicly available which clearly describe the two available SID codes (mastering SID code and mould SID code). Since quite a few months Discogs has two fields available in the " Barcode and Other Identifiers " (BaOI) section: Mould SID code Mastering SID code A few questions immediately popped up in my mind: how many releases don't have a SID field defined when there should be (for example, the free text field indicates it is a SID field)? how many releases have a SID field with values that should not be in the SID field? how many release have a SID field, but a wrong year (as SID codes were only introduced in 1994) how many vinyl releases have a SID code defined (which is impossi

SPARS codes (part 1)

Let's talk about SPARS codes used on CDs (or CD-like formats). You have most likely seen it used, but maybe don't know its name. The SPARS code is a three letter code indicating if recording, mixing and mastering were analogue or digital. For example they could look like the ones below. There is not a fixed format, so there are other variants as well. Personally I am not paying too much attention to these codes (I simply do not care), but in the classical music world if something was labeled as DDD (so everything digital) companies could ask premium prices. That makes it interesting information to mine and unlock, which is something that Discogs does not allow people to do when searching (yet!) even though it could be a helpful filter. I wanted to see if it can be used as an identifier to tell releases apart (are there similar releases where the only difference is the SPARS code?). SPARS code in Discogs Since a few months SPARS is a separate field in the Discogs

Country statistics (part 2)

One thing I wondered about: for how many releases is the country field changed? I looked at the two most recent data dumps (covering February and March 2019) and see where they differed. In total 5274 releases "moved". The top 20 moves are: unknown -> US: 454 Germany -> Europe: 319 UK & Europe -> Europe: 217 unknown -> UK: 178 UK -> Europe: 149 Netherlands -> Europe: 147 unknown -> Europe: 139 unknown -> Germany: 120 UK -> US: 118 Europe -> Germany: 84 US -> UK: 79 USA & Canada -> US: 76 US -> Canada: 65 unknown -> France: 64 UK -> UK & Europe: 62 UK & Europe -> UK: 51 France -> Europe: 51 Saudi Arabia -> United Arab Emirates: 49 US -> Europe: 46 unknown -> Japan: 45 When you think about it these all make sense (there was a big consolidation in Europe in the 1980s and releases for multiple countries were made in a single pressing plant) but there are also a few weird changes: