SID codes (part 1)

One thing that I only learned about after using Discogs is the so called Source Identification Code, or SID. These codes were introduced in 1994 to combat piracy and to find out on which machines a CD was made. It was introduced by Philips and adopted by IFPI, and specifications are publicly available which clearly describe the two available SID codes (mastering SID code and mould SID code).

Since quite a few months Discogs has two fields available in the "Barcode and Other Identifiers" (BaOI) section:

• Mould SID code
• Mastering SID code

A few questions immediately popped up in my mind:

1. how many releases don't have a SID field defined when there should be (for example, the free text field indicates it is a SID field)?
2. how many releases have a SID field with values that should not be in the SID field?
3. how many release have a SID field, but a wrong year (as SID codes were only introduced in 1994)
4. how many vinyl releases have a SID code defined (which is impossible, as SIDs are specific to CDs, DVDs, SACD, BluRay, etc.)

In this blogpost I am going to answer the first question. I will try to answer the other questions in later blogposts.

Releases in Discogs with a SID field

I grabbed the latest data dump from Discogs (with all releases until November 1 2017) and looked at how many releases have a SID code defined, but don't have it in the SID field.

Like with other fields there are lots of misspellings of the names of the SID codes (check the scripts to see what I found). It confirmed once again that having structured data helps (not saying "fixes", just "helps").

First of all I looked at how many releases actually have a SID code defined. There are 111479 Mastering SID Code fields in 65877 releases. Looking at how these release are distributed over time, I can see the following (first bar has release numbers < 1 million, second bar releases between 1 million and 2 million, third bar between 2 million and 3 million, etc.):

For Mould SID Code fields there are similar numbers: 180360 fields in 78986 releases and the distribution looks almost the same:

It is not surprising that there are not many releases in the last bar, as there were a bit over 80,000 releases with a release number bigger than 11,000,000 in the latest Discogs dump (with data until November 1 2017).

What is surprising that most of the SID codes are in the oldest releases. I have no idea what is going on here (and that sounds like a good research question for a future blogpost), but a few possible explanations are:

• older releases are better maintained
• there are relatively more older releases than newer releases because of merges (this sounds a bit like a contradiction, but when people add duplicate releases those releases are merged, so it might be that the problem space for newer releases is a bit more "sparse"

Older releases appear to be better maintained.

Releases missing a SID field

Next I looked at how many fields are there that have indicated that there is a SID code of some sorts but the field wasn't a Mastering SID Code field or Mould SID Code field. It must be said that this is a bit of a conservative guess, and likely a few thousand entries are missing.

My scripts could find 12795 mould codes in 12315 releases. These were distributed as follows:

I could find 13003 mastering SID codes in 12807 releases, distributed as follows:

What immediately stands out is the very sharp drop which happened somewhere around release 4,500,000 (or perhaps a bit earlier). I didn't check, but this could very well coincide with the introduction of the SID code fields. If this is indeed the case, then it seems to suggest that contributors actually do use new fields when those are introduced.

Statistics clearly show that contributors use new fields that are introduced.

What puzzles me is why the number of releases with SID code fields didn't go up significantly. I'll leave that for another time.