How sparse is Discogs?

In Discogs each release is assigned a number. At the moment (November 27 2017) the number of releases in Discogs is a bit over 9,200,000, while the latest release number is a bit over 11,200,000. That means that 2 million release numbers (around 17.85%) have gone from the database and about 82.15% of releases that were added are still in the database.

Around 2 million release numbers have been removed from the Discogs database.

[EDIT 2017-11-28: I was told by Discogs that every Draft is also assigned a release number and many drafts don't ever get submitted to the database, so they simply do not appear in the database]

In a previous post I wondered if Discogs is getting increasingly "sparse". I kept thinking about it, so I decided to just look at the data to see if my suspicion ("Discogs is getting increasingly sparse") was right.

Why releases disappear from Discogs

There are several reasons why releases disappear from Discogs. The most common reason is that a release is added as a duplicate of an already existing release and it is merged with an already existing release. Another reason is that some releases are deleted because they are bogus entries, like spam or vandalism. Unfortunately this information is currently not available for mining.

Distribution of releases in Discogs

I looked at the distribution of the release numbers, counted how many releases survived in the database for each batch of 1 million release numbers and what percentage that is. The raw numbers:

1 - 999,999: 795,792 releases (79.58%)
1,000,000 - 1,999,999: 867,198 releases (86.72%)
2,000,000 - 2,999,999: 907,216 releases (90.72%)
3,000,000 - 3,999,999: 896,593 releases (89.66%)
4,000,000 - 4,999,999: 837,183 releases (83.72%)
5,000,000 - 5,999,999: 811,840 releases (81.18%)
6,000,000 - 6,999,999: 806,288 releases (80.63%)
7,000,000 - 7,999,999: 804,439 releases (80.44%)
8,000,000 - 8,999,999: 799,594 releases (79.96%)
9,000,000 - 9,999,999: 752,377 releases (75.24%)
10,000,000 - 10,999,999: 763,821 releases (76.38%)
11,000,000 - 11,081,999: 65.087 releases (79.38%)

and as a picture:

To me it looks like Discogs is indeed getting increasingly sparse. Unfortunately I need more data to be able to answer why this is.

Comments

SID codes (part 1)

One thing that I only learned about after using Discogs is the so called Source Identification Code, or SID. These codes were introduced in 1994 to combat piracy and to find out on which machines a CD was made. It was introduced by Philips and adopted by IFPI, and specifications are publicly available which clearly describe the two available SID codes (mastering SID code and mould SID code). Since quite a few months Discogs has two fields available in the " Barcode and Other Identifiers " (BaOI) section: Mould SID code Mastering SID code A few questions immediately popped up in my mind: how many releases don't have a SID field defined when there should be (for example, the free text field indicates it is a SID field)? how many releases have a SID field with values that should not be in the SID field? how many release have a SID field, but a wrong year (as SID codes were only introduced in 1994) how many vinyl releases have a SID code defined (which is impossi...

SPARS codes (part 1)

Let's talk about SPARS codes used on CDs (or CD-like formats). You have most likely seen it used, but maybe don't know its name. The SPARS code is a three letter code indicating if recording, mixing and mastering were analogue or digital. For example they could look like the ones below. There is not a fixed format, so there are other variants as well. Personally I am not paying too much attention to these codes (I simply do not care), but in the classical music world if something was labeled as DDD (so everything digital) companies could ask premium prices. That makes it interesting information to mine and unlock, which is something that Discogs does not allow people to do when searching (yet!) even though it could be a helpful filter. I wanted to see if it can be used as an identifier to tell releases apart (are there similar releases where the only difference is the SPARS code?). SPARS code in Discogs Since a few months SPARS is a separate field in the Discogs ...

Using matrix numbers of CDs to verify release years (1)

One of the first steps when making CDs is to produce a "glass master". From that (using a few more steps) a "stamper" is created which is then used to press the actual CDs. When looking at a CD you can often see text in a ring in the middle of the CD. This is the so called matrix which comes from the glass master. Apart from the matrix other text (like the IFPI mastering SID code) could possibly also come from the glass master. A bit of background information can be found on page 7 of the IFPI SID code implementation guide although I would also highly recommend watching some of the clips about glass masters on YouTube, which are highly informative. One company making a lot of these glass masters is Cinram . For most of their glass masters they stored the production date in matrix. This is good news, because it means that it could possibly be used to verify releases and see if the declared year of the release is right: it could never have been released before...

Vinyl & Data

Search This Blog