Skip to main content

What happened in Discogs in September 2017?

The new monthly datadump is available, so I downloaded it and processed it with my scripts to see what happened in September 2017.

Release statistics

The new dumpfile was published on October 4 2017 and has 8,996,419 releases. The previous dump (published September 4 2017) had 8,878,391 releases. That means 118,028 more releases in the database.
  • 3,158 releases were removed in the new dumpfile
  • 121,186 releases were added in September
  • 8,456,324 releases remained the same
  • 418,909 releases were changed
  • 205 releases had the status Draft, Deleted or Rejected set
  • 11 releases that were not Accepted were present in both the September 2017 and October 2017 data dump
  • 1 release moved from Draft to Accepted
In total there were edits for 540,095 releases (changed, plus new). There were 43,200 minutes in September 2017, meaning that there was a minimum of 12.5 edits (new or changed releases) per minute, meaning a bit more than one edit every five seconds. There were also edits for the new releases, but these are not recorded in the dump (and all I can see is that there has been a new release), so this is a lower bound.
In September 2017 there was one edit in the Discogs catalogue every five seconds

Releases with known smells

Using another script I looked at some of the known smells in the data. The good news is that it seems to be going down. There were around 15,000 less known smells on October 1 compared to September 1, with more releases (but still there are around 218,000 releases where I know there is a problem).
On October 1 2017 there were 15,000 releases with known smells less than on September 1 2017
One field for which fewer errors was the Depósito Legal field with about 7,500 fewer bugs. Almost 4,000 SPARS Code fields were corrected.

Of course this is just the low hanging fruit: it is very easy to find out that something that is listed is syntactically incorrect (wrong field, wrong formatting, and so on), but not if it is factually correct, or that there is information is missing, so the numbers above should always be taken with a grain of salt as many more releases are in desperate need of improvement. Doing more thorough checks requires a lot more (human) effort, but still it seems that it is heading into the right direction, so I am quite positive about what I see.

Also, the scripts that I wrote are not perfect and do not catch everything. There are many more checks that can be done, and will be done in the future, so I am expecting the numbers to go up again.

As soon as the dump with the data of October has been released (in early November) I will rerun the (improved) scripts (for both old and new data) and see if there is continued improvement.

Comments

Post a Comment

Popular posts from this blog

SID codes (part 1)

One thing that I only learned about after using Discogs is the so called Source Identification Code, or SID. These codes were introduced in 1994 to combat piracy and to find out on which machines a CD was made. It was introduced by Philips and adopted by IFPI, and specifications are publicly available which clearly describe the two available SID codes (mastering SID code and mould SID code). Since quite a few months Discogs has two fields available in the " Barcode and Other Identifiers " (BaOI) section: Mould SID code Mastering SID code A few questions immediately popped up in my mind: how many releases don't have a SID field defined when there should be (for example, the free text field indicates it is a SID field)? how many releases have a SID field with values that should not be in the SID field? how many release have a SID field, but a wrong year (as SID codes were only introduced in 1994) how many vinyl releases have a SID code defined (which is impossi
1 comment
Read more

SPARS codes (part 1)

Let's talk about SPARS codes used on CDs (or CD-like formats). You have most likely seen it used, but maybe don't know its name. The SPARS code is a three letter code indicating if recording, mixing and mastering were analogue or digital. For example they could look like the ones below. There is not a fixed format, so there are other variants as well. Personally I am not paying too much attention to these codes (I simply do not care), but in the classical music world if something was labeled as DDD (so everything digital) companies could ask premium prices. That makes it interesting information to mine and unlock, which is something that Discogs does not allow people to do when searching (yet!) even though it could be a helpful filter. I wanted to see if it can be used as an identifier to tell releases apart (are there similar releases where the only difference is the SPARS code?). SPARS code in Discogs Since a few months SPARS is a separate field in the Discogs
Post a Comment
Read more

Country statistics (part 2)

One thing I wondered about: for how many releases is the country field changed? I looked at the two most recent data dumps (covering February and March 2019) and see where they differed. In total 5274 releases "moved". The top 20 moves are: unknown -> US: 454 Germany -> Europe: 319 UK & Europe -> Europe: 217 unknown -> UK: 178 UK -> Europe: 149 Netherlands -> Europe: 147 unknown -> Europe: 139 unknown -> Germany: 120 UK -> US: 118 Europe -> Germany: 84 US -> UK: 79 USA & Canada -> US: 76 US -> Canada: 65 unknown -> France: 64 UK -> UK & Europe: 62 UK & Europe -> UK: 51 France -> Europe: 51 Saudi Arabia -> United Arab Emirates: 49 US -> Europe: 46 unknown -> Japan: 45 When you think about it these all make sense (there was a big consolidation in Europe in the 1980s and releases for multiple countries were made in a single pressing plant) but there are also a few weird changes:
Post a Comment
Read more