Vinyl & Data

Time to return to an older subject: ISRC codes. I have written about ISRC codes before so if you don't know what I am talking about, you might want to read the older posts first and then return to this one. I wondered how in the past few months the number of obvious ISRC errors would have gone down. In my previous post about this subject I saw a massive decrease: May 2019 dump: 22,391 releases June 2019 dump: 21,096 releases July 2019 dump: 19,214 releases August 2019 dump: 15,161 releases September 2019 dump: 13,390 releases October 2019 dump: 12,090 releases November 2019 dump: 11,360 releases December 2019 dump: 10,578 releases January 2020 dump: 8,893 releases and I wanted to know if this trend continued: February 2020 dump: 6,840 releases (but this dump covered changes in January and most of February, not just January) March 2020 dump: 6,722 releases April 2020 dump: 6,468 releases May 2020 dump: 6,313 releases The trend is still downwards, but the pace seems to have slowed ...

Because a new data dump was released today I decided to look a bit further into the ISRC codes and how they are being fixed. Before you read further I would advise to read the previous blog post about this topic first. In the previous article I had the following data for the amount of releases with a (possible) ISRC code, but not using the ISRC field: May 2019 dump: 22,391 releases June 2019 dump: 21,096 releases July 2019 dump: 19,214 releases August 2019 dump: 15,161 releases September 2019 dump: 13,390 releases October 2019 dump: 12,090 releases November 2019 dump: 11,360 releases December 2019 dump: 10,578 releases In the January 2020 dump this is further down to 8,893 releases, so that's good news. I am hopeful that in the next few months that all the obvious mistakes will have been fixed (there are also a few edge cases I need to research further so I can filter them out). To catch that in bar charts: I plotted the release numbers that are in need of fixing f...

One field in the "barcode and other identifiers" section in Discogs that is fairly new is the ISRC field. I have written about ISRC before, so you might want to read those posts first. Before there was a dedicated ISRC field the ISRC codes would be put into fields like "Other" or "Barcode" so I wondered: how well are these being updated and replaced by ISRC fields? I checked the dumps from the last 6 months: May 2019 dump: 22,391 releases June 2019 dump: 21,096 releases July 2019 dump: 19,214 releases August 2019 dump: 15,161 releases September 2019 dump: 13,390 releases October 2019 dump: 12,090 releases November 2019 dump: 11,360 releases December 2019 dump: 10,578 releases So in 8 months time the number of problematic releases has more than halved, while the amount of releases in the database with ISRC codes has increased. With a bit of luck all of these will have been fixed in the next half year. Impressive.

Since the ISRC codes were introduced people have gradually been adding them, or fixing them, which is a good thing. It also means that I can mor easily detect errors. Back in April 2019 I already wrote about detecting duplicate ISRC codes in a release so you probably want to read that and some older articles first. So, what's the situation like half a year later? I took the Discogs dump covering data up until October 31 2019 and found 188 unique releases, and created the following chart, which shows where in the data set these releases can be found (but I cannot conclude anything useful from it): Releases with duplicate ISRC codes in Discogs in October 2019 So it is slowly increasing and there are about 60 more than in April. I still have to go through these releases to see if these are data entry errors by Discogs users, if the label made a mistake (which also happens), or (in case of CDs) perhaps a drive with a malfunctioning firmware was used, as apparently there are C...

A bit over a year ago I looked into ISRC codes for checking release dates. The ISRC code actually contains a year component which can be used to check dates, as (except in rare old cases) the release is always after the ISRC code has been assigned, so it cannot be earlier. A few people on Discogs seem to disagree, but meh. For more details you can read an earlier post . So back in February 2018 I got the following graph: Distribution of smells in releases with known ISRC fields in February 2018 which also included some other smells and not just date mismatches. Back then I concluded that there were 273 releases that had a date mismatch. So I decided to rerun my analysis with the latest data dump (up until and including April 30 2019) now that it seems that there are many more ISRC codes in the database, because people seem to be fixing the ISRC field . As expected the amount of releases with a date mismatch is now a lot higher: 1414. They are distributed across the database ...

ISRC codes are supposedly unique: given an ISRC it should be possible to uniquely identify a particular recording. But how many releases have duplicate ISRC codes? I already looked into this in part 5 of the series on ISRC, which was from a year ago. Back then I didn't find that many but expected numbers to go up as soon as more releases start using ISRC fields properly. Back then I found 48 releases and in April 2019 it looks like this: Releases in Discogs with duplicate ISRC codes, April 2019 As you can see it is indeed more than 48 releases: 129 releases to be exact. Some of these will be data entry errors, but probably there are also a few new ones where errors have been made on the release itself.

Sometimes it is good to revisit an older topic to see if findings have changed. This time I am digging into ISRC codes again, and will be comparing with an older blog post from December 2017 . ISRC tagged values in BaOI, but not using ISRC fields Back in December 2017 I posted the following graph, depicting the distribution of fields that indicated having some ISRC related value, but not using the ISRC field in BaOI: Fields with possible ISRC values, December 2017 I reran the analysis using the data released in April 2019 and got the following graph: Fields with possible ISRC values, April 2019 There are a few things that are noteworthy: first of all, the amount of errors in new releases is very low. This is good news. Second, the shape of the rest of the data is almost identical as the first graph, but overall there are far far fewer releases that are in need of fixing (40% for the highest peak). So this seems to indicate that people are indeed fixing these fields...

I am still not done digging into the ISRC data from Discogs, as I see it as a source of errors and I am in ranting mode. At the moment having errors in ISRC codes is not a big problem, as Discogs is not using it yet (right now you cannot specifically search on ISRC codes). Unlocking this data could actually quite handy in the future ("Oh, I like this track that I downloaded and want it on a physical release. On which physical releases was it published?") but I will let them discover that business case themselves. If you don't know what ISRC codes are I suggest you start with reading one of my previous posts about the subject and follow the links there. What I wondered is: how many times can you find the same ISRC code on a single release, for example if a contributor makes a copy/paste error and forgets to change the code? So I adapted my scripts and ran a test to see in which releases (where ISRC codes actually marked as such with a proper ISRC field) there are dup...

Time to look at ISRC codes again because I am still not done researching them. If you don't know what it is, I would suggest that you first read part 1 , part 2 and part 3 . To better understand some of my complaints below you should also read about why I think that Discogs stores the ISRC fields in the wrong place . So now that we've got that out of the way we can start. The ISRC field in Discogs is in the "Barcodes and Other Identifiers" (BaOI) section, which I already explained is not the correct place and it should be with the individual tracks. Right now people are using descriptions (a free text field) to indicate which ISRC code belongs to which release. Because I know how bad people are with typing in correct information (and of course I am not immune to this) I was wondering in how many releases people make mistakes. I added a very simple check to my scripts to see if descriptions were being reused (copy/paste errors, or "off by one" errors). ...

Time to revisit the ISRC codes. I already talked about these twice, namely how many errors for these codes there are in Discogs , as well as how to extract them from a CD . In the first of those posts I already hinted at that there is actually a year component in the ISRC codes. The ISRC code for a single track has 12 characters. Characters 6 and 7 should be digits indicating the year the code was assigned (although in the early days some ISRC codes were handed out where the year the song was written or when the recording was made). It would of course been easier if they would have used 4 characters, but they didn't. That means that the year component from the ISRC code can be used to check whether or not a release is correctly dated, because a release cannot be from earlier than the year the ISRC was assigned. I adapted my scripts to check for a few things: is the format confirming to the ISRC standard? is the year recorded in the ISRC not later than the date of the rele...

One of the fields in the "Barcode and Other Identifiers" section of a release in Discogs is the Rights Society field. These societies are for things like collecting royalties for artists, although I have no idea how they exactly work (and, to be honest, I am not that interested in knowing at the moment). Examples of rights societies are ASCAP, GEMA, BUMA , SGAE and more. The Rights Society field has been around in Discogs for some time, but still quite a few errors are being made. For example, I often see the Other field being used, but also ISRC , Barcode and other fields. A while ago I added support for detecting these to my scripts , and recently refined detection so I could write this blog with some more accurate information. Rights Society errors In the latest Discogs data dump you will find about 30,000 releases where the Rights Society field is not used where it possibly should have been used. This excludes instances of where the value is actually not a rig...

It is time to dive into ISRC codes a bit more. If you don't know what these codes are, then it is probably good to read part 1 first. In this article I will not really look at ISRC codes in Discogs (OK, just a tiny bit), but instead look at how to extract ISRC codes on a Linux machine using Python. For this I will use libdiscid and its Python bindings . What you need for this: a computer with a CD/DVD drive (which is getting surprisingly hard with laptops these days) a Linux distribution (in my case Fedora 27, but any recent Linux distribution will do) libdiscid with associated Python bindings (either Python 2 or Python 3) It is actually very simple and can be done from either the Python prompt or from a simple script. It is literally as simple as this (Python 3 code, I stored it in a file called isrc.py): import libdiscid disc = libdiscid.read() try:     track_isrcs = disc.track_isrcs     for i in range(0,len(track_isrcs)):   ...

One thing that I had never heard about is the International Standard Recording Code (ISRC). According to the Wikipedia page about ISRC (worth reading) it is actually already almost 30 years old. These codes are interesting for datamining for a few reasons: the year in which the code was assigned is embedded in it, even though there are some exceptions: apparently in early days the recording year was used. they are specific to a single recording These two characteristics make it an ideal piece of data for checks and comparisons. Sometimes the ISRC code is printed on releases, but sometimes also embedded in some of the metadata on CDs. I never knew about ISRC, as I don't have releases with the code (or at least, I never paid attention) and I never have had a (standalone) CD-player that displayed these codes by default. ISRC in Discogs The ISRC field is a relative new addition to the Barcode and Other Identifers (BaOI) section. In retrospect it might have been better to ...