Vinyl & Data

In the last few months I had very little time to look at anything related to Discogs because of work. The fact that Discogs was also very late releasing their monthly data dump in February (and changed the format, which kind of broke my scripts) didn't exactly help either. In the last few days I revisited Discogs and after a few minutes the frustration I have felt in the past immediately returned. Basically it all boils down to the following: people are (still!) allowed to put all kinds of garbage into the database and very often this can be completely prevented, as I have argued before and let Discogs staff know many times over the last (nearly) 3 years. I started doing some minor cleanups and it just feels a bit useless: I have been doing the same clean ups for the last few years, so why do I even bother fixing it if the guardians of the database don't seem to care themselves? A lot of time is now wasted on trivial cleanups which also could have gone to adding data to the...

Having looked through many releases in Discogs I often see that data is not entered as it is on the release. One example: rights societies are typeset in many different ways, where sometimes you will have an all uppper case name (for example: "BIEM"), sometimes there is spacing ("B I E M"), sometimes there are dots in the name ("B.I.E.M."), and so on. In the examples mentioned above the values are all basically referencing the same rights society so it is worth asking if the values entered on each release page should be "as written on release" or if they should be stored as the value of whatever it is pointing it? Said differently: should the release page focus on "syntax" (how it is written on the release) or "semantics" (what it means)? There are things to be said for both: there are releases where the person designing the sleeve or label made an error, for example: releases with "BIEN" instead of "BIEM...

In some countries (especially in Spain) it was/is custom to translate titles of songs and print them on the sleeve (sometimes without the original title). The cool thing about this is that these releases have unique sleeves and labels. One example is Queen's "Hot Space", which was translated to " Espacio Caliente ". Frequently songs are also covered in a different language than the original and given a new title. From a collector point of view this could be interesting: there are actually people who collect covers of specific songs. The challenge is that if only the translated title is printed on the release (or entered in Discogs) it is harder to find out which track it actually is. For regular releases (where the title simply has been translated) it is usually not that difficult. But, for cover versions it can be. How to indicate translations has been the matter of debate inside Discogs. At some point the solution was to enter both the original title and...

Sometimes duplicate entries get added to the Discogs database, for various reasons: inexperienced users: it takes some time to understand the Discogs workflow. This happened to me as well when I was starting out and it is because the Discogs edit interface throws you into expert mode, instead of trying to guide you through the process (which I have written about before here and here ). stupid sellers: some sellers still don't understand that Discogs is a catalog where you simply pick a release (unlike for example eBay), so they add releases that are already in the catalog. entry errors: sometimes errors are made, which makes it hard to find that an entry has already been added, for example when it isn't clear what the correct label is, or the artist, and so on and then someone adds the same release twice. disagreement about when a release is a variation: people on Discogs frequently disagree about when a release is actually a different release, or simply a variation. For ...

I started this blog in September 2017 with the intention of finding interesting patterns in the Discogs data. It fairly quickly turned into a blog pointing out what doesn't work in Discogs, which I must admit is a bit negative and not what I originally wanted. I am honestly trying to keep it positive this simply isn't always possible. The reason: in the last two years very little has been done to get rid of data entry errors that can be prevented. The only thing that I have seen is the automatic correction of capitalization errors (even though it is a bit difficult to trigger, although it does a good job). Due to work obligations I couldn't look at the data for a few months and had hoped that things would have been different when I returned to processing the data, but this wasn't the case. I have had conversations with people at Discogs and those conversations were actually very nice, but it doesn't seem to lead to improvements. Why this is I really don't know...

I just need to rant again about something in Discogs that I really do not like and that is the voting system. In Discogs you can vote on the quality of the data of a release. For example, if someone completely screwed up an existing release, you can vote that it is "Entirely Incorrect" and that will then revert the commit and restore it to the previous state. Or, you could vote that it is "Complete and Correct", signalling that all the data that can be on there is on there and it is correct as well. There are also: Correct - not all information might be there, but at least it is correct Needs Minor Changes - information is there, just needs a quick brush up Needs Major Changes - the release needs a lot of work This is to have some sort of "self cleansing" mechanism, to alert people to mistakes, and to weed out bad contributors, as getting many bad votes will put you into "Discogs school" (the "Contrinbutor Improvement Program", o...

Whenever a change is made in Discogs a mail is sent. In my opinion the system behind these e-mails is broken, as I have already said before . One of the biggest issues that I have with the current e-mails being sent by Discogs is the subject line of the e-mail, saying: Recent changes affecting releases in your collection or Recent changes affecting your contributions but nothing else. I first have to click on the message to see to full content. It might be that I find some of the changes more interesting than the others. For example, if images are changed, then it is likelier that I will want to take a look because sometimes people change images, with ugly consequences . So I would like to have a little bit more information in the e-mail subject, for example: [IMAGES] Recent changes affecting releases in your collection would already be an improvement over the existing mails being sent as it would allow me to prioritize which mails to check first (at the expense of some e...

I am going through a few hundred 7" to see if they are already on Discogs (which in itself is already a big task) and I am seeing quite a few that are not on there (so setting them apart to make scans, which is even more work). What I see way too often is so called "release hijacking" where an existing release is changed into a different release. This seriously pisses me off and I have written about it before . Some examples: someone posts pictures saying "alternate labels" (which according to the guidelines should be turned into a different release) or "better pictures" which turn out to be for a different release altogether (different texts, labels, etc.). Or, they don't read the notes that are for a release that say something like "This is the release with X, for the release with Y, see Z" and then add pictures for the release with Y. Sigh. What these people do not seem to realise is this vastly reduces the value of the database f...

Some people, including me, use Discogs for having an overview which releases they have. Though it is not perfect because people can, and do, change releases so it matches their own copy (even when it should have been a separate release), but it is better than nothing. There are a few things that annoy me in the way that Discogs organizes collections. One of them is the way that releases can be organized into folders. I understand the rationale behind it: you can organize releases into folders and then look at these folders separately instead of at a big list which would work for a few items, but not when having thousands or tens of thousands of items. Except: for me it is too restrictive. Sometimes I would like to have releases in more than one folder, for example I might want have a release in a folder "death metal", a folder "dutch", a folder "coloured vinyl" and a folder "football" at the same time (although I am not aware of any football ...

I have been thinking a bit about how to increase data quality when entering data. There is already a lot that could be done by using wizards and asking users the right questions . But I think that it can be made even easier when using graphical hints and explicitly pointing out to users what information should be entered in the database. In the previous article I mentioned using a wizard and guiding users through the process of entering data. When the right questions have been answered (such as country and label) the user could be asked what the label of the release looks like . For example they could be given the following choice (left: typical label EMI used in Spain in the 1980s, right: typical label EMI used in Spain in the 1970s): Examples of labels EMI used in the 1980s (left) and 1970s (right) Based on this they could then be guided through the process of picking the right data. Also, using which label was picked already means that some checks can be applied. For exa...

I just helped someone add a few releases to Discogs and it was, again, a quite frustrating experience, even though I have added releases to Discogs before. Adding releases to Discogs is actually a lot of work. These are the steps I typically take for a 7" single: see if the release is already there on Discogs. see if the release I have differs from any of the listed releases copy an existing release to draft, or start from scratch fill in all the details that I am sure about make scans, crop the scans, scale them add scans Adding a release properly, with all the details, can easily take 15 to 20 minutes, and then I still only have a fairly barebones release. What I typically don't get right due to lack of knowledge are things like: composers printing companies manufacturing companies pressing plants country specific peculiarities etc. These usually require specialist knowledge about releases from a certain country, label or artist, which I simply don't...

I think it is time for another usability rant. Although I myself am far from a usability expert (the best courses I took at university were about usability that taught me to stay far away from it) I do suffer from user interfaces with quirks and Discogs is no exception. I have written about this subject before ( images , data entry , etc.) but it bites me and my friend gerjolp every time we are on Discogs. While editing releases for correctness (such as the depósito legal fixes described an an earlier post ) where people made snarky remarks about whether or not we didn't have anything better to do. This was a bit strange, as the data for the releases is more complete now and this should be encouraged. My guess is that the real reason is not that the releases were edited but the fact that Discogs sends a notification e-mail for every change that is made and some people got hundreds of e-mails with all an identical message about a similar change in many releases. Instead of flagg...

Although I see the amount of obvious errors decrease (although veeeery slowly) there are always releases where people introduce new errors. I already looked at this several times before, but it never hurts to keep pointing it out. In March 2019 for a total of 1241 releases an error was introduced either because information that contained an error was added, or because fields were changed to something wrong. This excludes possible tracklist errors. The errors are distributed like this across the data. Errors introduced in March 2019 In my opinion this looks quite a bit like the monthly changes and my guess is that if you would normalize it (look at the errors that were changed, compared to the number of releases that were changed) it might look different. I should look into that.

I was gone from Discogs for a while due to lack of time but also because I was a bit frustrated about the lack of quality of data in Discogs and the (seeming) unwillingness of Discogs to fix this (although it is also very likely that, like me, they are completely swamped and there are only so many hours in the day). One thing that keeps bugging me in Discogs is that people add or change information in such a way that they actually contain wrong information. I already looked at this problem once before and I was wondering if it is as bad as it was back then, or if things have gotten better in Discogs. A few unscientific tests didn't seem very promising so I grabbed the dumps of February and March 2019 and simply counted how many old releases were in the later dump, but not in the earlier. Ideally the errors found in the older releases are known errors, so for old releases this number should be 0 in the best case. But, that is not what is happening. As it turned out, quite a fe...

A few times I have already talked about my love/hate relationship with Discogs. I love it because there is so much data, but I dislike the quality of the data. I like it that on the Discogs blog there are now finally frequent blog posts about how the database evolves, with pretty graphs (as I told some staff members they should consider), but I don't like that they are not looking at errors and increasing quality. Also, I don't like that they are not publishing the actual data sets used to generate those pictures, which brings me to the core of this post: What I love is that (most of) the data is available under a CC0 license (and others share this view ) but as I have ranted about before: this is not actually all of the data from the catalog (I am not interested in the sales data). Specifically, all of the historical edit information is missing , which potentially contains very valuable hints about how releases have evolved over time, which would allow people to dig int...

There are some things in Discogs that really irritate me. One of the things that I personally dislike a lot is the voting system, as it is not granular enough. But what's worse is that there are ways to (partially) edit release and where the voting system can be bypassed: pictures. Normally when editing a release in Discogs that has received a vote (either correct, incorrect, etc.) its status is set back to "needs vote", except when only changing images. In that case the voting status stays as it is. I have seen instances where an entry was voted "complete and correct" 7 years ago, but that pictures that were completely different replaced the original ones and no one noticed. This so called "hijacking" of releases is very much frowned upon and unfortunately it happens far too often. So what Discogs should do is quite simple: make sure that every edit, including an edit related to images, causes the status to be set to "needs vote".

I have been digging into the Discogs dataset for over 9 months now and blogging about it since September. During this period I have made a few observations. In short I must say that I have mixed feelings about Discogs because it isn't clear what Discogs actually is and what they want to achieve. I talked about this earlier , but it basically comes down to this: Is it a catalog? Is it a marketplace? Is it a place to organize your collection? Discogs is trying to be all but not succeeding because there are tensions between the different use cases: enforcing correctness chases sellers away from the marketplace and selling records is what is bringing in the money for Discogs and which keeps the other fires burning. But the costs for this is that the data in the catalog is sometimes blatantly incorrect, which vastly reduces its value. Personally I don't care about the marketplace, as I am a collector and I care about the catalog. Discogs has enormous potential and for collecto...

I am still not done digging into the ISRC data from Discogs, as I see it as a source of errors and I am in ranting mode. At the moment having errors in ISRC codes is not a big problem, as Discogs is not using it yet (right now you cannot specifically search on ISRC codes). Unlocking this data could actually quite handy in the future ("Oh, I like this track that I downloaded and want it on a physical release. On which physical releases was it published?") but I will let them discover that business case themselves. If you don't know what ISRC codes are I suggest you start with reading one of my previous posts about the subject and follow the links there. What I wondered is: how many times can you find the same ISRC code on a single release, for example if a contributor makes a copy/paste error and forgets to change the code? So I adapted my scripts and ran a test to see in which releases (where ISRC codes actually marked as such with a proper ISRC field) there are dup...

Time to look at ISRC codes again because I am still not done researching them. If you don't know what it is, I would suggest that you first read part 1 , part 2 and part 3 . To better understand some of my complaints below you should also read about why I think that Discogs stores the ISRC fields in the wrong place . So now that we've got that out of the way we can start. The ISRC field in Discogs is in the "Barcodes and Other Identifiers" (BaOI) section, which I already explained is not the correct place and it should be with the individual tracks. Right now people are using descriptions (a free text field) to indicate which ISRC code belongs to which release. Because I know how bad people are with typing in correct information (and of course I am not immune to this) I was wondering in how many releases people make mistakes. I added a very simple check to my scripts to see if descriptions were being reused (copy/paste errors, or "off by one" errors). ...

Time to look into a few more aspects of the Discogs data model and where I think it could be improved. If you haven't read the first part I would suggest you do that first. This time I want to look at a few more: rights society, mastering and mould SID codes, and pressing plant. Rights Society Currently rights societies are stored in the "Barcodes and Other Identifiers" (BaOI) section which covers the whole release. The correct place for this information would actually be per track. There are some releases in Discogs (like some compilations) where it is indicated which rights society applies to which track. Mastering and Mould SID codes Like rights societies the mastering and mould SID codes are stored in the BaOI section that covers the whole release. The correct place for these identifiers would actually be a physical disc, as they frequently are different in releases with multiple discs per release. The solution that people use is to tag it in the free text fie...