Vinyl & Data

I am going to talk about the monthly statistics again, but I am going to skip a few months: that's right, no drill down statistics for January, February and March 2020. The reason is that Discogs had changed the dump format and I first needed to fix my scripts and give them a good brush up. The dump file that I researched contains the data as it was on May 1 2020 (I think as it was at 00:00 UTC, but I am not entirely sure). I compared this to the database from April 1 2020. there were 12,458,766 releases up from 12,302,145 releases, so that's 156,621 more 159,735 releases were added 3,114 releases were removed 11,676,243 releases stayed the same 622,788 releases were modified The modifications in the database were distributed across the data as follows: Releases in Discogs changed in April 2020 As I found out recently there are certain edits that are not very relevant , such as the YouTube links. These are irrelevant for me. The 268,187 releases with only irrelevant edits can...

Recently Discogs changed its data dump format, making it more difficult to compare releases the way I used to do and which is described in a previous blog post . So, the only thing I could do: dive into the XML a bit more and compare elements in the XML output for releases that have been changed. The results are quite interesting. I took the data dumps of March 1 2020 and April 1 2020, split these data dumps into individual XML files, computed SHA256 checksums for each of these files and ignored the ones that were the same in both dataset, leaving me with 531,444 releases to look at: Releases that were changed in March 2020 This graph is the same graph that I have been seeing for the last 2.5 years, but what I wanted to know is: what are the releases with relevant changes and where are those changes? Personally I don't think that the YouTube videos are very relevant, and since that was the biggest change in the Discogs data dumps I decided to filter these and then compare. What I ...

Because Discogs changed the data format of the data dumps and I have not yet fixed the scripts that I use there are no monthly statistics yet. But I can still say something about how many previously correct releases actually had errors introduced. As I said (or at least implied) in the previous article this doesn't necessarily mean that the data of the release is now more wrong: it could be that data was added that wasn't previously there and in the added data there is an error (this is what I mostly see). In January 2020 the 1564 unique releases that had errors introduced were distributed over the data as follows: Releases in the Discogs database where an error was introduced in January 2020 and which were previously correct Now, it should be said that this data actually seems to cover a large part of February 2020 as well. The name of the data dump file seems to suggest that it includes data up to February 20. The 321 unique releases that were changed between February 20 202...

One thing that I am always interested about: how many releases that my scripts thought were OK now have errors? I compared the data of smells of the data dump of December 2019 with the data dump of January 2020 and got the following chart (as always, the columns indicate the range of release numbers in the database: first column is everything with release number < 1,000,000, second column is everything between 1,000,000 and 2,000,000, and so on): Releases in the Discogs database where an error was introduced in January 2020 December 2019 and which were previously correct So actually: not too bad. It is also consistent with the change patterns I have seen over the years. The reasons for the peaks for older releases and newer releases: older releases are expanded with new information. This also means that more errors are introduced there. newer releases are usually added first, and expanded later by other people. This also increases the chances of errors being introduced.

Normally I would have expected this post to have been published earlier, but Discogs didn't release its data earlier. In the past the data dumps would be released around the 4th or 5th of the month, but now it took until February 24th before there was a data dump. I then looked at the data and just threw it into a corner and left it there for a few months. This is going to be a little bit different than the other blog posts in this series . The reason is that the internal format seems to have changed again. In the past the dump file would grow with around 100 MiB per month and it has done this very consistently over years. So when I saw that in one month the archive had grown with 1 GiB (about ten times the expected increase) I knew something was up. A short inspection: a lot of the YouTube video information is now included, so comparing the XML data as I have done so far actually makes no sense at all, as with the new data that is included a release would have been marked as ...

Today another data dump was released covering all releases up until (and including) December 31 2019, so I can look at some statistics again. If you don't know how this works, I would recommend reading the previous articles . Release statistics I looked at the dump file with new data entered and changes made from December 1 - December 31 2019. This dump file has 11,955,493 releases, whereas the previous one had 11,854,877. That means 100,616 releases more. Also: 11,187,982 releases stayed the same 663,648 releases were changed 103,863 releases were added 3,247 releases were removed from the database 208 releases had status Draft, Deleted or Rejected 0 releases that were not Accepted were in both dumps 3 releases were moved from Draft to Accepted Changes in the data (that is: changes to already existing releases) are distributed as follows: Existing releases changed in December 2019  What is striking is that there are significantly more edits. I looked thr...

A few days ago another data dump was released covering all releases up until (and including) November 30 2019, so I can look at some statistics again. If you don't know how this works, I would recommend reading the previous articles about it, for example the article about October 2019 . Release statistics I looked at the dump file with new data entered and changes made from November 1 - November 30 2019. This dump file has 11,854,877 releases, whereas the previous one had 11,751,696. That means 103,181 releases more. Also: 11,249,513 releases stayed the same 500,202 releases were changed 105,162 releases were added 1,981 releases were removed from the database 588 releases had status Draft, Deleted or Rejected 0 releases that were not Accepted were in both dumps 3 releases were moved from Draft to Accepted Changes in the data (that is: changes to already existing releases) are distributed as follows: Existing releases changed in November 2019 Smells I fou...

Phew, I finally managed to process all the data that I wanted to and I should be up to date, at least for the time being. So, let's look at what happened in Discogs in October 2019! For those of you who already know what to expect: it is the same as in previous months. For those of you who don't know, please read the post about September 2019 and work your way back . Release statistics I looked at the dump file with data covering October 3 - October 31 2019 (the previous datadump was for a little bit more than September). This dump file has 11,751,696 releases, whereas the previous one had 11,653,094 releases. That means 98,602 releases more which is less than normal, but also keep in mind that this dump file covers fewer days. Also: 11,211,165 releases stayed the same 439,651 releases were changed 100,880 releases were added 2,278 releases were removed from the database 223 releases had status Draft, Deleted or Rejected 0 releases that were not Accepted were in ...

Normally I don't think too much about statistics of a particular month, but September is different, because of Discogs' "September Pledge Initiative" or "S.P.IN". You can read Discogs' own take on it on the Discogs blog , but let's see what the data says. If you don't know what everything here means, just look at the previous blog post and work your way back. Release statistics I looked at the dump file with data covering September 1 - October 3 2019 (according to the name of the datadump). This dump file has 11,653,094 releases, whereas the previous one had 11,530,540 releases. That means 122,554 releases more which is actually not more than in a normal month (at least, that's what I think). Also: 11,021,531 releases stayed the same 506,609 releases were changed 124,954 releases were added 2,400 releases were removed from the database 477 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in...

Time to dig into some statistics about how Discogs was changed in August 2019. Did the excessive heat reduce the number of contributions, or was it business as usual? If you haven't read it yet I would recommend reading the blog post about contributions in July 2019 first. Release statistics I looked at the dump file with data covering August 1 - 31 2019. This dump file has 11,530,540 releases, whereas the previous one had 11,430,638 releases. That means 99,920 releases more which is only a little bit less than normal. Also: 10,897,333 releases stayed the same 529,657 releases were changed 103,550 releases were added 3,648 releases were removed from the database 403 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 3 releases were moved from Draft to Accepted What is interesting: quite a few more releases were removed from the database, so my guess is that there was some cleanup of sorts. Changes in the data (t...

It is already November, so it is time to write an update about July. I couldn't do it earlier because I got completely swamped in work, but I will be making a few extra blog posts to compensate for the lack of blog posts. Anyway, back to the monthly statistics! If you don't know how this works, you should read the previous entries, for example about what happened in June 2019 and then work your way through history. You will notice certain patterns. Release statistics I looked at the dump file with data covering July 1 - 31 2019. This dump file has 11,430,638 releases, whereas the previous one had 11,334,008 releases. That means 96,630 releases more which is a bit less than normal, but that can probably be explained by summer on the northern hemisphere. Also: 10,853,019 releases stayed the same 478,710 releases were changed 98,909 releases were added 2,279 releases were removed from the database 344 releases had status Draft, Deleted or Rejected 0 11 releases t...

A new datadump was released last night, so time to look at what happened in Discogs in June 2019. If you don't know how this works, you should read the previous entries, for example about last month . Having done this for nearly two years I am pretty sure that it wil be nearly identical to the previous few months. Release statistics I looked at the dump file with data covering June 1 - 30 2019. This dump file has  11,334,008 releases, whereas the previous one had 11,231,882 releases. That means 102,126 releases more. Also: 10,723,325 releases stayed the same 506,571 releases were changed 104,112 releases were added 1,986 releases were removed from the database 167 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 1 release was moved from Draft to Accepted Changes in the data (that is: changes to already existing releases) are distributed as follows: Existing releases changed in June 2019 Although it still h...

A new data dump has been released by Discogs, so time to look at some statistics again, although I am certain that it is very similar to previous months . In fact, I can already tell that it is almost exactly the same: the contributions and to and changes in Discogs are very very consistent. Release statistics I looked at the dump file with data covering May 1 - 31 2019. This dump file has  11,231,882 releases, whereas the previous one had 11,123,192 releases. That means 108,690 releases more. Also: 10,619,089 releases stayed the same 500,776 releases were changed 3,327 112,017 releases were added 112,017 3,327 releases were removed from the database 180 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 14 releases were moved from Draft to Accepted Changes in the data (that is: changes to already existing releases) are distributed as follows:  which is, again, almost the same graph as in previous months....

A new data dump has been released by Discogs, so time to look at some statistics again, although I am sure that it is very similar to previous months . Release statistics I looked at the dump file with data covering April 1 - 30 2019, although judging by the name of the previous dump file it might actually be April 2. This dump file has 11,123,192 releases, whereas the previous one had 11,019,051 releases. That means 104,141 releases more. Also: 10,536,611 releases stayed the same 479,953 releases were changed 106,628 releases were added 2,487 releases were removed from the database 380 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 0 releases were moved from Draft to Accepted What is surprising is the high amount of draft/deleted/rejected releases. Of these 353 are Draft releases. I looked at a handful of releases, but I couldn't find a clear pattern. Changes in the data are distributed as follows: Again, ...

If you are reading this, then you are probably aware of the blog posts where I dig into what happened in Discogs in a particular month. If not, please read a few of the blog posts first. As I saw a few patterns emerge I wanted to know if I would be seeing the same patterns if I would look at the data for an entire year, or if it would be different. So I took the data dumps covering March 2018 and March 2019 and I compared them. Originally I wanted to use data from 2018 only, but since the XML dump format changed a few times I could not without a lot of extra work. Release statistics At the end of March 2018 there were 9,680,263 releases, at the end of March 2019 there were 11,019,051 releases, meaning an increase of 1,338,788 releases. 22,089 releases were removed 1,360,877 releases were added 6,250,678 releases stayed the same 3,407,496 releases were changed So that means that quite a few releases have not been changed. This immediately makes me wonder: are these r...

Discogs has been growing very steadily and the patterns of where and how changes are made are also very consistent. Would this month be any different than previous months (spoiler: not really)? Release statistics I looked at the dump file with data covering March 1 - 31 2019. This dump file has 11,019,051 releases, whereas the previous one had 10,893,841 releases. That means 125,210 releases more, which is quite a bit more than in previous months. Furthermore: 10,291,906 releases stayed the same 599,579 releases were changed 127,566 releases were added 2,356 releases were removed from the database 156 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 3 releases were moved from Draft to Accepted Changes in the data are distributed as follows: Again it is nearly the exact same graphs as for the last year: most changes are in older releases and the newest releases. Smells I found 2622 smells in newly added releas...

I am almost done with my backlog of blog posts about how Discogs changes monthly, with just a few more to go. It is advised to first read last month's blog as I will be making a few references to it. I also advise you to read Discog's own blog post about what happened in February although I am puzzled why we get slightly different numbers. Release statistics I looked at the dump file with data covering February 1 - 28 2019. This dump file has 10,893,841 releases, whereas the previous one had 10,785,723 releases. That means 108,118 releases more. Furthermore: 10,257,122 releases stayed the same 526,740 releases were changed 109,979 releases were added 1,861 releases were removed from the database 261 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 1 releases was moved from Draft to Accepted Last month I said that there was a very significant jump in the number of releases added and changed. On first sight i...

Continuing the spring cleanup of my backlog of blog posts, so: what happened in Discogs in January 2019? If you don't know how this works, please read the previous blog post as well as the previous posts. Release statistics I looked at the dump file with data covering January 1 - 31 2019. This dump file has 10,785,723 releases, whereas the previous one had 10,668,146 releases. That means 117,577 releases more. Furthermore: 10,045,154 releases stayed the same 620,007 releases were changed 120,562 releases were added 2,985 releases were removed from the database 155 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 4 releases were moved from Draft to Accepted Quite interestingly, it seems that there was a very significant jump in the number of releases added and changed. Is this perhaps because people added their Christmas presents early in January? The graph of were the changes are is very similar, again, to prev...

Time for a spring cleanup of my backlog of tasks, so: what happened in Discogs in December 2018? If you don't know how this works, please read the previous blog post . Release statistics I looked at the dump file with data covering December 1 - 31 2018. This dump file has 10,668,146 releases, whereas the previous one had 10,560,935 releases. That means 107,211 releases more. Furthermore: 10,070,747 releases stayed the same 488,022 releases were changed 109,377 releases were added 2,166 releases were removed from the database 212 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 1 releases was moved from Draft to Accepted The graph of were the changes are is very similar, again, to previous months: Smells I found 2137 smells in newly added releases (ignoring tracklist errors) and they all look very similar as before: a bit more than half label codes, a quarter SID codes, and then the rest.

Making up for lost time: data for November 2018! As always it will be in the same format as for previous months . Release statistics I looked at the dump file with data covering November 1 - 30 2018. This dump file has 10,560,935 releases, whereas the previous one had 10,449,676 releases. That means 111,259 releases more. Of those: 9,976,683 releases stayed the same 470,852 releases were changed 113,400 releases were added 2,141 releases were removed from the database 233 releases had status Draft, Deleted or Rejected 11 releases that were not Accepted were in both dumps 2 releases were moved from Draft to Accepted Changes look like this. I find it amazing that every month the graph is almost exactly the same except for the last column:   Smells I found 2199 smells in newly added releases (ignoring tracklist errors) and they all look very similar as before: about half label codes, a quarter SID codes, and then the rest.