Big data is being integrated into nearly every field. It should be no surprise that the multi-billion dollar music industry wants in. There are two major ways big data is already influencing the music industry: music creation and music selection. The second one, however, is making far more waves.

It’s no secret that the music industry often chooses the next hit star, and pushes them to receive more air time. Numerous studies show that people like music that sounds familiar. This means that there is a certain circular quality to pop music. The more you are forced to listen to that inescapable new song, the more you like it. Your liking it means more music of the same type will be produced. Data, however, is making the new era of music one of the most populistic.

Spotify knows what listeners like and want. The ability to listen to music is only the most basic feature of Spotify. They constant compile data and create algorithms to suggest new music to listeners. Their Discover Weekly is like a fresh mix-tape made just for you. Powered entirely by algorithms and computers, the mixes are astonishingly well put together. While a listener may not love every single suggested track, it is lightyears from the old Pandora suggestions of 2005. Songs can be broken down into specific data points that betray not just what listeners like, but why. In fact, Spotify users create some 600 GB of this data daily. But it doesn’t stop there.

Users love apps like Spotify, but companies love Shazam, Next Big Sound, Find, and the appropriately titled HitPredictor. While Shazam is ancient in technology years, it has hit its stride among execs and professionals. Rather than scanning a library of whole songs, each song is put through an algorithm that makes it easy to find. The app has been downloaded over 500 million times and over 30 million songs have been shazamed. Yes, that is a lot of a data—natural, unsolicited data. Data that shows what songs listeners want to connect with.

Following Shazam searches can even show exactly how a song has spread. That’s right. Companies don’t just know that a song is becoming popular but where. When analyzed correctly, these numbers can easily predict the upcoming artists and songs. This is a powerful tool for labels who want real proof of a hit, rather than hunches and hopes.

Last year, HitPredicor accurately predicted 48 of the top 50 hits. Thanks to their algorithms, there is no longer a need for talent scouts to go crawling through bars, or even overly rely on their “gut.” Proposals are handed in attached to a real-world indicator of popularity (like Shazam search numbers).

This could lead to some very unexpected discoveries. Artists who would otherwise never make it out of their state are much more visible to big, global companies. However, this also creates a large degree of concern among musicians. Allowing listeners to pick the next artists actually means creating a lot of the same music. Because listeners are happy to hear sounds and styles they already know, we are happily creating a bubble of louder, less diverse music. It seems data-driven music has created an incredible paradox. Any song from any singer now has the capacity to get discovered, yet we are fueling an unusually homogenous series of artists and albums. Only time will tell what the data-driven radio will bring.

Using data and algorithms to create perfect music

The logical extension of data-driven music is data-created music. Don’t just wait around for the next big star to pop up—engineer them from scratch. For many, this is a horrifying, terrible idea. What if the art of music creation could be entirely removed from the process? A recent study on human- versus robot-made music may restore faith in humanity’s future.

Researchers from both Harvard University and the Max Planck Institute in Göttingen, Germany studied a drummer and his drumming patterns (among other subjects). The goal was to find what made the rhythm more or less appealing to listeners. Of course, a computer program could produce the same rhythm. Moreover, it could produce the rhythm perfectly, with zero flaws. However, the human ear tends to dislike that absolute perfection. It simply sounds off. This is why such programs add a “humanizing” option to change the music enough to make it imperfect. Attempting to add random bouts of imperfection to make the music humanized, however, does not generally succeed.

The team found that human error was not quite random. Changes in tempo revolve around the human clock. There are rhythms in the human brain that don’t exist in a computer. This is what physicists suspect to be the culprit behind the human distaste for purely digitized music. Moreover, the result is that human-made errors don’t occur at random. They have long-range correlations. Holger Hennig, first author of the study, explained this perfectly for the Harvard gazette.

“For example, the drummer plays ahead of the beat for 30 consecutive beats, while half a minute earlier, he tended to play slightly behind the metronome clicks. These trends are pleasant to the ear.”

However, even if data were to be pulled from around the globe to create the infectious rhythm of a sure-fire radio hit, it would not necessarily mean success. Even after pinpointing the exact details of desired rhythmic fluctuations, this did not mean human ears accepted algorithm-based music (or humanization) as preferable. Rather, the magical numbers remain elusive.

There is one example of data-created music. The “Data-Driven DJ” is a project intended to, in the DJ’s own words, “explore new experiences around data consumption beyond the written and visual forms by taking advantage of music’s temporal nature and capacity to alter one’s mood.” By transforming numbers and charts into sound, a new genre of music is created. Thus far, he has made music out of the global refugee movement, Beijing’s smog and air quality data, as well as data on race and attraction. This idea is intriguing, as high art forms often trickle down into pop culture. While this is a highly specialized use of data, the Data-Driven DJ may be driving much more in the long run.

It seems music creation will remain in the hands of musicians for some time to come. Data and analysis has reinvigorated the music industry by measuring responses. It makes it easier than ever to see how populations are responding to new music. Data-driven creation, however, has not changed excessively. Yes, music companies can use data to infer what style will be the most profitable to fund, but data is not creating music from scratch. Yet.

Like this article? Subscribe to our weekly Newsletter.

Over the past few years, music subscription services have seen incredible growth, with revenues up by 51 percent in 2013 and the $1 billion threshold crossed. In contrast to that, over a decade of stagnation has seen Europe’s overall record sales grow by just 4.3 percent. Undoubtedly, services like Spotify, Deezer, Grooveshark and Pandora have all played a major role in maintaining the industry’s steady progress.

These companies have revolutionised the way people listen to music – especially Spotify which delivered about 4.5 billion hours listening time in 2013 alone – and it is beyond doubt that their use of big data has played a pivotal role in the way the industry has evolved. These streaming services use data not only to enhance engagement and provide a personalised experience, but also to discover upcoming artists and make forecasts on their potential for success.

Spotify, which is allegedly preparing for an IPO later this year, is a great example of the power of big data analytics. With over 6 million paying subscribers, 1.5 billion playlists and more than 20 million songs, an estimated 1.5 terabytes of compressed data is produced daily. More poignantly, the company has one of the biggest commercial Hadoop clusters in Europe, with over 694 heterogeneous nodes running approximately 7,000 jobs a day.

Interestingly, the company used its collection of data last year to see whether the decisions of the Recording Academy (the people who vote on the winners at the Grammy Awards) reflected the habits the public on the streaming service. Out of 8 categories, Spotify attempted to predict the winners by looking at its users’ data (listening habits, subscribers to a playlist, popularity of an artist, etc.) The accuracy? 67%. That figure may seem low, but you should consider how many nominees had to be considered.

Spotify’s predictive capabilities will only improve. In March, the company spent $200 million to acquire Echo Nest, one of a number of companies involved in music analytics and recommendation technology. The company mines data from millions of songs and has compiled about a trillion data points from 35 million songs by 2.5 million artists.

As Echo Nest’s co-founder, Brian Whitman, describes:

“We crawl the web constantly, scanning over 10 million music related pages a day….Every word anyone utters on the Internet about music goes through our systems that look for descriptive terms, noun phrases and other text and those terms bucket up into what we call “cultural vectors” or “top terms.””

Examples pertaining to big data analytics are not confined to Spotify; they are widespread across the industry. Earlier this year The Guardian reported on music analytics firm, MusicMetric, was able to predict with 90% accuracy who would be number one three months in the future. Describing the process involved, Gregory Mead, the chief executive behind Semetric (the company behind MusicMetric), said:

“It’s no different to a sensor in a factory that’s detecting vibrations on a machining piece and when the vibrations start vibrating in a particular way they can detect that it’s going to fail….For a big artist like Katy Perry, there’s about 19,000 different signals we have just for that artist.

With this data, MusicMetrics can turn those “signals into useful information a manager can use when deciding where to take their artist on tour, or that a talent scout can use when deciding which recommendations are worth following up.” (Source)

Similarly, Shazam utilises a combination of critics’ reviews and the number of times people look up a song to judge an artist’s popularity. In so doing, it can predict which songs, and in which cities, a particular artist will gain recognition.

For example, when ID tags showed up in Shazam’s database identifying that the rapper SoMo’s single “Rise” was being searched in Baton Rouge, Louisiana, and Phoenix, the artist’s label, Republic Records, made the decision to push the song to other major cities in the States — landing it in iTunes’ R&B Top 10 a short while after. As Peter Szabo, Shazam’s head of music, notes:

“Our data has shown that we can typically predict 33 days in advance what’s going to be at the top of the Billboard Hot 100. It’s fun to see the epidemic start to spread — the growth of these songs, starting in a city.”

Unsurprisingly, Shazam’s predictive capabilities are garnering serious corporate attention. Warner Music Group signed a deal with the app this February, and interest from record companies has reached a level that Shazam is considering charging companies for additional statistical insights.

Music is becoming yet another industry to embrace data as a powerful resource. Where an artist’s success would once have relied on time and some degree of luck, the music industry is now leveraging data science to harness consumer insights and encourage emerging talent at its most nascent stages.

Follow @DataconomyMedia

Furhaad Shah – Editor

Furhaad worked as a researcher/writer for The Times of London and is a regular contributor for the Huffington Post. He studied philosophy on a dual programme with the University of York (U.K.) and Columbia University (U.S.) He is a native of London, United Kingdom.

Email: furhaad@dataconomy.ru

Interested in more content like this? Sign up to our newsletter, and you wont miss a thing!

[mc4wp_form]

(Image Credit: Rich Jacques)