Dr. Joss Wright is a Research Fellow at the Oxford Internet Institute (OII), where his current research focuses on analysing Internet censorship and data anonymization. Prior to the OII, Dr. Wright worked at the University of Siegen in Germany examining security and privacy issues in cloud computing. He has a PhD in Computer Science from the University of York.

There is a lot of debate about privacy. Where it came from, where it is going, and what it means for society. Undoubtedly, privacy is certainly under threat and will never be the same again. A lot of people will point out that privacy did not really exist in law internationally until quite recently. The first really significant bit of law was a 1898 legislation in the United States, from Samuel Warren and Louis Brandeis, who defined privacy as the right to be let alone. However, privacy has really existed long before this; it was just, and this is slightly controversial, more intrinsic.

We worked on a human scale back then. You said something to someone, and they could re-tell it, a rumour could spread, a story could be told, but it was on a human scale. You would forget and everyone knew it would change in the telling. Then technology brought about this erosion of a right that had always been very intuitive. Even now when you ask somebody to define privacy, it’s very very tricky, but if you say to somebody x, y, z happens – has your privacy been violated? People can instantly say ‘yes’, or ‘no’.

We must look at these tech companies also. Google, for example, makes over 95 percent of their profit from targeted advertising. We are now working on a scale we were not built to predict. As a human, we can’t make good privacy decisions; we get short-term easy rewards like access to Facebook, or access to Gmail. The privacy risks that come with that, the risks of our data being used against us, or being used in a way that is not within our control, is a long-term potential probabilistic concern.

I think privacy is something we can still preserve, albeit not to the same extent we used to be able to. I think we should try, not in an attempt to fix a status quo of ‘what is private now should always be private’, but to guide a society towards a society we want to live in, so that we do not have the risk of all data being shared with everyone, and have this transparent society like David Brin writes about.

We need to build systems that do that, and we need to have legal backing to enforce the companies that do not treat data they are not suppose to treat with strong sanctions, like the European Union is doing with the proposed general data protection regulation that is coming into force hopefully in 2016. These strong sanctions against companies will go a long way.

Follow @DataconomyMedia

Interested in more content like this? Sign up to our newsletter, and you wont miss a thing!

[mc4wp_form]

The use of socially generated “big data” from our daily activities has become a new technique to understand and predict our collective behaviours. Big data techniques have multifarious applications — for example predicting flu outbreaks based on the volume of tweets mentioning flu-related keywords, understanding the patterns of human mobility by analysing the records of mobile phone calls, or forecasting the financial success of a movie by studying the page view statistics of the Wikipedia articles about the movie. What all these examples have in common is the concept of quantifying and measuring activity of individuals at the collective level to understand and model human societies in a computational framework.

The desire of mankind to know about the surrounding world has led to huge advancements in knowledge in recent centuries. Today we know a lot about the universe; from the super small elementary particles to very far galaxies, there is nothing left not yet investigated by scientists. Although there are still many unanswered questions, the high pace of knowledge creation and the increase in our understanding of nature is undeniable. Examples of inventions in medicine, natural sciences, engineering, are numerous and their effects on our daily life is clearly evident.

However, despite the considerable amount of effort, our understanding of ourselves and more precisely human societies is underdeveloped. In contrast to the huge developments in science and technology, our societies are still suffering from very old and basic problems. Social unrest, riots and crimes, political conflicts and wars, economic crises, poverty, inequality and dictatorship are only few examples of the social disorders that we still have not found any solution for. Compared to the natural sciences and technological advancements, social scientists have not had the same kind of success; improvement in knowing our societies has been very slow.

The huge improvements in natural sciences in 17^th and 18^th centuries, especially in physics, are mostly due to the new convention of modern science, which is based on experiments, measurements and quantitative modelling. Only by following this, scientist have been able to understand the universal patterns and laws which govern the natural phenomena in a very accurate way such that by knowing the current states of a system, in many cases, the future behaviour of it is predictable.

In contrast, in social sciences, performing near real-experiments, quantifying and measuring involved parameters, and providing a mathematical model to describe empirical observations are all quite challenging and in many cases impossible. In studying natural systems, one could observe and monitor the system under study continuously and perform all the necessary measurements. Whereas when we study social systems, not only is complete observation of all the actions and interactions very difficult, but it is also challenging to definine measurable parameters.

How can we quantify the level of dissatisfaction of the members of a society? How to measure the kindness of a person? How can we define the strength of social interactions and peer pressures? And even if we are able to do so, how do we monitor the system and record all these parameters continuously and under different conditions? These kinds of questions have made the social sciences limited to qualitative descriptions of observations without any ability to predict and forecast the future behaviour of the system accurately.

However, things are about to change. Our lives are being transformed to a digital world, where our social interactions leave a digital footprint. Our daily social transactions are being recorded and producing a Big amount of Data. The amount of digital data that we produce through our daily life activities, ranging from financial activities in online banking and e-commerce, our social communications via phones and online social networks, to our online socio-political movements such as online petitions and campaigns, is huge. Most of these data are being recorded and stored for various reasons; your cell phone provider records your communications to be able to issue you the bill, and Google records your search queries to provide better search results in the future. Amazon analyses your purchases to make more accurate product recommendations, and Facebook keeps track of your “likes” and “pokes” to facilitate your online social networking. Apart from various uses and applications that recording and analysing these data could have in enterprise and corporations, a very important usage of it would be in the recently emerging field of Computational Social Science.

These days, one could quantify and measure the popularity of a politician by considering the number of her twitter followers or the likes given to her Facebook posts. This is a very easy task compared to classic methods of social science based on surveys and questionnaires. Today, by analysing the volume of Google search queries for relevant keywords, scientist can forecast the financial moves in the markets and by counting the number of edits to Wikipedia articles about movies, box office takings can be predicted with ground-breaking accuracy. And more importantly, now we can perform large scale analysis to reveal the gender dependent features of our communication patterns.

If the invention of telescopes provided us with the ability to understand how galaxies behave, and the microscope allowed us to find the cure of such a huge amount of diseases, this century we are going to understand much more about the social systems because of big data. There is no doubt that humans are much more complicated than atoms or even planets and stars, but with the help of powerful mathematical tools and our ever-faster computers we will be able to find and reveal the universal laws of human societies in a numerical framework.

It is easy to imagine the future “smart-cities” being designed to be functioning based on the real-time data analysis of our daily activities. Adaptive transport systems which self-organise themselves to optimise the flow, better and more efficient health services elaborated by better prioritisation of demands and assignment of resources, more fluid and transparent financial models and more democratic and horizontal processes of data-driven policy making are all possible practical outcomes of the data revolution.

Big data techniques and their use in computational social sciences will provide us the ability to cope with socio-economical crises in a better way and decreases the costs of taking blind risks based on inaccurate qualitative speculations. Our “self-aware” societies of the future will be better places to belong to.

Dr. Taha Yasseri is a researcher at the Oxford Internet Institute, University of Oxford.  His main research focus is on online societies, government-citizen interactions on the web and structural evolution of the Internet. He uses mathematical models and data analysis to study social systems quantitatively. Prior to Oxford Internet Institute, he spent two years as a Postdoctoral Researcher at the Budapest University of Technology and Economics, working on socio-physical aspects of the community of Wikipedia editors, focusing on conflict and editorial wars, along with Big Data analysis to understand human dynamics, language complexity, and popularity spread. Taha completed his PhD on spontaneous pattern formation in complex systems.

(Image Credit: William Pearce)

My colleagues and I have devised a mathematical model which can be used to predict films that become blockbusters or flops at the box office – up to a month before the movie is released.

Our model is based on an analysis of the activity on Wikipedia pages about American films released in 2009 and 2010. After examining 312 movies, taking into account the number of page views for the movie’s article, the number of human editors contributing to the article, the number of edits made and the diversity of online users, we could come up with good estimations for the prospective popularity of a movie at box office. The results obtained using this model, and the actual figures (published in Internet Movie Database (IMDb)) showed a high degree of correlation.

Actual first weekend box office revenue in the United States against its predicted value based on Wikipedia data 30 days before the release. The green line, indicating the perfect prediction, is drawn for comparison. Each dot represents a movie from the sample and the size of the dot indicates the amount of the error in the prediction. Predictions for more successful movies are more accurate.

Their mathematical algorithm has allowed us to predict box office revenues with an overall accuracy of around 77 per cent. This level of accuracy is higher than the best existing predictive models applied by marketing firms (which they estimate to be at around 57 per cent). We could predict the box office takings of six out of 312 films with 99 per cent accuracy where the predicted value was within one per cent of the real value. Some 23 movies were predicted with 90 per cent accuracy and 70 movies with an accuracy of 70 per cent and above.

The more successful the show, the more accurately we were able to predict box office takings. This is possibly due to the increased amount of online data generated by films that turn out to be successful. The model correctly forecast the commercial success of Iron Man 2, Alice in Wonderland, Toy Story 3 and Inception, but failed to accurately forecast the financial return on less successful movies Never Let Me Go, and Animal Kingdom.

These results can be of great value to marketing firms but more importantly for us; we were able to demonstrate how we can use socially generated online data to predict a lot about future human behaviour.

We have demonstrated for the first time that Wikipedia edit statistics provide us with another tool to predict social events. We studied the problem of predicting the financial success of movies and concluded that, in some aspects, forecasting based on Wikipedia outperforms tweets as Wikipedia activity has a longer timescale which enables earlier predictions.

The efficiency of the predictions might be improved by applying more sophisticated statistical methods, such as including the controversy measure of an article.

Taha Yasseri is a Big Data Research Officer at the Oxford Internet Institute. Prior to Oxford Internet Institute, he spent two years as a Postdoctoral Researcher at the Budapest University of Technology and Economics, working on socio-physical aspects of the community of Wikipedia editors, focusing on conflict and editorial wars, along with Big Data analysis to understand human dynamics, language complexity, and popularity spread.

This Research has been published in PLoS ONE and can be accessed at “Mestyán, M., Yasseri, T., and Kertész, J. (2013) Early Prediction of Movie Box Office Success based on Wikipedia Activity Big Data. PLoS ONE 8 (8) e71226.”

(Image Credit: Brett Sayer)