Could we use Twitter to predict the future?

Albert Arnold Gore Jr, the 45th Vice President of the United States, once said: "I have always been fascinated with those who try to look over the horizon and see things that are coming at us." The concept of knowing unknown and predicting the future has always fascinated humans since the beginning. There were many unexpected, as some would say, events which recently took place across the world including:

• Brazil and South Korea impeaching their presidents
• The coup in Turkey failing
• Britain voting to leave the European Union
• Donald Trump winning the U.S. Presidency

The most interesting aspect of these events was that the outcomes were mostly unexpected and led to many continuous small events. This phenomenon highlights that the event is not a one-off occurrence of something, rather it is a continuous process. For example, Donald Trump winning elections led to major changes in immigration policy, which later resulted in skill worker shortage in few sectors. Moreover, the United States of America pulling out from the Paris Climate Agreement and its impact on green energy economy.

Many cooperate companies are particularly interested in knowing various events which would have a direct impact on commodities such as, oil, metal and carbon credit. Another example to understand this is Brexit. Immediately after Brexit's referendum result, the financial markets in UK went into turmoil. The pound collapsed and till today it seems to be in volatile state.

So, what if?

The above mentioned real-world examples, motivate researchers to ask questions such as what if we could predict the outcomes of the events before they occur and predict the linkages it may create over the time with other emerging but premature events?

The use of various social media platforms is massively increasing. Just in the UK, it is estimated that more than 80% of the households will have access to internet and they use internet on go using smartphones and other devices. This sudden rise also provides us the opportunity to analyse the streaming data it produces. The first research question which arises is how do we effectively monitor and analyse social, academic and industrial information sources for decision making?

The role of big data

The velocity in which the data is generated is faster than ever before. It is in a host of varieties with varying veracity and complexity. By the year 2020, it is estimated that 1.7 megabytes of new information will be created every second for every human being on this planet. This will increase 4.4 zettabytes data of today to around 44 zettabytes (44 trillion gigabytes). This expanding sphere of the digital universe, constitutes data from all sources in various formats; including images, videos, social media, electronic sensors, publications and etc.

In the public sector, from the National Health Service, to managing driving licenses and issuing passports, the use of Big Data is witnessed. For instance, the HM Revenue and Customs hold approximately 80 times more data than the British Library. In public and private sector it is observed that big data analytics has reduced the work force by increasing productivity and precision, compare to human beings. We witness the use of big daily analytics daily, whether, while using Tesco Clubcard, Google search results or Amazon recommendations. The analytics of big data could be categorised into two categories; examine previous patterns and correlation or make a predictive analysis. The latter mentioned approach could be combined with scenario planning to provide forecasts in specific domains.

What role does the internet play?

Over the years, Web 2.0 applications have boomed because of the new internet pattern of users creating their own data. Online social networks have become the most popular Web 2.0 application for people to communicate and distribute ideas, news, and advertisements in virtual world.

Moreover, due to the easy and convenient access, online social networks bring almost all sorts of real-life social events together for online users in real-time. Among social networking sites, Twitter is widely used as an effective source of data collection. Twitter is a form of microblogging where people share their views and updates them in real-time. The use of Twitter for getting sentiments linked with opinion, elections, and changes in stock market remains active research area.

To monitor social, academic and industrial information sources and apply novel analyses to predict trends, present problems and semantically link developments based on user-defined criteria are active research topics. I am currently conducting research to develop a new data analytic system and supporting techniques to scan and analyse the relevant data in a considerably more efficient and timely manner to find various events for enhancing the decision-making process based on emerging cutting-edge technologies in data mining, data processing and horizon scanning.

Could Twitter help predict the future?

Yes, we could use Twitter to predict future events. In my research, we performed experimentation and our preliminary results are encouraging which show that the outcome of Brexit's referendum could have been predicted. Moreover, some researches are already published which indicates the use of Twitter to predict earthquakes and various disease outbreak.