Big Data in Politics

With the U.S. in the early stages of a presidential election I started asking myself what role big data can play when it comes to elections.  I did a little bit of reading and it turns out big data has a huge role to play.

To improve their chances of success campaigns need to use every dollar wisely.  There is a saying that “political campaigns run on dollars” and that is very true.  Very few candidates have unlimited budgets and so it is very important to make the most of the money that a candidate has.  This can be achieved by using data about voters to run a campaign more efficiently. 

Since I started this post talking about the U.S I will use that country as an example.  Barack Obama’s presidential campaigns in 2008 and 2012 are often cited as examples of the power of big data in election campaigns. Two technological tools the Obama campaign team developed were called Narwhal and The Optimizer.  Narwhal integrated voter registration data with online activity to determine potential supporters.  Once these potential supporters were identified the campaign followed up on these leads through various activities to persuade these potential supporters to vote.  The Optimizer was geared towards television and was used to place targeted television adverts in front of certain audiences all while minimising cost.

The theme that emerges from the Obama example is that of targeted campaigning whereby campaign messages are tailored to suit certain audiences.  The power of big data is in identifying these audiences, figuring out where they physically are and determining what is important to them.  Of course historical election data can be used for this but there is the danger that the data is outdated.  By harnessing social media and other more current sources of data a campaign can develop a much more accurate picture of the electorate.

Targeted campaigning is much more effective than a blanket approach because a campaign is better able to connect with voters by speaking to them about their specific concerns.  If someone shows concern for the environment why bombard them with information on national security.  Rather spend more time getting your green energy policy across to them and explaining its merits. In doing this not only do you forge a better relationship with voters but you use your resources much more efficiently and get more out of every dollar you spend.

What data is collected?

So what sort of data is collected anyway?  Well it’s any data that can help understand a citizen better.  This includes age, race, gender, education level and income.  In terms of online activity it would be the websites a person visits, social media activity or which campaign emails devoted to which issues did they open.  All of this data is useful in building a profile of a potential voter.

Potential challenges

As with all big data initiatives data privacy is a massive concern.  There are laws governing what data can be stored about an individual and it is important that a campaign be aware of and abide by these laws.  It also important to safeguard the information that is stored as the threat of data theft is very real and can have negative ramifications for the campaign.  You do not want to go through all the hard work of convincing people to vote for you only to lose their support over a data breach. Another problem is the lack of skills.  Data scientists are in short supply and decision makers may not possess the necessary skills to apply what the data is telling them.  There are many ways to address privacy in skills challenges related to big data so I will not delve into them here.

In my earlier posts I mentioned that big data has many applications beyond business.  Politics is another example of this.  Although the context may differ the goal of big data remains the same; to use data to make better decisions.  It would be interesting to find out on a global level the extent to which big data has been adopted in elections.  Do people buy into the concept as yet or are they still sceptical?  What do you think?

Big Data Events

I found two conferences for anyone interested in learning more about big data.

Big Data Innovation Summit
This is a conference being held in Las Vegas, U.S.A on the 28th and 29th of January 2016.  The conference boasts some well respected speakers who are experts in the field and is a great opportunity to learn and network.  Also, it's in Vegas!
http://theinnovationenterprise.com/summits/big-data-innovation-summit-las-vegas-2016

MIT Sloan Sports analytics Conference
In one of my previous posts I discussed Big Data in sports.  The MIT Sloan school of business management in conjunction with other partners have been hosting this conference since 2009.  It provides an opportunity for professionals, fans and students to come together to discuss and learn about sport analytics and the growing role of big data in that space.  The conference is being held on March 11 and 12 2016 in Boston, U.S.A.

http://www.sloansportsconference.com/?page_id=2

Addressing the Big Data Skills challenge

Finding the necessary skills to help you on your big data journey can be a challenge.  You need people to help you setup your technological environment and then people to help you use and maintain that environment.  For help with implementing your big data technology you can just turn to the vendor.  They will undoubtedly offer services to install and configure the products because that is one way they make money.  Where you will struggle is finding people to help you use that technology in your organisation, specifically with the interpretation of the data.  These people are commonly referred to as data scientists although they can have other titles (IT industry loves creating titles).  Data scientists are typically masters and phd graduates in statistics or data science and are hard to find in large numbers.  McKinsey estimates that the U.S alone faces a shortage of up to 190 000 people with such skills.  So if you can’t find any of these data scientists to employ what can you do?  Well you can make them.  If you have an analytics department it is possible that you have employees that have the aptitude for data science.  You can send them on a few courses or even an entire degree to learn the necessary skills.  It may be slower that employing someone directly but it will pay off as you are will have someone who already knows the business and will have a good idea how big data can be leveraged.  Another option is to create a team with data science capabilities.  You do not necessarily need to employ one person with all the skills, you can employ a team of people who collectively have the necessary skills such as statistics, business domain knowledge, business analysis and so on.  There are ways around this skills shortage, you just need to get creative.

Big Data: Just passing through or here to stay?

There have been many technologies and concepts in IT over the years.  Some of them endure and become mainstream, for example ERPs, speech recognition, mobile commerce and smart phones just to name a few. Ping Wang published a paper about chasing the hottest IT in which he described such technologies as fads. Other technologies seem to wither and die a slow death, for example service oriented architecture (SOA) for those of you who remember what that is. Richard Baskerville and Michael Myers in their paper titled ‘Fashion Waves in IS research and practice’ described these transitory technologies as fashions. So what is big data, a fashion or a fad?  Well according to these two papers fashions and fads are quite similar in the initial stages.  Big data certainly ticks that box.  It has been progressing through the Gartner hype cycle (see below) and is one of the most searched   for terms on the internet.  The difference between a fashion and a fad is what happens when the technologies mature.  A fad will eventually become mainstream while a fashion quickly disappears.  Unfortunately it is still too early to tell whether big data is a fashion or a fad as the concept and associated technology has not become mainstream as yet.  When Big Data does become mainstream I believe it will stick around for a long time because if you look at the technology that underpins big data such as the internet of things, wearable computing, social media, these technologies are unlikely to go away anytime soon which means the data they generate, which is big data, is not going to go away. People are not going to suddenly stop using Facebook and Twitter or ditch their smartphones. Therefore, in my humble opinion, big data is a fad and is going to be with us for the foreseeable future.

 

Big Data in Practise

 

For those of you who may be considering implementing a big data solution in your organisations I thought it would be useful to examine how other organisations have applied big data.  Below are three interesting cases:

 

Dickeys Barbecue Pit is a restaurant chain which has 514 restaurants throughout the United States.  Dickeys uses a big data software system called Smoke Stack.  Smoke Stack collects and analyses data from diverse sources such as point of sale systems, marketing promotions, inventory systems and customer surveys in order to provide accurate real-time feedback on restaurant performance.  This enables Dickey’s management to react to events on the ground at certain restaurants much faster.  For example if one of the restaurants experiences low sales at lunchtime and has excess inventory such as ribs then a ribs special can quickly be communicated to the surrounding community in order to improve the sales and reduce the excess inventory.

 

Sears Holdings is an American corporation that owns the retail store brands Sears and Kmart.  Several years ago Sears Holdings was struggling to put together personalized promotions in a timely manner.  Due to the large volume and fragmented nature of their data it was taking Sears up to 8 weeks to create a personalized promotion package for a customer.  To address this Sears turned to Hadoop, a big data technology.  Sears started using Hadoop to store their incoming data and also integrated their existing data warehouses into Hadoop.  This enabled Sears to bypass the time consuming step of integrating data from different sources.  The result: the time taken to create a personalized promotion package dropped from 8 weeks to 1 week.

 

New South Wales State Emergency Services (NSWSES) is the organisation responsible for providing emergency services such as natural and man-made disaster relief in the state of New South Wales in Australia.  The agency is responsible for an area of 800 000 square kilometres and relies on a core staff of 280 supplemented by over 9 000 volunteers.  NSWSES also works closely with the meteorology service and the state fire department.  Given that time is critical in saving lives in disaster situations NSWSES felt that in order to perform their role more effectively they would need to integrate and analyse data from multiple sources, such as the meteorology service and social media, and also be able to share information with other services and the public in real time.  Using a combination of SAP and Web 2.0 technologies NSWSES create a platform that enables the agency to communicate and collaborate with external parties as well as manage internal resources much more efficiently.  The agency has been able to derive several tangible benefits from their big data implementation.  For example, real-time monitoring of disaster related information enables management to quickly identify where personnel are most needed.  Another benefit that the big data solution offers is predictive analytics.  Predictive analytics helps the agency to anticipate natural disasters before they happen enabling response plans to be activated in advance.

 

These are just three examples of big data at work in organisations.  There are many more stories out there.  What is important to note is the different industries that these organisations are in.  This highlights how big data analytics can be applied in many different industries.  As long as you collect and analyse data in your organisation there is a possible need that big data can fulfil so don’t immediately dismiss big data because you think you cannot make use of it.

 

Three things to think about when considering whether to start collecting and analysing big data

 

 

1)      Where will you get your big data from?

You need determine what the possible sources of big data are for your organisation. Examples of big data sources include sensors, RFID tags, audio and video files, internal and external reports and social media.  Some of the external sources are free and publicly available such as government and company reports for example.  Here is a nice article categorising big data sources. 

 

2)      Will big data add value to your organisation?

Collecting data is not hard, figuring out what to do with the data is the difficult part. Think about how big data will add value.  The main benefit of information is to enable better decision making.  So ask yourself what do I need more information about in order to improve my business?  Do you need to know more about, for example, your customers, your manufacturing processes, the political environment or the stock market?  Having some business cases in mind will give your big data initiative some direction.

 

3)      Educate yourself

When embarking on an IT initiative it is important to understand the technology or the concept so it can be applied appropriately.  The more you understand the technologies and concepts that underpin big data the better able you will be to extract value from them.  If you are really interested in getting into the nuts and bolts of big data analytics you could always go back to school.  It might be a good opportunity to take that sabbatical you always wanted.  This article here is a nice list of graduate programs related to data analytics and data science.  Just study full-time if possible.  As a part-time masters student I can assure you part-time study can be a painful exercise!

 

 

Can Big Data be used in sports?

Is there a place for Big Data analytics in Sports?

Since there is currently a rugby world cup taking place I thought it would be appropriate to discuss whether big data can be applied to sports.  It turns out big data actually has a massive role to play in sports and not just to examine player performance but to manage injuries and analyse fan and spectator behaviour as well.

Which Sports codes use big data?

Several sports codes use big data analytics.  In football (or soccer if you are from North America) Arsenal FC of the English premier league are using a system developed by a sports analytics provider called Prozone.  The system uses cameras installed in a stadium to track players and their interactions every second.  The data is analysed automatically using algorithms embedded in the technology and used to provide insights on player performance.  Big data is also used in Rugby.  IBM have developed a system called TryTracker that uses data from previous matches between two opponents to determine the targets that either team needs to achieve in order to win a game.  A target can be, for example, a certain number of line breaks or goal kicking accuracy percentage.  These are just two examples of sports that use big data analytics.  Other sports such as basketball, tennis, formula one, cycling, baseball and mixed martial arts also make use of big data.

Big Data success stories in sports

One of the more famous examples of big data in sports is the use of big data by the German men’s national football team for the 2014 FIFA World Cup.   A team of German university students in conjunction with SAP and the football team manager compiled a database of football related information called Match Insights.  Match Insights contained information about both the German team and other national teams.  Using this database the team management was able to provide much more specific information to the players about their own individual performance as well as analysis on opposition players.  At a team level the data helped improve several facets of the Germans’ game such as their possession speed.  Analysis of the 2010 world cup showed that the German team had an average of 3.4 seconds on the ball.  Once aware of this the team management adjusted the style of play and the time was reduced to 1.1 seconds.  All teams at the 2014 tournament used data analysis of some sort but Germany seems to be the only one that went to the extent of building a database with a custom built application that provided such in depth analysis.  The extent to which the team performance improved because of Match insights is debateable.  However given that the Germans won the tournament and were the only ones to use Match Insights this does suggest that there was some benefit derived from using Match Insights.

Big data analytics is not only used for analysing players

Big data analytics is also used to analyse fans as well.  According to Christy King, Vice President of IT for Ultimate Fighting Championship (UFC), big data analytics can be used to improve the fans’ experience at a venue.  For example high foot traffic locations can have memorabilia stands close by and real time monitoring of the bathrooms can tell spectators which bathrooms have shorter queues.  Big data analytics can also be used to examine spectator emotions.  Academics did a study using social media analytics on tweets from the U.S.A during the 2014 men’s FIFA world cup football when the U.S.A team was playing.  The findings confirmed what many people already knew which is that when a person is watching a team that they support that person becomes more heavily invested in the game and shows signs of fear and anxiety as well as happiness.  However when a person is watching two teams that they have no interest in there mainly feelings of happiness and enjoyment.  Although this research did not provide any surprising insights it does demonstrate how big data can be applied to fans..

Wearable computing enabling sports analytics

One of the things that stands out for me when considering big data in sports is how big data is enabled by other trends.  In the case of big data and sports, wearable computing plays a big role.  GPS trackers, heart rate monitors and other gadgets to monitor performance are small enough to be worn on the body of players which makes big data analytics in sport possible.  It would be much harder to analyse aerobic performance if training sessions had to continuously be interrupted so a player’s breathing rates could be measured by medical staff.

Big Data does not replace coaches though

One of the questions around big data is whether or not it eliminates the need for human analysis or intuition in decision making.  Are the instincts and knowledge that coaches have gathered over years of involvement with the sport becoming obsolete?  I don’t believe so.  Big data is simply providing better information on which to base a decision on.  Consider fantasy football (or soccer) for example.  You can have information on two opposing teams such as win-loss ratios, goals scored and against, number of shots on target and yellow cards received.  This information will give you an idea of who is likely to win from a statistical point of view.  However statistics do not always tell the full story of how dominant a team was in the last game or how close a game was and how this will impact on the team’s next performance.  Only experience and intuition will inform that aspect of sports knowledge.  Therefore analytics and intuition are not mutually exclusive but rather complementary.