Think Bigger: Developing a Successful Big Data Strategy for Your Business
3/5
()
About this ebook
Offering real-world insight and explanations, this book provides a roadmap for organizations looking to develop a profitable big data strategy and reveals why it's not something they can leave to the I.T. department.
Big data--the enormous amount of data that is created as virtually every movement, transaction, and choice we make becomes digitized--is revolutionizing business. Sharing best practices from companies that have implemented a big data strategy including Walmart, InterContinental Hotel Group, Walt Disney, and Shell, this helpful resource covers the most important big data trends affecting organizations, as well as key technologies like Hadoop and MapReduce, and several crucial types of analyses.
In Think Bigger, you will find guidance on topics such as:
- how to ensure security,
- respecting the privacy rights of consumers,
- and how big data is impacting specific industries--and where opportunities can be found.
Big data is changing the way businesses--and even governments--are operated and managed. Think Bigger is an essential resource for anyone who wants to ensure that their company isn't left in the dust.
Mark Van Rijmenam
Mark van Rijmenam is the founder and CEO of Datafloq.com. He is an expert on AI, Blockchain and Big Data, a highly sought-after speaker and author of the book Think Bigger. He is also co-author of the book Blockchain: Transforming Your Business and Our World, on how blockchain can be used for social good. He is named a global top 10 Big Data influencer and one of the most influential Blockchain people. He is pursuing a PhD at the University of Technology, Sydney on how organizations should deal with Big Data, Blockchain and AI and he is a faculty member of the Blockchain Research Institute.
Related to Think Bigger
Related ebooks
Digital Transformation at Scale: Why the Strategy Is Delivery Rating: 0 out of 5 stars0 ratingsThe Big Data-Driven Business: How to Use Big Data to Win Customers, Beat Competitors, and Boost Profits Rating: 0 out of 5 stars0 ratingsDigital Transformation at Scale: Why the Strategy Is Delivery: Why the Strategy Is Delivery Rating: 5 out of 5 stars5/5Delivering on Digital: The Innovators and Technologies That Are Transforming Government Rating: 0 out of 5 stars0 ratingsWTF?: What's the Future and Why It's Up to Us Rating: 4 out of 5 stars4/5The Billion Dollar Byte: Turn Big Data into Good Profits, the Datapreneur Way Rating: 0 out of 5 stars0 ratingsData Crush: How the Information Tidal Wave Is Driving New Business Opportunities Rating: 4 out of 5 stars4/5Big Data for Executives and Market Professionals - Third Edition: Big Data Rating: 0 out of 5 stars0 ratingsWhat Is Big Data Rating: 0 out of 5 stars0 ratingsPractical DataOps: Delivering Agile Data Science at Scale Rating: 0 out of 5 stars0 ratingsDriving Digital: The Leader's Guide to Business Transformation Through Technology Rating: 5 out of 5 stars5/5Nice Companies Finish First: Why Cutthroat Management Is Over--and Collaboration Is In Rating: 0 out of 5 stars0 ratingsDigital Master: Debunk the Myths of Enterprise Digital Maturity Rating: 0 out of 5 stars0 ratingsAnalytics: The Agile Way Rating: 5 out of 5 stars5/5Going Digital: Harnessing the Power of Digital Innovation Rating: 0 out of 5 stars0 ratingsThe 24-Hour Customer: New Rules for Winning in a Time-Starved, Always-Connected Economy Rating: 5 out of 5 stars5/5Message Me: The Future of Customer Service in the Era of Social Messaging and Artificial Intelligence Rating: 0 out of 5 stars0 ratingsBusiness Value in an Ocean of Data: Data Mining from a User Perspective Rating: 0 out of 5 stars0 ratingsFail Fast, Learn Faster: Lessons in Data-Driven Leadership in an Age of Disruption, Big Data, and AI Rating: 4 out of 5 stars4/5The End of Business As Usual: Rewire the Way You Work to Succeed in the Consumer Revolution Rating: 5 out of 5 stars5/5Lush's Hierarchy of Digital Transformation: A Prescription for Cloud Platform Value Rating: 0 out of 5 stars0 ratingsData Strategy A Clear and Concise Reference Rating: 1 out of 5 stars1/577 BUILDING BLOCKS OF DIGITAL TRANSFORMATION: THE DIGITAL CAPABILITY MODEL Rating: 5 out of 5 stars5/5Technology-as-a-Service Playbook: How to Grow a Profitable Subscription Business Rating: 1 out of 5 stars1/5Winning with Data: Transform Your Culture, Empower Your People, and Shape the Future Rating: 0 out of 5 stars0 ratingsJourney To Centricity: A customer-centric framework for the era of stakeholder capitalism Rating: 0 out of 5 stars0 ratingsData Ownership A Complete Guide - 2021 Edition Rating: 0 out of 5 stars0 ratingsConfessions of a Successful CIO: How the Best CIOs Tackle Their Toughest Business Challenges Rating: 3 out of 5 stars3/512 CIO Personas: The Digital CIO's Situational Leadership Practices Rating: 0 out of 5 stars0 ratingsThe Metail Economy: 6 Strategies for Transforming Your Business to Thrive in the Me-Centric Consumer Revolution Rating: 0 out of 5 stars0 ratings
Marketing For You
On Writing Well, 30th Anniversary Edition: An Informal Guide to Writing Nonfiction Rating: 4 out of 5 stars4/5Exactly What to Say: The Magic Words for Influence and Impact Rating: 4 out of 5 stars4/5Emotional Intelligence: Exploring the Most Powerful Intelligence Ever Discovered Rating: 4 out of 5 stars4/5Building a StoryBrand: Clarify Your Message So Customers Will Listen Rating: 4 out of 5 stars4/5The Millionaire Next Door Rating: 4 out of 5 stars4/5The Psychology of Selling: Increase Your Sales Faster and Easier Than You Ever Thought Possible Rating: 4 out of 5 stars4/5Marketing Made Simple: A Step-by-Step StoryBrand Guide for Any Business Rating: 4 out of 5 stars4/5The YouTube Formula: How Anyone Can Unlock the Algorithm to Drive Views, Build an Audience, and Grow Revenue Rating: 4 out of 5 stars4/5The Best Credit Repair Manual Ever Written Rating: 5 out of 5 stars5/5Win In Court Every Time Rating: 5 out of 5 stars5/5Everybody Writes: Your Go-To Guide to Creating Ridiculously Good Content Rating: 4 out of 5 stars4/5The Catalyst: How to Change Anyone's Mind Rating: 4 out of 5 stars4/5The Passive Income Playbook Rating: 5 out of 5 stars5/5Invisible Influence: The Hidden Forces that Shape Behavior Rating: 4 out of 5 stars4/5The Freedom Shortcut: How Anyone Can Generate True Passive Income Online, Escape the 9-5, and Live Anywhere Rating: 5 out of 5 stars5/5The Passive Income Cheat Sheet Rating: 4 out of 5 stars4/5Robert Cialdini's Influence: The Psychology of Persuasion Summary Rating: 4 out of 5 stars4/5Pre-Suasion: A Revolutionary Way to Influence and Persuade Rating: 4 out of 5 stars4/5How to Write Copy That Sells: The Step-By-Step System For More Sales, to More Customers, More Often Rating: 4 out of 5 stars4/5How to Start a Nonprofit Organization: The Complete Guide to Start Non Profit Organization (NPO) Rating: 4 out of 5 stars4/5Alchemy: The Dark Art and Curious Science of Creating Magic in Brands, Business, and Life Rating: 4 out of 5 stars4/5Advertising Headlines That Make You Rich: Create Winning Ads, Web Pages, Sales Letters and More Rating: 4 out of 5 stars4/5Quiet Leadership: Six Steps to Transforming Performance at Work Rating: 4 out of 5 stars4/5Mastering ChatGPT: 21 Prompts Templates for Effortless Writing Rating: 5 out of 5 stars5/580/20 Sales and Marketing: The Definitive Guide to Working Less and Making More Rating: 4 out of 5 stars4/5Propaganda Rating: 4 out of 5 stars4/5Built to Last: Successful Habits of Visionary Companies Rating: 4 out of 5 stars4/5
Reviews for Think Bigger
3 ratings0 reviews
Book preview
Think Bigger - Mark Van Rijmenam
THINK BIGGER
Developing a Successful Big Data Strategy for Your Business
MARK VAN RIJMENAM
American Management Association
New York • Atlanta • Brussels • Chicago • Mexico City • San Francisco Shanghai • Tokyo • Toronto • Washington, D. C.
Bulk discounts available. For details visit:
www.amacombooks.org/go/specialsales
Or contact special sales:
Phone: 800-250-5308
Email: [email protected]
View all the AMACOM titles at: www.amacombooks.org
American Management Association: www.amanet.org
This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional service. If legal advice or other expert assistance is required, the services of a competent professional person should be sought.
Library of Congress Cataloging-in-Publication Data
Van Rijmenam, Mark.
Think bigger : developing a successful big data strategy for your business / Mark van Rijmenam.
pages cm
Includes bibliographical references and index.
ISBN-13: 978-0-8144-3415-4
ISBN-10: 0-8144-3415-0
1. Information technology—Management. 2. Big data. 3. Business—Data processing. 4. Database management. I. Title.
HD30.2.V3647 2014
005.74068'4—dc23 2013032722
© 2014 Mark van Rijmenam.
All rights reserved.
Printed in the United States of America.
This publication may not be reproduced, stored in a retrieval system, or transmitted in whole or in part, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of AMACOM, a division of American Management Association, 1601 Broadway, New York, NY 10019
The scanning, uploading, or distribution of this book via the Internet or any other means without the express permission of the publisher is illegal and punishable by law. Please purchase only authorized electronic editions of this work and do not participate in or encourage piracy of copyrighted materials, electronically or otherwise. Your support of the author's rights is appreciated.
About AMA
American Management Association (www.amanet.org) is a world leader in talent development, advancing the skills of individuals to drive business success. Our mission is to support the goals of individuals and organizations through a complete range of products and services, including classroom and virtual seminars, webcasts, webinars, podcasts, conferences, corporate and government solutions, business books and research. AMA's approach to improving performance combines experiential learning—learning through doing—with opportunities for ongoing professional growth at every step of one's career journey.
Printing number
10 9 8 7 6 5 4 3 2 1
To my parents and my sister
Contents
1 The History of Big Data
The Twentieth Century
The Twenty-First Century
2 What Is Big Data?
Explanation of the 7Vs
Eight Realities of Big Data You Should Already Know
The Impact of Big Data on Society
Takeaways
3 Big Data Trends
On-the-Go Big Data
Big Real-Time Data
The Internet of Things
The Quantified-Self
Big Social Data
Public Big Data
Gamification
Takeaways
4 Big Data Technologies
Hadoop HDFS and MapReduce
Open-Source Tools
Big Data Tools and Types of Analysis
Takeaways
5 Big Data Privacy, Ethics, and Security
Big Data Privacy
Big Data Ethics
Big Data Security
Takeaways
6 Big Data in Your Organization
Key Characteristics of Information-Centric Organizations
Some Generic Big Data Uses
Big Data and Return on Investment (ROI)
Big Data on the Balance Sheet
Internal Big Data Opportunities
Big Data Roadmap
Big Data Competencies Required
Small and Medium Enterprises (SMEs) Can Achieve Remarkable Results with Big Data
Governance for the Accuracy of Big Data
Takeaways
7 Big Data by Industry
Agriculture Industry
Automotive Industry
Consumer Goods
Education Industry
Energy Industry
Financial Services Industry
Gaming Industry
Healthcare Industry
Legal Profession
Manufacturing Industry
Not-for-Profit Sector
Media and Entertainment Industry
Oil and Gas Industry
Public Sector
Retail Industry
Telecom Industry
Transportation Industry
Travel and Leisure Industry
Takeaways
8 The Future of Big Data
Glossary
Notes
Index
About the Author
Free Sample Chapter from Wiki Management by Rod Collins
1
The History of Big Data
Of all the data in recorded human history, 90 percent has been created in the last two years. However, the need to use and interpret such Big Data has been around for much longer. In fact, the earliest examples of using data to track and control businesses date back 7,000 years, when Mesopotamians used rudimentary accounting to record the growth of crops and herds. Accounting principles continued to improve, and in 1663, John Graunt recorded and examined all information about mortality rolls in London. He wanted to gain an understanding of and build a warning system for the ongoing bubonic plague. ¹ In the first recorded example of statistical data analysis, he gathered his findings in the book Natural and Political Observations Made upon the Bills of Mortality, which provides great insights into the causes of death in the seventeenth century. Because of his work, Graunt can be considered the father of statistics.
The nineteenth century witnessed the start of the information age. Modern data was first gathered in 1887, when Herman Hollerith invented a computing machine that could read holes punched into paper cards to organize census data.²
THE TWENTIETH CENTURY
In 1937, during Franklin D. Roosevelt administration, the United States created the first major data project to keep track of contributions by more than three million employers and 26 million employees under the new Social Security Act. IBM was awarded the contract to develop a punch card-reading machine for this immense bookkeeping task.³
The British developed the first data-processing machine in 1943 to decipher Nazi codes during World War II.⁴ The device, named Colossus, searched for patterns in intercepted messages at a rate of 5,000 characters per second. This reduced the time required to perform the task from weeks to merely hours. It was a huge step forward.
In 1952 the U.S. National Security Agency (NSA) was created and, within 10 years, it had contracts with more than 12,000 cryptologists.⁵ They were confronted with information overload during the Cold War, as they started collecting and processing intelligence signals automatically.
In 1965, the U.S. Government decided to build the first data center to store its more than 742 million tax returns and 175 million sets of fingerprints.⁶ Employees transferred all those records onto magnetic computer tape that was stored in a single location. The project was later dropped out of fear of Big Brother,
but it represented the beginning of the electronic data storage era.
Then, in 1989, British computer scientist Tim Berners-Lee developed what eventually became the World Wide Web.⁷ He wanted to facilitate the sharing of information through a hypertext
system. Little could he know at that moment the impact his invention would have on everyone.
Beginning in the 1990s, data was created at an amazing rate as more and more devices were connected to the Internet. In 1995, the first supercomputer was built; it performed as much work in a second than a calculator operated by a single person could do in 30,000 years.⁸
THE TWENTY-FIRST CENTURY
In 2005, Roger Mougalas of O'Reilly Media coined the term Big Data,
a year after the company created the term Web 2.0.⁹ He used the term to refer to a large set of data that is almost impossible to manage and process using traditional business intelligence tools.
In that same year, Yahoo! created Hadoop on top of Google's MapReduce.¹⁰,¹¹ Its goal was to index the entire World Wide Web; nowadays, many organizations around the world use the open-source Hadoop to crunch massive data sets.
As more and more social network sites appeared and Web 2.0 took flight, more and more data was created daily. Innovative startups slowly mined this vast amount of data and governments also began Big Data projects. In 2009, the Indian government decided to take an iris scan, fingerprint, and photograph of all of its 1.2 billion inhabitants.¹² All this data is stored in the largest biometric database in the world.
By 2010, when Eric Schmidt, the Executive Chairman of Google, spoke at the Techonomy forum in Lake Tahoe, California, he put the information revolution in perspective by stating that every two days now we create as much information as we did from the dawn of civilization up until 2003…. That's something like five exabytes of data….
¹³
In 2011 the well-received McKinsey report on Big Data: The Next Frontier for Innovation, Competition, and Productivity,
concluded that by 2018, the United States would face a shortage of 140,000 to 190,000 data scientists, as well as 1.5 million data managers.¹⁴ The job Big Data Scientist is therefore often coined the sexiest job of the twenty-first century.
In the past few years, there has been a massive increase in the number of Big Data startup companies. All are trying to help organizations manage and understand this explosion of Big Data. As more companies are slowly adopting Big Data, just as with the Internet in 1993, the Big Data revolution is still ahead of us, so a lot will change in the coming years.¹⁵
In fact, the amount of data is growing at such an explosive rate that we have gone past the decimal system. Today, U.S. agencies, such as NSA and the FBI, are talking about yottabytes when calculating the size of their files. In the (near) future, we will be talking about brontobytes regarding sensor data. Therefore, new terms have been created to describe the amount of data that is expected to be created in coming years (see Figure 1-1).
Big Data will completely change organizations and societies around the world. It is expected that the amount of data currently available will double every two years worldwide.¹⁶ So, let's take a closer look at what Big Data exactly is.
Figure 1-1 Brontobytes Infographic
2
What Is Big Data?
As you know, everything that is digital is data. Nowadays, existing hardware and software cannot deal with the vast amounts of different types of data being created at such high speeds. Big Data has become too complex and too dynamic to be able to process, store, analyze, and manage with traditional data tools. The amount is so vast that we are not able to ask the right questions to find the right answers.
Fortunately, with the hardware, tools, and algorithms that have been developed, all that data can be turned into useful information. The insights provided by this information can be used to improve decision-making at your organization, improve your efficiency, decrease your costs, and increase revenue. The implications of the Big Data revolution are vast, and it will impact all industries and all types of companies.
EXPLANATION OF THE 7Vs
It is generally accepted that Big Data can be explained according to three Vs: velocity, variety, and volume. However, I would like to add veracity, variability, visualization, and value to better explain the impact and implications of a well-thought through Big Data strategy.
Velocity
Velocity is the speed at which data is created, stored, analyzed, and visualized. In the past, when batch processing was still common practice, it was usual to update the database every week or every evening. Computers and servers required substantial time to process the data and update the databases. In the Big Data era, however, data is created in real time or near real time. With the availability of Internet connected devices, wireless or wired, machines can pass on data the moment it is created.
The speed at which data is currently created is almost unimaginable: Every minute, we upload 100 hours of video to YouTube. In addition, over 200 million emails are sent every minute, around 20 million photos are viewed and 30,000 uploaded on Flickr, almost 300,000 tweets are sent, and almost 2.5 million queries on Google are performed.¹–⁴
The challenges organizations have is to cope with are the enormous speed at which data is created and the need to use it in real time. We will discuss real-time Big Data in Chapter 3.
Variety
In the past, all data was structured data that fit neatly into columns and rows, but those days are over. Now, 90 percent of data is unstructured.⁵ Data now comes in many different formats, including structured, semistructured, unstructured, and even complex structured.
Each type of data requires a different type of analysis and different tools to interpret. Social media, such as Facebook posts or Tweets, can provide insights in how your customers think of your brand, service, or product, while sensor data provides information on how a product or machine is used and provides insights about how you can improve the product.
Chapter 4 discusses the various types of analyses that can be performed on the different types of data and what tools can be used.
STRUCTURED DATA
Flat files in record format
A flat file can be a plain text file, usually containing one record per line. Single fields can be separated by commas. A .CSV files is a flat-file document.
GEO location data
Data about the real-world geographic location of an object.
HTML5 microdata
Microdata can include semantics in existing content to provide a richer browsing experience. The data can be extracted by search engines, crawlers, and browsers.
Legacy data
Information stored in old or obsolete formats or computer systems that is difficult to access or process.
Log files
Log files lists actions that have occurred in a standard format. Log files can give users a good indication of what is happening.
Microformats
A microformat reuses HTML/XHTML tags to convey metadata and other attributes.
Sensor data
Data collected by sensors that monitor equipment or products. Sensors can include RFID tags that store and send information.
Spreadsheets
A spreadsheet contains data in tabular format, meaning with tables and headings.
SEMISTRUCTURED DATA
Documents containing metadata tags
Metadata can include information such as author and time of creation; it can be stored easily in a relational database management system (RDBMS).
EDI documents
Electronic Data Interchange (EDI) documents imply a sequence of messages between two parties in a predefined format.
RSS feeds
Rich Site Summary (RSS) feeds are full or summarized text, including metadata, used to publish frequently updated work (e.g., blogs). RSS feeds are standardized xml file formats that allow the information to be published once and viewed by many different programs.
XML objects
Extended markup language (xml) objects are defined by a set of rules that allows formats to be readable by humans as well as machines.
UNSTRUCTURED DATA
Binary Large Objects (BLOBs)
BLOBS consists of binary data stored as a single file in a database management system. These BLOBs can be images, audio materials, or any other multimedia objects.
Business records requiring control
Documents, paper, or electronic files that are subject to business controls over storage, retention, disposition, and deletion to comply with legal, regulatory, or industry requirements or to support litigation and discovery.
Content management data
Data related to content management systems of online platforms. Wordpress or Joombla are content management systems.
Digital assets
Digital assets are documents that require specialized storage and transport to guarantee performance because they consist of large and/or special encoded files.
Dynamic content (multiple users)
Content that may be created, edited, reviewed, and approved by multiple people or groups. A good example is Google documents, which allows multiple people to work on the same document at the same time.
Email, text messages, chat
Data that represents correspondence or other communications between individuals, between companies and individuals, or between machines and individuals.
Intellectual property data
Data associated with intellectual property rights. Access to this data must be controlled. Property rights must be managed to avoid violation of contract or licensing terms regarding the use of the material.
Social data
Any data shared on social networks, such as Facebook posts, Tweets, YouTube movies, LinkedIn profiles, and so on.
Specialized content (web data)
Specialized content that requires special access, control, content entry, rendering, and other functions. A good example is online platforms.
Static documents
Documents in Word, PowerPoint, or ExCel that can be edited by one person at a time.
Taxonomies/ontologies
Ontology presents knowledge as different concepts (things, relations, or events), and it shows the relationship among these different concepts.
Voice recognition data
The translation of spoken words into text or data that can be understood by a computer. Voice recognition data includes Siri, podcast searches, speed-to-text processing, and voice-controlled systems.
COMPLEX STRUCTURED DATA
Hierarchically structured data (xml-based MISMO)
XML datasets related to complex financial products.
Volume
At the rate data is now being created, the amount will double every two years.⁶,⁷ In 2011 we created a staggering 1.8 zettabytes of data. By 2020, the world will generate 50 times that amount of data, according to a 2011 study by IDC.⁸ The sheer volume is enormous, and a very large contributor to the ever-expanding digital universe is the Internet of Things,
with sensors all over the world in all kinds of devices creating data every second.
Let's look at some examples. Airplanes generate approximately 2.5 billion terabytes of data each year from the sensors installed in their engines.⁹ The agricultural industry generates and collects massive amounts of data from sensors installed in tractors. John Deere equipment uses sensor data to monitor machine optimization, control the growing fleet of farming machines, and help farmers make better decisions. Shell uses supersensitive sensors to find additional oil in wells and, if the company installed these sensors at all 10,000, approximately 10 exabytes of data would be collected annually.¹⁰ And, that is absolutely nothing if we compare it to the Square Kilometer Array Telescope that will generate 1 exabyte of data per day.¹¹
In the past, the creation of so much data would have caused serious problems. Nowadays, with decreasing storage costs, better storage solutions, such as Hadoop, and the algorithms to create meaning from all this data, storage is not such a big problem any more.
Veracity
The generation and collection of a lot of data at high speed is worthless if that data is flawed. Incorrect data can cause significant problems for organizations, as well as for consumers. If you want your organization to become information-centric, you need to ensure that your data and its analysis are correct. This is especially true in automated decision making, where no human is involved anymore. Shockingly, one in three business leaders do not trust the information they use in decision making.¹² Therefore, if you want to develop a Big Data strategy, you should strongly focus on the correctness of the data as well as the accuracy of its analysis. I will dive deeper into this important topic in Chapter 6.
Variability
Big Data is extremely variable. Brian Hopkins, a Forrester principal analyst, defines variability as the variance in meaning
¹³ He refers to the supercomputer Watson that won on the quiz show Jeopardy. The supercomputer had to dissect an answer into its meaning and…figure out what the right question was.
That is extremely difficult because words have different meanings depending on context. For the correct answer, Watson had to understand the context.
Variability is often confused with variety. If a bakery sells ten different types of bread, that is variety. If, however, the same type of bread tastes and smells different every day, that is variability.
Variability is thus very relevant in performing sentiment analyses. Variability means that the definition is changing (rapidly).¹⁴ In similar tweets, a word can have a totally different meaning. To perform a proper sentiment analysis, algorithms need to decipher the exact meaning of a word in its context. This is still very difficult.
Visualization
This is the hard part of Big Data. It means making that vast amount of data comprehensible in a manner that is easy to read and understand. With the right visualizations, raw data can be put to use. Visualizations, of course, are not ordinary graphs or pie charts. They are complex graphs that can include many variables of data while remaining readable and understandable.
Visualizing might not be the most technologically difficult task, but it certainly is the most challenging. The use of a graph to tell a complex story is very difficult, but also extremely crucial. Luckily, there are more and more Big Data startups that focus on this aspect of the challenge. In the end, visualizations will make the difference.
Value
All this available data will create a lot of value for organizations, societies, and consumers. Big Data means big business, and every industry will reap its benefits. McKinsey states that the potential annual value of Big Data to the U.S. healthcare industry is $300 billion, which is more than double the total annual healthcare spending in Spain.¹⁵ The report also mentions that Big Data has a potential annual value of €250 billion to Europe's public sector administration. Further, McKinsey's highly respected report of 2011 states that the potential annual consumer surplus from using personal location data globally could reach $600 billion in 2020. That is a lot of value.
Of course, data in itself is not valuable at all. The value is in the analyses done on that data and how the data is turned into information and, eventually, knowledge and wisdom. The value is in how organizations use that data to create information-centric companies that base their decision making on insights derived from data analyses.
EIGHT REALITIES OF BIG DATA YOU SHOULD ALREADY KNOW
Now that we have defined Big Data, you need to know what the most important aspects of Big Data are that your organization should be aware of when developing a Big Data strategy. Big Data does require a paradigm shift, and knowing these realities will help your organization move forward. As they are so important, they will be discussed briefly here, but they are presented in greater detail in later chapters.
1. Big Data requires a different culture.
To truly take advantage of Big Data, your organization needs to become an information-centric company. This culture shift will result in more data-driven decisions and will give your employees the opportunity to develop new operational, tactical, and strategic plans based on real data instead of calculated estimates. A Big Data culture requires that employees are encouraged to collect data at every point of customer contact. They need to ask the right questions and answer them with accurate data. Of course, changing a company culture is difficult and not easily achieved, but this book will provide guidance with a roadmap that will help in creating an information-centric culture.
2. The real driver behind Big Data is the people within the organization.
Although a cultural shift is important to take full-advantage of all the possibilities of Big Data, development of a Big Data strategy comes from the people in an organization. In particular, managers and executives should be aware of what Big Data is and how it can be applied to their organization. The more decision makers are aware of the benefits of Big Data for the organization, the more likely it is that a successful Big Data strategy will be developed and implemented.
One of the most important facts to remember is that IT should not be responsible for a Big Data strategy. IT is merely a means to achieve your Big Data strategy, but it should not be responsible for overall strategic planning. There is a useful comparison to make here with the first years of social media. A few years ago, everyone thought that social media were the Holy Grail for marketing. Today, we see it, correctly, just as a means to achieve the marketing and strategy goals developed by the company. That's how Big Data should also be seen, as an important part of the overall strategy that is dealt with by the board or executive management.
To start to develop Big Data successfully, begin by finding the right sponsor within the organization, especially as the returns at first are uncertain and the costs can be high. The best choice is a senior manager or board member, as those individuals have the power to support the project even if the initial results prove to be negative.
3. Big Data is everywhere, even at places you do not expect.
Everything digital is data. More and more items are digitized and connected to the Internet. This means your organization is receiving new data from completely new areas. The Internet of Things movement shows that any product or device can be connected to the Internet and therefore provide data.¹⁶ Organizations should use this information and not be afraid to digitize products. Big Data literally is up for grabs; you only need to open your eyes to understand where you can find it and how to analyze and use it.
The Internet of Things even makes it possible to turn a cup of coffee into data! Just add a few sensors to the cup of coffee, and you can analyze when someone drank the coffee, how fast, where, for how long, with how many sips, what the temperature of the coffee was, how much coffee and water was used, and so on. If you collect and properly analyze the data about how coffee is consumed, you can turn it into information that can be used to understand the coffee drinking habits of your employees. This is of course a bit of a joke, as I doubt that any senior manager would be interested in such information, but it shows that to really incorporate Big Data into your organization, you should look out of the box.
But, you should also start looking in the public data marketplaces that are starting to appear everywhere. These data marketplaces collect free and paid public and open datasets from around the world. If your own data is combined with these new existing datasets, you will gain completely new insights and information.
4. Big Data engineers will be scarce, so you better start looking around.
McKinsey predicts a shortage of about 140,000 to 190,000 Big Data engineers in the United States alone in 2018.¹⁷ The report also predicts a shortage of 1,500,000 Big Data managers who can supervise the Big Data engineers and can connect the IT aspect of Big Data with the strategy aspect. So, Big Data employees will be scarce in the future.
Big Data engineers and managers, however, are not the only personnel you should employ. Big Data analysts, Big Data solution architects, and, of course, the illusive Big Data Scientist should also be on your list. In particular, the Big Data Scientist will be hard to find and, therefore, expensive to employ. It is often called the sexiest job of the twenty-first century and, currently, only a handful of people around the world can truly call themselves Big Data Scientists.
For organizations to avoid missing out on future developments, they should start training their IT personnel in Big Data technologies, particularly if they want to develop Big Data solutions in-house. Universities are already creating Big Data engineering courses to prepare students for the future. In fact, more and more universities are already offering a Big Data study program, as well as courses on open online platforms, such as Coursera.¹⁸,¹⁹
5. Big Data does require big security measures.
Organizations that gather large valuable datasets need to protect that information from criminals whose goal is to steal the data and use it to their advantage. In recent years, a number of large, online companies and government organizations have been hacked, including LinkedIn, Evernote, and even Bitcoin.²⁰–²² Therefore, it is extremely important to protect all data that is collected. There are several ways to secure your data, the most common of which is to correctly encrypt the information. But, of course, there are many other ways to protect data, so security should always be part of your Big Data team.²³
Every