Meeting the Challenge of Big Data

Table of Contents

  • 1
    Table of Contents
    Chapter 1 - The Rise of Analytics 3
    Chapter 2 - Dealing with the Data Deluge 6
    Chapter 3 - Discovering Business Value 8
    Chapter 4 - Manufacturing 11
    Chapter 5 - Healthcare 14
    Chapter 6 - Retail and Consumer Goods 17
    Chapter 7 - The Big Data Platform 20
    Resources 23
  • Big data is the electricity of the 21st century – with the power to transform everything it touches in business, government and private life. However, it's not raw data that powers change; it's the insight derived from the data that leads to better outcomes. The key is analytics.

    Companies are already organizing data to model business processes and run the business. Big data is also about taking new data as-is to learn what it can do: using this data to help change the business.

    Companies need analytic solutions to cut the cost of forming and testing hypotheses on the torrent of new data coming at them, as well as solutions to cut the cost of standardizing and controlling processes that apply it. These two approaches are more powerful together than either alone. Combining them is what we call Big Data At Work.

    Analyst Report: Big Data: The Next Frontier for Innovation, Competition, and Productivity.
    Read the McKinsey Report

    Today's analytic tools can be used to reveal insights from historical data as well as from real-time information streams. These tools help you analyze past events, understand current activities, and predict future outcomes. With this wealth of business insight comes a rare opportunity to outpace competitors and exceed stakeholder expectations. With big data, organizations can know their customers better, introduce new products and services, and manage risk and capital better.

    Article: Taming the Data Deluge.
    Read the Profit Magazine Article

    In Meeting the Challenge of Big Data: Part One we gave an overview of how to acquire, organize, and analyze big data. In this e-book, we focus on analytics, and look in more detail at how data discovery, predictive analytics, and automated decision-making can deliver business value from big data.

  • In this e-book, you'll read profiles of innovators in a variety of industries and market situations, and see the power of big data at work.

    Video: Holland America Enhances Customer Experience
    Watch the Video (2:06)
    "Now we have actual data about customer purchase behavior."
    Therron Hofsetz, Director of IT Applications, Holland America
    Video: New in Big Data
    Join MIT Principal Research Scientist Andrew McAfee and Oracle Big Data Strategist Paul Sonderegger as they discuss how big data is making its mark on business—one data-driven decision at a time.
    Watch the Video (25:10)

    With data volume doubling every two years, many organizations need to manage and analyze petabytes of structured and unstructured data. Unlike traditional corporate data that can be modeled and stored in relational tables, today's data comes from many different sources and appears in many different forms such as web pages, social media sites, e-mail exchanges, search indexes, click streams, equipment sensors, and all types of multimedia files including audio, video, and photographic.

    Just about every organization can use analytics to reveal hidden patterns, gain strategic insight, and generate value from this huge volume of data. In many cases, the data is already being captured, but it is not fully leveraged. You need powerful analytics to derive meaningful insights from it that can transform your business.

    Questioning your analytic readiness? You're not alone.
    Read More

    Is it worth the trouble? Nucleus Research thinks so. According to its research, organizations earn an average of US$10.66 for every dollar spent on deployments of analytics applications.1 If you think you are not up to the challenge, you are not alone: 38 percent of executives say they don't have the right systems in place to gather the information they need.

  • According to global survey of 226 senior executives conducted by Oracle, although 67 percent of executives say that the ability to draw intelligence from their data is a top priority for their organization, 29 percent give their organization a D or F in preparedness for a data deluge.2

    At face value, the task appears overwhelming. As Andrew McAfee and Erik Brynjolfsson of the Harvard Business Review point out, more data now crosses the internet every second than was stored in the entire internet 20 years ago. A recent survey titled "From Overload to Impact: An Industry Scorecard on Big Data Business Challenges" revealed that 93 percent of executives believe their organization is losing revenue as a result of not being able to fully leverage the information they collect. This lost opportunity is valued at 13 percent of annual revenue for companies that average more than US$1 billion in revenue.

    Video: Key Technologies for Big Data
    "Big data is only interesting if it helps your enterprise solve a business problem."
    Pauline Nist, GM of Enterprise Partner Strategy, Intel®
    Watch the Video (2:30)

    There are two broad categories of big data: human-generated and machine-generated. Human-generated data includes documents, transactions, e-mail messages, social media content, and many more—a steady stream of data from nearly every activity that we perform on our computers, our mobile devices, and in our data centers. Market leaders leverage this data to monitor customer sentiment, detect rogue trading activity, fulfill compliance mandates, develop targeted marketing campaigns, and support many other business endeavors.

    Video: Dawn of a Revolution The revolution in business analytics has begun. Witness how big data is transforming cultural and commercial activities around the world.
    Watch the Video (2:16)

    "Data has become a torrent flowing into every area of the global economy. Companies churn out a burgeoning volume of transactional data, capturing trillions of bytes of information about their customers, suppliers, and operations."1

    1 McKinsey Global Institute, May 2011.

  • Machine-generated data comes from sensors embedded in machinery, electronics equipment, power grids, automobiles, and millions of industrial and consumer products. Many physical activities can be measured and monitored in ways that we could scarcely have imagined a few years ago. For example, sensors embedded in medical devices enable healthcare professionals to remotely monitor patients. And sensors on pipelines allow energy companies to monitor flow rates and foresee equipment problems.

    Article: Billions Of Reasons To Get Ready For Big Data.
    Read the Forbes Magazine Article

    The sensors themselves aren't new, nor is the ability to remotely monitor precision equipment. What's changed is the way companies are using the data generated by these tiny embedded devices. The ability to monitor in real time is transforming the way analysts use the information at hand. They can aggregate the data to look for patterns, build models, and make predictions, whether it's identifying patients with a high risk of heart attack, or determining when a petroleum pumping station might break down. Knowledge obtained from big data allows users to predict potential problems and take corrective action.


    Market leaders are building big data applications to help with everything from fine-tuning supply chains to gauging customer sentiment. Detecting and combating fraud is a particularly fruitful domain for big data analytics. Fraud-detection tools can analyze not only the financial details of discrete transactions, but they can also factor in IP addresses, browser information, and other technical data to help companies predict a fraudulent transaction, identify it in real time, and prevent it from being completed.

    Turkcell Case Study: Analyzing Transactions to Detect Fraud.
    Read the Case Study

    Oracle Advanced Analytics is ideal for uncovering hidden relationships in big data sources. As Turkcell has demonstrated, organizations can build and apply predictive models to prevent fraud and solve many other analytic challenges.

    Using Advanced Analytics to Detect Fraud
    Read More

    In the remainder of this e-book, we focus on the effect big data is having on three important industries: manufacturing, healthcare, and retail.

  • The modern factory floor is a marvel of automation. Advanced robotic machinery works with tremendous precision to assemble products with little or no human intervention. Today's assembly lines generate information from networked sensors that can be used for equipment configuration, troubleshooting, quality control, and maintenance.

    Fine-Tuning Manufacturing Strategies
    Read More

    Sensors and data-capture devices have proliferated in consumer and industrial products as well. For example, modern automobiles continuously measure the workings of critical onboard components, from tires to brakes to fuel-injection systems. This data is stored in an onboard computer, captured when the car is serviced, and analyzed to provide insight into engineering, manufacturing, and maintenance issues.

    Studying these data streams allows manufacturers to improve their products and devise more accurate service cycles. Vehicle recalls are extremely expensive and can have an adverse impact on customer perceptions of the brand. Being able to detect defects and respond quickly helps manufacturers to minimize these expansive scenarios.


    Warranty analysis is another example of how collecting and analyzing data on a car-by-car basis has a ripple effect that influences many other parts of the industry. There are three critical metrics to warranty performance: cost per vehicle serviced, number of repairs per vehicle serviced, and cost per repair. Each warranty claim includes dozens of additional data points covering all the parts, components, diagnostic codes, and labor for each job, along with textual descriptions from field reports. Oracle Endeca Information Discovery software makes it easy to discern insights from reams of messy, non-conformed data.

    Video: Finding the Root Cause of Warranty Claims
    Watch the Video (4:22)

    Data discovery tools help you get answers fast by analyzing information from interconnected companies, supply chains, and databases. They enable you to go beyond traditional reports and analytics to analyze both structured and unstructured information, and then correlate your discoveries with KPIs and business metrics.

  • Although the immediate benefit of these trends is superior medical services for each patient, healthcare providers and health plans are also discovering how to mine this information at an aggregate level to improve diagnostic capabilities, disease management programs, and standardized treatment regimens, and ultimately make better public health predictions. For example, the US Centers for Disease Control can predict the incidence and severity of flu outbreaks by analyzing hospital admissions records, weather forecasts, pharmaceutical sales, and relevant search engine terms such as "flu symptoms."

    "Personalized medicine is about developing individualized treatments for every single patient."
    Lisa Khorey, Vice President, Enterprise Systems and Data Management, UPMC

    There are also exciting developments in the field of predictive diagnostics. As the cost of DNA sequencing becoming more affordable, many practitioners are combining clinical and genomics research data to devise personalized treatments. This not only leads to a growing body of genomics data, but to an immense base of research that can be mined for insight. For example, the National Cancer Institutes used Oracle Big Data Appliance to measure how frequently 17,000 genes associated with cancer were mentioned in more than 20 million research papers.


    UPMC is at the forefront of the rising wave of personalized healthcare with its use of computational science to analyze a growing base of information. The goal is to gain a more detailed understanding of the impact of genetics in treating diseases as researchers and clinicians discover clues that lead to better medical decisions.

    Video: Personalized Medicine at UPMC
    Watch the Video (3:16)

    UPMC expects to solve this big data challenge by funneling clinical data, genomic data, administrative data, and financial data into a unified system anchored by Oracle Enterprise Healthcare Analytics and other Oracle software, improving the quality of patient care while reducing costs.

    Article: UPMC Unlocks Secrets of Human Health.
    Read the Forbes article
  • Retailers collect a steady stream of data about customer activities, preferences, and purchasing habits. They can target consumers with special offers and follow their browsing habits when they shop online. By analyzing web logs, traffic patterns, and point-of-sale systems, retailers can learn what works and what doesn't.

    Sold on Big Data
    Read More

    A large consumer products company found that traditional data sources were often insufficient to explain variations in sales and performance, and it wanted to better understand its consumers to more effectively market to them. Company leaders suspected that nontraditional data sources such as social media, customer service call reports, and customer satisfaction surveys would help explain the discrepancies, but they didn't have the technologies to combine data from these sources with pricing and promotions data.


    By implementing Oracle Endeca Information Discovery solutions, the company was able to combine information from its own marketing database with price point and sale information from grocers, as well as with data from AC Nielsen about TV viewing figures from ad spend, promotions, and markdowns. It also looked at comments from its customer service call line and from social media.

    Video: Answering the Million Dollar Question with Oracle Endeca
    Watch the Video (5:19)

    While exploring the data, a spike in sales was discovered. Nothing in price and promotion data explained it. Nothing in customer service call data explained it. But social media data uncovered a spike in mentions of the product that correlated with the spike in sales. Turns out that a celebrity voice over artist for one of their brand commercials had made headlines during that time period, causing the commercial to go viral. The company was able to discover a signal—a root cause of an otherwise unexplainable spike in sales—out of noisy data.

  • Some of the world's largest and most successful retailers have mastered the art of personalization to create unique shopping experiences for their customers. Personalization leads to an increase in sales for the retailer and a more productive and enjoyable experience for the shopper. Dell inc. has woven these concepts into every aspect of its marketing strategy. By deploying Oracle Real Time Decisions, the retail giant was able to carefully track the many ways that customers interact with the company, from web clicks to support calls, social media, and e-mail. Dell analyzes this "fast data" to surmise what each customer needs each moment.

    Video: Dell Leads with Mass Personalization
    Watch the Video (2:21)

    The stats tell the story: click rates for personalized e-mails improved 30 percent, revenue-per-click is up 40 percent, and close rates have improved 19 percent since Dell started conducting these analyses.


    Big data promises tremendous advances in retail, financial services, healthcare, communications, manufacturing, and other industries. To take advantage of these advances, organizations need analytics solutions that can process and store data continuously and much faster than traditional transactional data systems. As industry analyst Richard Winter asks, "How can you accomplish the needed intensive, frequent, and sometimes immediate analysis—analysis that is difficult or impossible to express in SQL? And how do you deal with the economics of such large volumes of data, particularly when you may not know in advance whether it contains information of value?"1

    Oracle and Intel® have the answers. Two decades of engineering collaboration between these two industry leaders has resulted in Intel® Xeon® processor–based solutions that deliver unprecedented performance, unparalleled reliability, and intelligent scalability for the most demanding enterprise environments.

    Video: Engineered Systems for Big Data
    Watch the Video (3:07)

    1 Winter, op. cit.

  • Intel and Oracle offer solutions that help you get new insights out of your data sooner.

    Big data can provide new business insight, which can lead to smarter, data-driven decisions. But this insight is only realized when you can perform integrated analysis of all the data. You need a platform that can work with data from many different sources, and then run powerful analytics to uncover the hidden relationships that not only provide new insights, but also give you the tools to forecast future results and map out alternative scenarios to optimize your business outcomes.

    Because today's data comes from many different sources and in many different formats, new software is needed to acquire and organize it. Your analytic platform must support new storage and analytics technologies like Hadoop and NoSQL as well as traditional enterprise data sources and business intelligence tools.

    Your existing enterprise architecture is already complex, and big data adds new requirements that can make it even more so. Engineered systems are proven to tame that complexity, reduce your risk, and accelerate productive deployments.

    Powered by Intel Xeon processors, and incorporating system optimizations by Oracle and Intel, Oracle's big data platform includes three key engineered systems. These systems can work together or separately, and they can integrate with the rest of your enterprise, whether it's from Oracle or other vendors.


    Oracle Big Data Appliance, which is optimized to run Hadoop efficiently, is an ideal platform for acquiring new data, organizing it, and conducting initial analyses to glean key summaries, trends, and associations from the raw data set before you integrate it with your existing data set.

    Oracle Exadata Database Machine runs your Oracle Database as the focal point of your data warehouse and online transaction processing systems. Its high-end data management capabilities make it the best solution for integrating new data with existing enterprise data. Oracle Exadata also runs powerful in-database analytics to uncover hidden relationships that can lead to new insight.

    Oracle Exalytics In-Memory Machine is the world's first engineered system specifically designed to deliver high-performance discovery, analysis, modeling, and planning. Built using industry-standard hardware, market-leading business intelligence software, and in-memory database technology, Oracle Exalytics delivers answers to all your business questions with unmatched speed, intelligence, simplicity, and manageability.

    Oracle Services are designed to work together seamlessly with Oracle hardware and software products to enable the success of your Oracle investments - throughout your IT lifecycle - and across the complete Oracle stack. From strategy and architecture, to managing upgrades, migrations and implementations, to ongoing support and education, Oracle Services can help you take advantage of best practices and technology to get the most from your Oracle Big Data Solution.

  • Resources
    White Papers
    Data Sheets
    Social Media