By Hortonworks:

A storm is coming in the insurance industry, and it will rain information. To remain competitive, insurance companies must be ready to mine information for insights in innovative ways. They will rely increasingly on big data in insurance to reduce claims, create value for their customers, and help proactively monitor risks to minimize customer losses.

In the past, an insurance company might have used historical data to predict the cost of future property damage claims. Now, real-time weather and sensor data can alert the company to an impending storm, and within 72 hours it can marry that information with geographical data to determine the probability of snow, ice, or flooding in low-lying areas. In the locations determined to be risk areas, the company can identify customers who live in low-lying elevations or blizzard-prone areas, and use digital channels to warn them about property protection. The public relations department can proactively conduct interviews with local news stations to further increase public awareness. Thanks to this activity, the insurance company might avert a large proportion of insurance claimsfrom customers in those locales. It would save hundreds of thousands of dollars, make customers safer, and increase overall customer loyalty. That’s big data at work.

So how do insurance companies get beyond the old way of thinking and embrace this new mindset? It all comes down to connectivity and data processing. Insurance companies in the property and casualty field face a major opportunity as technology develops: they have more and more information on which to base their risk assessments. Internet-connected devices make it possible to gather more data than ever before about the state of property and its environment.


While insurance companies and brokers once relied on historical data for actuarial calculations, they can now draw on data sources that update by the second. This enables them to be more responsive in an increasingly volatile risk environment. Climate data used to be relatively stable, making it less important to monitor on an ongoing basis. Today, climate change is a clear and present risk for insurance companies, and changes must be monitored to predict future trends. Using big data, insurance companies can enhance weather information with data from countless environmental sensors to understand wind speed, barometric pressure, temperature, and changes in the jet stream.

On a different scale, the connected car is changing the way companies view automotive insurance. Vehicles today send thousands of data points to servers every second, detailing everything from their location to braking and their speed. In the future, vehicle-to-infrastructure solutions will communicate high-resolution road conditions—down to the pothole and puddle level—on a real-time basis. Insurance companies can use this data to make real-time decisions that manage risk, perhaps even using telematics and automated-driving technology to advise drivers on the least dangerous route to take.

The use of big data in insurance is already transforming the industry. For example, Ford has partnered with IVOX, developer of the DriverScore app. This uses privacy-enhanced technology to tell insurance companies how drivers are performing in order to potentially lower their premiums. Similarly, connected homes are getting better at communicating their environmental conditions. How about using basement moisture sensors to provide an extra layer of intelligence about potential flooding?


These are all opportunities for insurance companies. The challenge lies in digesting a rising tide of data and making sense of it. To get these fast-firing insights, insurance companies must expand their analytics practices beyond traditional ad hoc reporting. In the past, they would run business intelligence reports on a batch basis—weeks or months apart—and use this data to make predictions about future trends. These practices are still valuable, but insurance companies must evolve their analytics processes to remain competitive: they must monitor information on an ongoing basis, analyzing it proactively to search for insights.

This level of maturity involves digesting new kinds of data. In the past, rigid, relational databases held the lion’s share of information. These data structures are no longer adequate in an age of real-time, unstructured information from a growing variety of sources. Insurance companies must pull these data elements together so they can analyze it all seamlessly. This means breaking down information silos. To do this, they will rely increasingly on the data lake and streaming information. This replacement for the traditional data warehouse is a vast collection of structured and unstructured data that companies can query and monitor proactively.


When mining this vast data lake, insurance companies will need to focus on speed. They can reduce query times by dividing them into parts and managing them concurrently. Systems like Apache Hadoop and MapReduce can help here. As companies struggle to cope with the increasing volume and velocity of big data in insurance, the tools they use will also rely increasingly on artificial intelligence. Machine learning algorithms and natural language processing will mine the data quickly to detect patterns that human analysts may miss or not have the time to analyze.

For any of this to happen, insurance companies will need to carefully nurture new skills. Data scientists will be in high demand, but they won’t be enough. Insurers must find ways to marry their technology and information skills with industry-specific domain expertise so they know what information to look for and how to articulate it.

The journey to this new operating model won’t be easy. It requires a careful blend of people, process, and technology to ensure that companies can pull the right insights from their data at the right time and generate actionable decisions. The sooner they get this right, the further ahead of the curve they will be—and the more traction they’ll have in a highly competitive industry that finds itself managing a greater variety of risks than ever before.