Applying big data to risk management: “Big Data represents the future of Risk Management” A comprehensive insight – Part 1

Part 1 of our Applying Big Data to Risk Management series.

This is a guest research article by Kieron Yorke, Director of Financial Sales Services at SinusIridum

As you are reading, the world’s data is exploding in unprecedented velocity, variety, and volume. It is now available almost instantaneously, creating possibilities for near real-time analysis. While Big Data is already being embraced in many fields, risk managers have yet to harness its power.

Big Data technology has revolutionary potential. It can improve the predictive power of risk models, exponentially improve system response times and effectiveness, provide more extensive risk coverage, and generate significant cost savings. In a world of increasing complexity and demand, the ability to capture, access and utilise Big Data will determine risk management success.

Over 90% of the world’s data has been created in the last two years. Forward-thinking industries and organisations have already begun to capitalise on this gold mine. But what does the Big Data revolution mean for risk management?

Big Data represents the future of Risk Management

Put simply, Big Data represents the future in this field. Why? Big Data technologies can help Risk teams gain more accurate risk intelligence, drawn from a variety of data sources, in almost real- time. Within the financial services industry, they can allow asset managers, banks and insurance companies to proactively detect potential risks, react faster and more effectively, and make robust decisions informed by thousands of risk variables. Already used widely across other sectors – particularly in eCommerce – Big Data is a true game changer.

Big Data is routinely defined as high-volume, high-velocity and high-variety information assets demanding new technological approaches to organisation and analysis. When applied to risk management within the financial services industry, we would add ‘high-veracity’ and ‘high-value’ to this list – effective analysis of this data has the potential to drive increased accuracy and reliability, and offers potentially significant cost savings by combatting the risks that can cost financial institutions billions. Big Data technologies are transforming the world of risk management.

The era of data warehouses – with their static structures and limited interaction paths – is over. The complexity and variety of sources now available (including social media, email, sensor data, business apps, archives and documents), and the speed required for retrieval and analysis, demand fresh new approaches. We are now moving into the era of data lakes.

The approach to data lakes is simple

The approach to data lakes is simple: instead of organising data in a siloed, prescriptive store, you can retain all types of data together in their original formats. Data lakes store both structured data – as contained in relational databases and spreadsheets – and unstructured data – such as social media, email and text documents. It can then be used far more rapidly and flexibly. This system allows users to run ad hoc queries, perform cross-source navigation, and make analytical decisions, all based on real-time information.

For example: imagine you want to conduct a credit check on a new customer. Data lakes allow a risk profile to be developed based on a range of data – including customer credit reports, spending habits, social media profiles, and credit card repayment rates – in seconds.

Worried about fraud on the trading floor? Rather than manually – and laboriously – track staff trading actions, data lakes allow the retrieval of an instant snapshot of activity, including information from chat room sites, mobile phones, and even door swipe cards. Suspicious activity can be identified and stopped as it is happening, before incurring fines and devastating damage to your bank’s reputation.

Big Data suggests big promises – but can it live up to them?

It is still very early days. I must point out, that with the amount of data being generated daily, the ‘noise (may well be) increasing faster than the signal. There are so many hypotheses to test, so many data sets to mine – but a relatively constant amount of objective truth.

Determining useful signal requires targeted strategies and appropriate technologies, or the sheer volume of data threatens to obscure insight and value. My own experience has provided me with a number of insights as to how Big Data might benefit the risk management sector, and how we can get closer to the ‘objective truth’ beneath the noise.