Financial Stability Board voices concerns over AI “black boxes”
Financial regulators, institutions and consumers are facing the problem of interpretability of AI decisions in areas like trading and investment, FSB warns.
The Financial Stability Board has voiced its concerns about possible risks associated with the increased adoption of artificial intelligence (AI) and machine learning in the financial services sector. In a report entitled “Artificial intelligence and machine learning in financial services. Market developments and financial stability implications”, the FSB notes the numerous advantages of such novel technologies but also highlights some risks that may stem from their use.
One of the biggest risks is that the use of AI and machine learning may create the so-called “black boxes” in decision-making that could result in complicated issues, especially during tail events. In particular, it may be difficult for humans to understand how decisions, such as those for trading and investment, have been formulated.
Given this problem with interpretability, if AI and machine learning based decisions cause losses to financial intermediaries across the financial system, there may be a lack of clarity around responsibility.
The use of complex algorithms could result in a lack of transparency to consumers when it comes to credit scores assigned by AI programs. When using machine learning to assign credit scores make credit decisions, it is generally more difficult to provide consumers, auditors, and supervisors with an explanation of a credit score and resulting credit decision if challenged. Additionally, some argue that the use of new alternative data sources, such as online behaviour or non-traditional financial information, could introduce bias into the credit decision.
With regard to the use of such novel technologies in portfolio management, where AI and machine learning tools are used to identify new signals on price movements and to make more effective use of available data and market research than with current models, one issue is that useful trading signals derived from AI and machine learning strategies may follow a decay function over time, as data are more widely used and hence become less valuable for gaining an edge over other investors.
Another risk stems from a situation where multiple market participants come to use similar AI and machine learning programs in areas such as credit scoring or financial market activities. If machine learning-based traders outperform others, this could in the future result in many more traders adopting similar machine learning strategies. As with any herding behavior in the market, this has the potential to increase financial shocks. Moreover, advanced optimisation techniques and predictable patterns in the behavior of automated trading strategies could be used by insiders or by cybercriminals to manipulate market prices.
High frequency trading (HFT) applications of AI and machine learning could be new sources of vulnerabilities. If a similar investment strategy based on AI and machine learning is widely used in HFT, it might amplify market volatility through large sales or purchases executed almost simultaneously.
Finally, network effects and scalability of new technologies may in the future give rise to third-party dependencies. This could in turn result in the emergence of new systemically important players that could fall outside the regulatory perimeter.
