Senior management to be responsible for processes based on big data and AI, German regulator says

Maria Nikolova

It is the responsibility of supervised firms to guarantee the explainability of BDAI-based decisions, BaFin says.

There will be no black box excuses for the management of financial services companies in Germany, in case they want to shift the responsibility for a decision involving big data and artificial intelligence (BDAI) to machines rather than human beings, the Federal Financial Supervisory Authority of Germany stated in a report published earlier today.

The report challenges the traditional approach to algorithms as black boxes. BaFin insists that senior management remains responsible for BDAI-based processes and decisions.

BaFin agrees that BDAI will create additional opportunities for automating standard market processes – from chatbots to liquidity management. Examples include the applications of automation for the lending business at banks or for claims management at insurance companies. Further automation is aimed at increasing efficiency and effectiveness – and thus cutting costs.

However, the regulator notes that this does not mean that responsibility for the results of BDAI-supported processes should be shifted to machines. When designing (partially) automated processes, it is therefore important to ensure that they are embedded in an effective, appropriate and proper business organisation.

Ultimately, responsibility for automated processes has to remain with the senior management of the supervised firm, says BaFin.

According to the regulator, supervised firms are the ones that have to guarantee the explainability/traceability of BDAI-based decisions. Supervisory and regulatory authorities will not accept any models presented as an unexplainable black box.

In the case of black boxes a user has no direct possibility of finding out why or how the algorithm has made its decision and thus produced the result. This is mainly due to the fact that the input and output values of neural networks are linked in a very non-linear and complex way.

According to the regulator, a better understanding of models provides an opportunity to improve the analysis process – for example, the users are then in a position to recognise problems such as overfitting and data bias.

As far as the intelligibility of AI systems is concerned, a distinction is made between transparency and explainability. Transparency describes the behaviour of the system being completely comprehensible. But this requirement can almost never be fulfilled since many models are necessarily complex. Put otherwise, a black box can hardly be made transparent.

In contrast, explainability describes being able to list the major factors of influence for a concrete individual decision. This is much easier to fulfill in terms of technology.

There are two main approaches for the creation of “transparent” systems:

  • An attempt can be made to generate comprehensible approximations for any models. This approximation should be characterised by being able to reproduce the behaviour of the original model with as little deviation as possible, despite being able to be depicted in a way that humans can understand. The TREPAN algorithm is an example of such an approach.
  • There are also approaches for the generation of directly comprehensible models from data, such as decision trees or subgroups. Such models are not always ideal for the learning task, yet if transparency is more important than the quality of the result, these methods are more appropriate.

The generation of explanations is currently a fast-developing field of research in machine learning. One of the most famous approaches is the LIME algorithm, which uses simpler methods to form a local explanation model for the individual case to be explained and similar data points. Other approaches are prototype-based and deliver representative individual cases to explain decisions.

The German regulator stresses that new approaches could provide at least some insight into how the models work and the reasons for decisions, even with very complex models. This is set to prevent models from being seen and treated purely as black boxes.

A consultation has been open into the detailed report which raises many important questions around data mining, privacy, and innovation. Feedback is expected not later than September 30, 2018.

Read this next

Retail FX

XPRTcoin Review 2022 – Is XPRTcoin Scam or Legit?

Read our XPRTcoin review today to learn all you need to know about XPRTcoin before signing up with the broker ( – by FinanceFeeds.

Retail FX

Financial services domains blocked by Consob rise to 708

In yet another episode in a long-running campaign, the Italian financial regulator, Consob, once again blocked access to a number of investment websites, including those holding licenses in other jurisdictions.

Digital Assets

Binance upgrades Bahrain license to offer full-suite of crypto services

The Central Bank of Bahrain (CBB) has granted Binance its Category 4 license as a fully-fledged crypto-asset service provider (CASP).

Digital Assets

BitMEX spot exchange hits $24 million in daily volume

The newly-launched spot market of crypto exchange BitMEX reported a record of $24 million in 24-hour trade activity on May 25.

Retail FX

Financial Commission certifies offering of trade copier 4X Solutions

The Financial Commission, an independent self-regulatory compliance specialist for the financial services industry, has certified the trading technology offered by trade copier 4X Solutions.

Industry News, Inside View

LIVE from Devexperts webinar for brokers on Fractional Trading

Finance Feeds is providing live coverage of the event that aims to help brokers discover fractional shares as a key tool for a successful brokerage business in today’s trading industry.

Industry News, Retail FX

ASIC celebrates retail ban on binary options as 68% of wholesale clients lose money

In the 13 months before the ban, between 74% and 77% of active retail clients lost money trading binary options. The product intervention order does not apply to wholesale clients.

Crypto Insider

Investing in crypto: how to stay away from weak players

The main reason behind the hacks of crypto exchanges is weak key management. For example, all 4 hacks that took place in 2021 were caused by the ability of hackers to obtain access to hot wallets. 

Retail FX

Vantage launches swap-free gold XAUUSD trades for all clients until end of July

“As the greenback continues to strengthen, we want to support traders who remain bullish on gold or seek short term trading opportunities amid the volatility”