evolution of its textual complexity – Bank Underground

James Brookes, Matthew Everitt and Quynh-Anh Vo

The Basel III framework put in place in the aftermath of the Global Financial Crisis 2007–08 consists of a range of regulatory standards, each addressing a specific source of financial instability. Its implementation has however led to active discussion about whether the complexity of financial regulations has materially increased. This blog post presents insights from an analysis on the evolution of textual complexity of the Basel framework.

The Basel framework is the full set of internationally agreed standards developed by the Basel Committee on Banking Supervision (BCBS). The first Basel Accord, known as Basel I, was announced in 1988 and consisted of a credit risk measurement framework with a minimum capital standard. Its revision, referred to as Basel II, contained three pillars and the treatment of market risk. Basel III was the result of the reform initiated by the BCBS in response to the financial crisis of 2007–08.

This post focuses on the difference, in terms of linguistic complexity, between Basel II and Basel III as well as between different standards of the latter. The Basel III texts analysed here include all standards that will be effective as of 1 January 2023. We also compare the network complexity of Basel II and Basel III. To do that, we rely on a recent paper which proposes to define regulatory complexity as the complexity that readers encounter when they process regulatory texts. It presents some established measures of textual complexity derived from network science, linguistics and legal studies.

How different are Basel II and III in terms of their linguistic complexity?

We begin by comparing the linguistic complexity of Basel II and Basel III. Following Amadxarif et al (2021), a regulatory text is linguistically complex if it is difficult for the user (eg banks, investors, supervisors) to understand. Linguistic complexity is multifaceted, covering many different levels of human language processing (see for example Munday and Brookes (2021)). We focus on four easily computable measures in this post:

• Length: The total number of words. Longer pieces of regulation are assumed to be more complex, because they contain more detail that needs to be digested and retained in memory.
• Lexical diversity: Language comprehension is facilitated when words are repeated. A linguistically simple piece of regulation would thus have many repetitions (the same concept discussed over and over). A linguistically complex piece of regulation would have relatively little repetition (it would cover many different concepts). We measure lexical diversity by using a measured called the type-token ratio, which is computed by dividing the count of unique words in a document by the total number of words in the document. A higher value of this measure indicates higher complexity.
• Conditionality: We measure conditionality by counting the number of conditional clauses or conditional expressions per sentence. We take the following words/phrases to indicate conditionality: if, when(ever), where(ver), unless, notwithstanding, except, but, provided (that). Conditionality contributes to complexity in two ways. First, conditionals often deal with possible and counterfactual worlds. So readers have to construct mental models of worlds that do not exist in order to be able to understand them. Second, if there are many different conditional clauses, readers have to integrate many different exceptions, which may interfere with their ability to understand the applicability of a given rule.
• Readability: To get an overall impression of the readability of a given standard, we use the familiar Flesch-Kincaid grade level readability metric. The resulting score can be interpreted as the number of years of education required to be able to understand the given standard.

We also look at two related aspects of linguistic complexity – vagueness and precision.

• Vagueness captures the extent to which the reader needs to use discretion and judgement in interpreting a given provision. We count the number of words expressing vagueness (eg appropriate, adequate, effective, fair, good, etc) in a given piece of regulation.
• Precision assesses the number of precise numerals in a given piece of regulation – specifically, amounts following indicators of currency (GBP, USD, etc) and per cents (%).

Our results are presented in Chart 1.

Chart 1: Comparison of the linguistic complexity between Basel II and Basel III

Our findings suggest that Basel III is generally more complex than Basel II. For instance, its length is more than twice that of the earlier framework. This can be attributable to the fact that Basel III deals with a much more comprehensive types of risk than Basel II. Basel III also contains more conditional expressions per sentence than Basel II. One can attribute this to the need for Basel III to be more risk sensitive. Basel III is also slightly less readable than Basel II, according to the Flesch-Kincaid grade level measure. To put this in context, a Bank Underground post indicated that broadsheet newspapers have a Flesch-Kincaid grade level score of about 11, about the same as a Thomas Hardy novel.

Which standards of Basel III are most linguistically complex?

Another interesting question is which parts of Basel III are the most complex. Using the same metrics as above, our results are shown in Table A.

Table A: Linguistic complexity of different Basel III standards

No standards stand out as most complex across all measures. Credit risk and Market risk are the longest parts of the Basel III standards, based on number of words. However, Operational Risk, Large Exposures, Leverage Ratio, and Net Stable Funding Ratio are more complex elements when looking at lexical diversity and conditionality. As expected, the qualitative aspects of the Basel III standards (supervisory review process, disclosure, and core principles for supervision) are the least specific aspects of the Basel III standards. Interestingly, margin requirements are also flagged as a particularly vague element of the standards. Finally, Flesch-Kincaid grade level readability scores indicate that all standards are roughly similar to each other and, overall, quite difficult to understand.

How does the network complexity of Basel II compare with that of Basel III?

Next, we compare the network complexity of Basel II and Basel III. Similarly, we define network complexity as per Amadxarif et al (2021). We use two fundamental building blocks for network analysis consistent with the literature: nodes and edges (links). Edges represent directed references between different parts of the framework, while nodes are Paragraphs in Basel II and Subparts in Basel III (eg CAP10.12, CAP10.16). The measures we use are:

• Size: Number of nodes in each framework.
• Volume: Number of references between rules.
• Degree: Count of incoming and outgoing connections to/from a node.
• Gini Coefficient: It measures the inequality in the distribution of degree across a framework. A higher value indicates the dominance (most connections) of single nodes, and sparse connections for the majority of other nodes.
• Reciprocity: Proportion of edges for which an edge in the opposite direction exists.

Cross-references reflect complexity that results from the structure, rather than the language, of rules. Starting from any given rule, two different networks can be generated.

• Centrality: The inward expansion identifies all nodes cross-referring to the initial rule, and expands in this direction until no further references are found. This assesses the number of rules a given rule would impact if the initial rule changed.
• Further context needed: The outward expansion identifies all rules which the initial rule refers to, and, expands until no further references are found.

For both, we look at the average length of chains originating from a node. A chain is an interrupted series of cross-references pointing in the same direction. A smaller example network is given in Figure 1 to help understand those metrics.

Figure 1: Illustrative example for different measures of network complexity

The table below shows the result of those metrics for the Basel II and Basel III frameworks.

Basel III is larger with four times the nodes, and three times the references compared to Basel II. However, on average, rules from Basel II make more references to other rules. The high Gini coefficients mean that both networks are mostly populated by rules which reference few other nodes, alongside some rules which make many connections. The relatively low reciprocity for both means that links mostly work in one direction. Figure 2 below shows the largest connected components, which have more than five nodes, of both networks.

Figure 2: Basel II network versus Basel III network

Basel II rules need more context than their counterparts with the average node having a chain length of .28 higher than Basel III. Relatedly, the table shows that alterations to rules in Basel III have a smaller knock on effect to rules further down the chain. While Basel III is significantly larger than the previous framework, its network is ‘simpler’, fewer references are made between rules, and chains are on average smaller.

The textual complexity of the Basel framework seems to increase in several dimensions. The increase in this complexity may negatively affect the ability of stakeholders to understand regulatory texts, which in turn may lead to negative consequences such as higher compliance costs or distortions in behaviour. Note however that the post looks only at one aspect of regulatory complexity and so cannot provide the full picture to assess the overall complexity of the Basel framework. Works on other aspects of regulatory complexity will therefore be helpful.

James Brookes and Matthew Everitt work in the Bank’s Advanced Analytics Division and Quynh-Anh Vo works in the Bank’s Prudential Framework Division.

If you want to get in touch, please email us at bankunderground@bankofengland.co.uk or leave a comment below.

Comments will only appear once approved by a moderator, and are only published where a full name is supplied. Bank Underground is a blog for Bank of England staff to share views that challenge – or support – prevailing policy orthodoxies. The views expressed here are those of the authors, and are not necessarily those of the Bank of England, or its policy committees.