Facts, Analysis and Comment.

The Behaviour and Psychology of Regulatory Organisations

This note is one of four which consider the behaviour of large organisations and their interaction with those who regulate them. This note concentrates on the behaviour of regulators themselves. Other notes look at ...

    •    the behaviour of larger organisations, and
    •    examples of the problems etc. that can arise in regulating such organisations, before finally considering ...
    •    the lessons that might be learned from the above analysis and examples.

Modern regulators are typically large, complex and well-established organisations, largely unaware of the problems that their size and age can cause. Also, because regulators have to follow complex legal and other processes, they typically devote little or no time to considering regulated organisations' likely responses to regulatory pressure. (In contrast, however, large regulated organisations do spend a lot of time and money considering how best to influence the behaviour of regulators.) This note seeks to explore some of the problems that can arise within regulators. (It is also well worth reading Mannie Sher's paper on The Psychology of Regulation‌, the second part of which describes the worrying - as well as entertaining and astonishing - outcome of experiments which examined the the behavioural dynamics of regulators and the regulated.)

Boards or Individuals?

The first utility regulators were strong-minded individual Directors-General such as gas regulator James McKinnon, telecoms regulator Bryan Carsberg, and rail regulator Tom Winsor. Their clear independence of government and tough approach to their industries was initially welcomed by most observers - others than those they were regulating of course. But successive governments grew nervous that they were giving such individual too much freedom, including the freedom to be weak regulators as well as strong ones. Over time, therefore, it became the norm to appoint Boards or Commissions rather than individual regulators.

It is almost impossible to tell whether this change has led to different regulatory outcomes. Strong Board members can in theory stiffen the resolve of their more timid colleagues. Alternatively, to the contrary, one nervous Board member can inhibit the activities of a Board that wants to proceed by consensus. The truth may be that the change did not often make much difference to regulatory practice because the new full time Chief Executives generally control the flow of information and advice to their part time, non-executive and often somewhat amateur Boards. Indeed, Chief Executives' power is often strengthened by the appointment of a number of his/her direct reports on the supposedly supervisory Board.

Those regulators most often accused of having relatively weak Boards include Ofgem and the OFT. Regulators at the other end of the spectrum include the Competition and Markets Authority ('CMA' - whose constitution, historically modelled on Whitehall's split between Ministers and officials - provides for decisions to be taken by 'Inquiry Groups' comprised of non-staff Commission Members advised by full-time staff) and Postcomm which has now been assimilated by Ofcom but whose constitution was modelled on both the CMA's predecessor organisation (from where it recruited its first Chairman) and Whitehall (from where it recruited its first Chief Executive).

Apart from the 'pros and cons' mentioned above, it is worth noting that strong regulatory boards tend to operate to a different rhythm from their Chief Exec dominated counterparts. Staff are less empowered and so significant decisions have to await the (often fortnightly or monthly) Board/Commission meetings. But the advice presented to such demanding Boards tends to be well researched and argued and so the decisions tend to be more robust, thus permitting somewhat faster progress towards significant regulatory objectives.

Bureaucracy v. Entrepreneurship

There is a significant and growing problem in that regulators are not well adapted to facilitating innovation in the sectors which they regulate.  They are often under intense pressure to intervene when companies make good profits (albeit within price controls), to get involved in all sorts of issues peripheral to their main remit, and also to become a complaint handling body.  Even though they may resist such pressure, they may not be keen to see their industry take the financial or performance risks associated with most innovation.  

Dieter Helm makes similar points in his Irresistible Urge to Meddle paper. Here is his summary:

When the great utilities were being privatised, one of the key objectives was to establish a regulatory regime which would be credible to investors and fair to customers. The way it worked was that the companies would be given fixed contracts ex ante, and these would be reset on a periodic basis. Within the period, prices would be fixed by an RPI-X formula.

The logic was very clear: the only way in which the performance of the companies could be improved was to incentivise them to do so. Fixed-price, fixed-period contracts gave them the chance to outperform, and it was this outperformance which managers would chase after in the interests of their shareholders. The regulators would re-base the prices at the end of the period as the companies revealed what they could actually deliver, rather than what they told the regulators ex ante.

It was all very simple, and that was its attraction. But it never worked out as intended, for the very good reason that regulators and politicians could not resist the urge to intervene. Faced with the chance to curry favour with the customers, politicians kept up the pressure and most regulators eventually obliged. Indeed it is hard to think of many periods when there has not been some form of intervention, and the companies quickly learned that there was, in effect, a tolerable band of outperformance, beyond which the risk of intervention would be high. 

Regina Finn & Simon Less'  essay on independence of thought and Regulatory Capture is also relevant.

The Principal-Agent Problem &
Regulatory Capture

The note about the behaviour of large regulated organisations includes a section on the principal-agent problem, which leads to shareholders and senior managers not knowing what is really going on in their organisations. But it is just as important to remember that regulators are themselves agents, working for a principal which is in this case the government or legislature of the day. The principal-agent problem accordingly raises its ugly head just as much in this structure, as it encourages regulators to enter into implicit understandings with the entities that they are supposed to be supervising. Such relationships are described by the participants as 'sensible', 'pragmatic' and/or 'light touch'.  Critics characterise such behaviour as 'regulatory capture', exacerbated in the case of economic and financial regulators by their almost total dependence on their industries for the information they need to do their jobs.

Evidence of regulatory capture includes

(a) quality and safety regulators giving notice of inspection visits,
(b) frequent staff interchange between regulator and industry,
(c) the reluctance of either party to be strongly critical of the other, and
(d) for them to engage in increasingly introverted consultation processes, baffling to the outside world.

It is interesting to compare, for instance, the nature and tone of energy regulation in the 2010s compared with the fierce arguments of the 1990s. But it is important to acknowledge that politicians are ultimately responsible for the character of the regulator, through the messages that are sent (e.g. about the virtue of light regulation), the institutional arrangements that they create in legislation (regulatory boards will be less 'extreme' than powerful individuals) and through the appointments that they make (politicians may well prefer to appoint those who will not 'rock the boat' and/or attract criticism from powerful business interests).

As noted above, it is well worth reading Regina Finn and Simon Less' 2013 essay on regulatory capture.

A prominent example of (b) staff interchange was ex-Financial Services Authority Hector Sants joining Barclays Bank in December 2012 only six months after leaving the FSA where he was responsible for regulating the wholesale and institutional markets from 2004, and Chief Executive from 2007 to 2012, i.e. well before and then throughout the financial crisis. He was recruited to lead Barclays' compliance function. Barclays chief executive said "Relationships with our regulators ... are obviously ... of critical importance to us. We must apply a renewed leadership focus on these to make them as constructive and productive as possible." In the short term, it seemed likely that Mr Sants would seek to improve Barclays very tarnished reputation with regulators by improving compliance (see examples of Barclays' and other banks' dreadful behaviour on this web page). In the longer term, however, he would only succeed if regulators begin to apply a less hostile, and then pragmatic and occasionally forgiving, attitude to Barclays:- classic regulatory capture. And Mr Sants would have been less than human if he had not - throughout his time at the FSA - been aware that he would one day join a big financial institution - and anxious to avoid appearing over-reactive or over antagonistic in his dealing with the sector ... which means being quite cautious when tempted to take strong action. See also Notes 2 and 3.

Eyebrows were also raised at the news that Tim Smith, the Chief Executive of the Food Standards Agency, was leaving that job in late 2012 to join Tesco, a huge supermarket chain which had for many years resisted the FSA's attempts to have the industry adopt traffic-light labelling for healthy and unhealthy foods. And Mr Smith had previously joined the FSA from the food industry so, whilst leading an important regulator, he had in fact been dealing all the time with both his recent and future colleagues.

Meanwhile, Private Eye reported in February 2014 that the financial establishment is well in control of both the audit and accountancy regulator, the Financial Reporting Council, and the Financial Conduct Authority which were together investigating KPMG's role in both the Co-op Bank/Britannia Building Society and the HBOS scandals. This was because the Chair of the FCA was the ex-Chair and senior partner of KPMG, whilst the Chair of the FRC was the former Chair of both Citigroup (Europe) and Lloyds Bank.

There was an interesting report in October 2015 of behaviour which appears to fall under (c) and (d) above - a regulator becoming far too close to the company that it was supposed to regulate. This example was the Drive and Vehicle Standards Agency (DVSA) which refused to disclose test results on Porsche cars. The tests allegedly showed that (in order to reduce fuel consumption at 30mph) the car failed to accelerate at that speed unless and until the accelerator had been pressed for 2.5 seconds - a long time if you are trying to pass a lorry in the face of oncoming traffic. DVSA argued that disclosing the results would reduce Porsche's 'proactive involvement' in safety testing - a suggestion which the Information Tribunal found 'disconcerting in the extreme'.

It is interesting to note that principal-agent theory postulates the slow growth of a number of overlapping networks or cliques within any large organisation, and so predicts that organisations will become steadily more bureaucratic as they age. This is because senior executives will learn that a substantial proportion of their employees are bound by their previous personal commitments to each other, so that it becomes imperative to overcome this by reducing the scope for personal discretion. It is also the case that organisations do learn the best way of carrying out certain tasks, and this too reduces discretion. As noted above, the cliques and teams will often respond by ignoring the bureaucratic rules. Even worse, however, the gradual reduction in tolerance for initiative and discretion leads to recruitment and promotion policies which favour those who prefer to work in rule-bound environments, so reducing the organisation's ability to innovate or to cope with problems which are not mentioned in the rule book. This is one of the factors which underlie the growth of the 'box-ticking' culture so prevalent in many modern regulatory factories. 

Should Regulators Assume Rational Behaviour?

The short answer to this question is 'No!'. There is a wealth of evidence that most of our thinking happens well below the level of our awareness, and these subliminal and/or emotional processes are certainly not logical. It is well established, for instance, that beliefs are much more powerful than rational arguments, whether scientific or otherwise. And we are certainly not self-contained individuals. Instead, we are deeply linked and respond very quickly to others in ways that we are not aware of, and we don't think about. Group-think, herd behaviour and the MacWhirr syndrome, discussed in this note about the behaviour of regulated organisations, are all examples of illogical behaviour, which occurs just as much within regulators as within the companies they regulate.

Good governance including effective accountability and strong consultation and transparency processes are all intended to reduce the impact of these behaviours, but they have their down-sides:-

What happens when Transparency meets Blame Avoidance?

The above heading was inspired by my reading of an article of the same name by Oxford University's Christopher Hood published in Public Management Review in 2007. In that article, Professor Hood argues that the pervasive prescription of transparency and accountability as a universal prescription for good governance might lead to highly undesirable consequences when accompanied by blame avoidance by regulators and others in the public eye. The conventional assumption is that the management of political and reputational risk involves an 'upside' of acquiring credit and a 'downside' of attracting blame. But he and other academics are beginning to believe that there is a a strong 'negativity bias' - so that fear of attracting blame way outweighs the attractions of bold and effective political or regulatory decision making.

The proposition, in more detail, is that negativity bias encourages regulators to avoid making contestable or appealable judgements that create obvious losers. Negativity bias also encourages regulators to develop policies and bureaucratic routines that minimise the risk of institutional or individual liability and blame. And they introduce protocols and automaticity which minimise the exercise of individual discretion by both staff and senior officeholders.

Possibly the worst example of blame avoidance was the Financial Services Authority's behaviour before the 2008 financial crisis. Analyst Daniel Davies noted that '... the defining weakness of the old FSA was a paralysing fear of doing something that might be criticised.'

This is not to say that access to judicial review is not a good thing if it merely encourages regulators to follow due process. But it is a very bad thing if regulators worry about losing in court. Every court case carries risk, and every court setback leads to a PR attack from the company concerned, followed by lazy journalists' easy headlines about regulatory incompetence. The fact is that good regulators are prepared - happy even - to defend their actions in court, even though this means losing on occasions. Indeed, regulators who have near 100% records in court battles are almost certainly being too cautious in the way they carry out their duties. But fighting court battles is time-consuming and expensive, and this is itself a growing problem for resource-limited regulators.

Limited Resources Leads to Box-Ticking

Ngaire Woods of Oxford University's Blavatnik School of Government made a strong and important point when interviewed by Civil Service World in June 2014: "[Creating regulators as well skilled and equipped as the banks would demand] absolutely phenomenal resources. [One consequence is] government being seen to regulate by requiring lots of box-ticking, and business being seen to comply by ticking those boxes".

Too Many Complaints?

A small but persistent and intractable problem is that regulators typically receive many more expressions of concern than they can sensibly handle. They therefore need to sift them so as to investigate the most serious. This can lead to their discounting concerns emanating from outside the regulatory system - as the SEC did when considering Harry Markopolos' submissions about Bernie Madoff's Ponzi scheme. And it almost always leads to their commencing more investigations than they can efficiently handle, so investigations proceed at far too sluggish a pace. This was probably a factor in the FSA's willingness to allow the UK's mini-Madoff (Terry Freeman) to continue to trade despite their receiving complaints about him, and despite his having previous convictions and being disqualified as a director. The Times reported in January 2011 that the FSA had been gathering intelligence on Freeman for more than two years but had not taken any enforcement action.

The number of complaints is a particular problem for larger regulators - see further below - and there is no easy solution, although all regulators would be well advised to have systems in place which ensure that all potential leads are reviewed by staff who are (a) experienced, (b) sceptical, as long the scepticism is directed at the regulated entities, and not the complainants.

Bigger is Better?

It is tempting to assume that a larger regulator will be a stronger regulator, but this is not necessarily so. Just as the Boards of large corporations lose touch with their businesses, so the Boards of large regulatory authorities often lose touch with much of what is done in their name, and are in practice unable to ensure that their staff are as focussed, determined and sensible as they would no doubt wish. This was one reason for the FSA's failures in the period running up to the 2008 financial crisis. Follow this link for a more detailed discussion of this subject.

There is still much Good Regulation

This is perhaps the point to mention examples of good regulation.

One such was the response of the Civil Aviation Authority (CAA) following the eruption of the Eyjafjallajokull volcano in Iceland in in early 2010. There was very little hard information available about the way in which aircraft engines respond to ash clouds of various densities etc., but it was certainly clear from numerous incidents that ash could bring down an aeroplane (see Note 1 below). The only existing guideline was "AVOID, AVOID, AVOID". The CAA therefore very wisely insisted on getting the aircraft and engine manufacturers to assess the dangers - a role with which they were very uncomfortable given the potential for legal liability if they were to get it wrong. The CAA also liaised very closely with all other relevant bodies, including the Department of Transport who were reasonably supportive. The CAA and its Chief Executive Andrew Haines were nevertheless unfairly pilloried by the industry. His response was perfect: "We will not listen to those who effectively say 'Let's suck it and see'". It was greatly to his and his colleagues credit that they withstood the pressure. Excellent regulation!

It was good, too, that the CAA and many others, including vulcanologists and meteorologists, subsequently got together to establish safe thresholds and guidelines which will reduce the scale of future disruption following large eruptions.

Greatly improved water industry efficiency, and telecoms/broadband local loop unbundling, are generally regarded as significant successes in utility regulation. 

Inspectors Inspected!

The National Audit Office published an interesting report (Inspection: A comparative study‌) In February 2015.  They had looked at the five home affairs and justice inspectorates and their report contains interesting comment on the real value of those inspectorates, whilst commenting that the inspectorates (and their sponsor departments) 'could do better'.


  1. In the three months to February 1989, five commercial jets were damaged by ash clouds from the Redoubt volcano in Alaska. And 7 airliners carrying more than 2000 people suffered dramatic engine failure amongst the total of 60 aircraft that were damaged in ash clouds in the 12 years to 1993. [The Guardian 21 April 2010]
  2. One FT reader commented as follows on Hector Sants' recruitment by Barclays: Sir, I read with much disappointment Lex's description of the appointment of the former chief executive of the Financial Services Authority by Barclays as "impressive" (December 13). Presumably, you meant the fact that we'd sleepwalked into the greatest financial crisis in living memory, almost bankrupting the country, under the regulator's watch and yet executives of this organisation still managed to get plum jobs in finance. Impressive? No. Outrageous? Probably yes. ... the ensuing carnage, regulatory and political incompetence at the time should ... never be forgotten.
  3. Sir Hector resigned from Barclays in November 2013 shortly after being signed off sick suffering from exhaustion and stress.