Saturday, 1 July 2017

Ivory Towers and the Grenfell Tower: The problem with Evidence

The Grenfell Tower fire represents a collapse of trust in expertise and evidence, and will bring about a reawakening of scepticism. Newsnight's report on "How flammable cladding gets approved" - raises questions about the role of evidence beyond fire safety. In policy in health, education, welfare, economics and housing evidence is the principal aid for decision-making. What Enid Mumford calls "dangerous decisions" are supported by studies which demonstrate x or y to be the best course of action. The effect of these studies is to attenuate the range of options available to be decided between. Of course, in that attenuation, many of the competing descriptions of a phenomenon or subject are simplified: many descriptions are left out, some voices are silenced. Usually, the voices that are silenced are those "on the edge": the poor, immigrants and the occasional "mad professor". From Galileo to Linus Pauling, history tells us that these people are often right.

Understanding "evidence" as "attenuation" helps us to see how easily "evidence-based policy" can become "policy-based evidence". Evidence can be bent to support the will of the powerful. The manifestations of this exist at all levels - from the use of econometrics to produce evidence to support austerity to the abuse of educational theory in support of educational interventions (which so many educational researchers, including me, are guilty of). But it helps academics to get published, to raise our status in the crazy academic game - and, once established in the sphere of the University, the habit sticks. Effective decision-making is intrinsic to effective organisation. If organisational pathology creeps in, decision-making within a pathological organisation will be constrained in ways which obscure real existent problems.

The deeper problems concern academia's and society's allergy to uncertainty. We hold to an enlightenment model of scientific inquiry, with closed-system experiments and the identification of causal relations through the production of event-regularities. Too often we pretend that the open systems with which we engage are closed systems whose event regularities are no longer physical events, but statistical patterns. So Stafford Beer's joke that "70% of car accidents are caused by people who are sober" entailing that we should all drink and drive, highlights the dangers of any statistical measure: it is an attenuation of descriptions - and often an arbitrary one at that.

The computer has changed the way we do science, and in almost all areas of inquiry from the humanities to physics, probabilities are what we look at. These are maps of uncertainty, not pointers to a likely successful outcome, or a statistically proven relation between an independent variable and a probability distribution. What is an independent variable, after all? It is a single description chosen out of many. But its very existence is shaped by the many other descriptions which are excluded by its isolation. And we don't seem to care about it! I review endless depressing papers on statistical approaches to education and technology, and I see these assertions being made without the slightest whiff of doubt - simply because that is how so many other papers which are published do it. I reject them all (although always gently - I hate horrible reviews - but always inviting authors to think harder about what they are doing).

Uncertainty is very difficult (probably impossible) to communicate through the medium of the academic journal article. The journal article format was devised in 1662 for an enlightenment science which is radically different from our own. Of course, in its time, the journal was radical. The effect of printing on a new way of conducting and communicating science was only just opening up. Printing was doing to the academic establishment what it did to the Catholic church a century before. Enlightenment scholars embraced the latest technology to harness their radical new practices.

We should be doing the same. The experiments on building cladding are easily demonstrable on YouTube. Equally, uncertainties about scientific findings can be expressed in rich ways using new media which are practically impossible in the journal. The scientists should learn from the artists. Furthermore, technology provides the means to democratise the making of descriptions of events. No longer is the description of an event the preserve of those with the linguistic skill to convey a compelling account in print. The smartphone levels the playing field of testimony.

Our decisions would be better if we became accustomed to living with uncertainty, and more comfortable living with a plurality of descriptions. The idea of "evidence" cuts against this. We - whether in government or academia - do not need to attenuate descriptions. Uncertainties find their own equilibrium. Our new media provide the space where this can occur. Universities, as the home of scholarly practice in science, should be working to facilitate this.

No comments: