Models and the perils of (too little) uncertainty

That  conference last week in Cambridge on “Challenging Models in the Face of Uncertainty” turned out to be more about uncertainty, risk and risk perception than about models. That was a disappointment as the risk stuff was mostly pretty routine, even when presented as news (a US researcher offering results on cultural cognitions which, although recent, seemed to derive directly from Mary Douglas’s application of her grid and group scheme to risk perception in the early 1980s, certainly made it seem there was nothing new under the sun).

So what follows is more a gathering of impressions than grappling with any of the presentations directly. I was surprised so much of the conference related to futures thinking. Probably shouldn’t have been. Even allowing for the fact I am bound to pick up on such references, I suppose a meeting on modelling, risk and uncertainty was always bound to feature the future fairly prominently.

There are uncertainties about the past, and models – climate models are the obvious example – may be calibrated or validated by running them through a period for which there is historical data independent of the model. Risks, though, I think are inherently related to discussion of possible future outcomes. An exception, perhaps, might be when newly-informed analysis reveals that we were subject to risks at some past time of which we were unaware. Recent examples there include the inclusion of lots of SV40 in polio vaccine preparations or, more potent because more recent, inadequate rendering of bovine carcasses which fails to denature prions. But these fall in the rrealm of “unkown unknowns”, which – surprise – were mentioned a lot, as Alice Bell has already pointed out.

However, I think the other two domains look mainly toward the future as well. Models, in the sense which was usually being discussed, are simulations of some situation which expresses our understanding of how it operates in a virtual world bounded by permissible (or computable) solutions of some coupled system of equations. Each run plots the course of a possible future of the system. And the uncertainties which concern us most, politically if not always personally, are about the consquences of some set of actions (or inaction) in the future we will actually see unfold.

Unfolding, incidentally, might be a metaphor best avoided, as it implies the future realised is already somehow mapped inside the preceding states in one version, but I’m not feeling linguistically agile enough today to write round it…  so read it as “unfolding”.

I’m not going to log all the times the F-word cropped up in the two days of the meeting. There were a lot. The strongest feeling I came away with, which in spite of the several mentions of cultural cognitions seemed unsufficiently acknowledged, was that the response to uncertainty, and especially to the acknowledgement of uncertainty in “scientific” modelling, depends on the context.

Two examples, one mentioned in the meeting, one which I think was not, although it could easily have come up. The first was the Royal Society’s revised summary of current climate model results, published last week. As widely noted, this is an attempt to express more clearly than previous efforts the uncertainties of climate science, while reinforcing the consensus that global warming is real and rather hazardous for all our futures.

If you want the most skewed sample possible, take a look at the comments on the Daily Mail’s carefully skewed piece on the report (which contains at least one quote which says something about its contents that is demonstrably untrue). The vast majority simply interpret “uncertainty” to mean “scientists really know nothing and are making stuff up to further their own interests”. It’s a bit of a leap, and your summary might be different, but that’s how they read to me. Not a great surprise, of course, but it does illustrate the difficulty of communicating the uncertainties – and what is and is not uncertain – in complex cases.

[NB: comments not available at this link at the time of writing. If they do not reappear, I will post some here later as I archived them for my own edification]

So would less uncertainty be better? Not so sure. Which brings the second example. David Runciman’s review of Tony Blair’s memoir in last week’s London Review of Books (the best I’ve read) relates the former PM’s conviction that his job as leader was to get a grip on things. One episode which reinforced this, apparently, was the outbreak of foot and mouth disease in 2001. Blair wasn’t happy with the way the agriculture ministry and the vets were dealing with it. He gets briefed and has a think over the weekend. Then, according to Runciman:

On his return to Downing Street on Sunday, Blair concludes he has no choice but ‘to grip the whole thing’. He gathers his close advisers, who in this case include his chief scientific adviser, David King. King explains to him what needs to be done. ‘Essentially, by means of graphs and charts he set out how the disease would spread, how we could contain it if we took the right culling measures, and how over time we would eradicate it.’ Blair was sceptical: ‘How could he predict it like that, with so many unknowns? But, almost faute de mieux, I followed his advice – and blow me, with uncanny, almost unnatural accuracy, the disease peaked, declined and went, almost to the week he had predicted.’

This was doubtless good for the reputation of scientific advisors. It also, less auspiciously, showed the need to resort to the armed forces in a crisis (they implemented the cull according to King’s criteria). And it validated Blair’s conviction that he could identify the right course of action and his job was to stick with it.

So there you are. Having a model chock full of (specified) uncertainties, like a global climate model, is a drag on implementing sensible policies. And having one which has fewer uncertainties, although immediately helpful, sets a precedent which may bring out the worst qualities in a decision-maker. Tricky eh?

Explore posts in the same categories: fear of the future, military, Uncategorized, virtual worlds

Tags: , , ,

You can comment below, or link to this permanent URL from your own site.

One Comment on “Models and the perils of (too little) uncertainty”

  1. […] a persuasive follow-on to the recent conference on uncertainty, modelling and decision-making in today’s Nature, in the form of a sympathetic […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: