Assume nothing
Believe nobody
Check everything
The same rules should be applied to scientific work. In fact, they probably should be applied to every aspect of life. But doubting everything all the time is exhausting and impractical, as Descartes noted centuries ago. So we end up taking shortcuts, assuming plenty, believing lots of people, and checking very little.
What most people do in their attempt to get the right amount of doubting and checking in their lives is to double-check only things that appear dubious for some reason. It's a technique that often works (that's why people do it!), but sometimes has fatal flaws. This week I have seen a few shining examples of this method failing miserably.
One way it can fail is because sometimes, things that appear simple, clear-cut and free from any dubious side at first sight, may actually be very different when they are examined more closely. This is what's happened to the solicitors of my Ex, who is currently in prison (you already know that if you have been following my blog). There was a phone video recording of him threatening my ex-neighbour. Everybody knows that recorded evidence is pretty solid, so why double-check? Well, it turned out that my Ex never had a chance to examine the recording closely. It had been shown to him once, but it had no sound. Afterwards, he found out that it did have sound, but he hadn't heard it until this week. The recording wasn't of him threatening my ex-neighbour. It actually was a recording of him saying in a calm voice: "The police has this anonymous letter from you saying..." and then, in a louder voice, he was quoting to my ex-neighbour his own threats. Nobody had heard before the first part because they didn't turn up the volume far enough. Without this key piece of evidence, the whole case folds over. And he's been doing two months in prison, just because the police, the defence solicitors, and the prosecution, couldn't be bothered to double-check what appeared like pretty solid evidence.
Another way it can fail is because it's easy to check if the things that you can see appear dubious, but you can't say much about things you can't see. The things that are missing can trip you as badly or more than the things that are there. This is, for example, what I think has happened with the IPCC scenarios. I recently sent an email to several scientists with what I think are the main flaws in the IPCC scenarios, and it isn't so much about the content of their models, but about the things missing in them:
1) Population seems to be derived from the UN projections in all of the models and scenarios. The UN projections, as far as I know, don't consider the possible effects of climate change in any way. However, it's obvious that future population figures will depend strongly on the availability of food, and the availability of food is likely to depend on climate change, as climate change is expected to have an effect on crops.
2) Economic growth seems to be used as an input in several of the models, as if some trends were estimated for economic growth and used as an input to calculate other variables. But it's much more realistic to represent in a model the reverse causality (more energy leads to more economic growth) or a feedback loop between energy usage and economic growth. Also, I can see nothing to indicate that GDP is calculated as a sum of the three traditional sectors (agriculture, industry and services), so I'm not sure how it's calculated. At one point the report says that all scenarios assume more affluent worlds, is this just taken a priori as an assumption?
3) Several models don't seem to contain any kind of land model, and those that do, don't seem to have good feedbacks into other parts of the model. It's fair to assume that if a region had problems to feed their population, they would divert more resources into food production.
4) None of the models seem to give any consideration to the idea that the quality of the remainder of fossil fuels and nuclear fuels gets lower over time, as the higher quality grade are exploited first. This has implications for the supply of energy and the resources devoted to obtain energy.
5) It's unclear if the models allow for the possibility of energy demand not being met by supply, due to depletion.
In a similar vein, a highly intelligent member of the Energy Group keeps sending emails around about people doing criticisms of minor parts of climate change models. He is personally convinced that checking all these dubious details is the key to finding the true answers in the end. But he misses a critical point: one could build a highly flawed model where all the details appeared perfectly correct - if the model was missing some key elements.
And a final way it can fail is because we can't help believing what we believe. If you believe something, it certainly doesn't look dubious. I saw an example of this on Monday, during the Transport Group meeting. The coordinator of the group believes a number of things about organizing, like: it's all about convincing some Council officers, or: people who are really interested will come, no need to make any big efforts to attract them. If he examined the evidence dispassionately, he'd realize that he's never been very successful at any campaign he's organized, so there must be things that he doesn't know and places where he's wrong. But when a few of us challenged gently some of those beliefs, he defended them with all his conviction.
So, if checking dubious things can sometimes fail miserably, what works better? My method, that works quite well for me, is this: Check the big things, whether they look dubious or not, and the small things only when they look very dodgy and you have time for them.
And how do I know if something is big? I just think: "Which things I must get right, because if I was wrong with them the whole thing would go to hell in a basket?" If you are a solicitor, one of the answers is: "The evidence". If you are a scientist in charge of one of the chapters of the IPCC, one of the answers is: "The major factors influencing greenhouse gas emissions". If you are an organizer, one of the answers is: "Knowing how to organize".
ABC is simple, but it isn't workable. Try instead CBA: Check Biggies Always.
No comments:
Post a Comment