Risk communication

Discussing asymptomatic women aged 40-50 who have participated in mammography, Gerd Gigerenzer writes in his excellent book Calculated Risks (pp41-3):

The probability that one of these women has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent tht she will still have a postive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?

Of 24 physicians who were given this information only two gave the correct answer - which is 9 percent. Probabilities are difficult for most of us to get a handle on. Gigerenzer is a persuasive advocate for instead using natural frequencies:

Eight out of every 1000 women have breast cancer. Of thse 8 women with breast cancer, 7 will have a positive mammogram. Of the remaining 992 women who don't have breast cancer, some 70 will still have a positive mammogram. Imagine a sample of women who have positive mammograms in screening. How many of these women actually have breast cancer.

It's much easier to see from this that only 7 of the 77 women who test positive actually have breast cancer, and indeed most of 24 (different) physicians given the information in this form, estimated correctly. Gigerenzer also discusses AIDS testing and the often tragic reaction of people who test positive when they, their testers and their counsellors know nothing about how frequently such positives are false:

Since the first AIDS cases were described in 1981, more manpower and money have been poured into researching HIV than any other disease in history. Little, in contrast, has been done to educate the general public about what an HIV test result means. (p139-40)

Ignorance about risk bedevils the law courts as well:

Many students who spent much of their life avoiding statistics and psychology become lawyers. Out of some 175 accredited law schools in the United States, only one requires a course in basic statistics or research methods.... [S]tudents who excelled in critical thinking could not evaluate whether a conclusion drawn from statistical evidence was correct or incorrect. (p159)

I think the lessons from this are important for policymaking, and not just because many of our policymakers used to be lawyers. One lesson is that it's quite possible for even well-meaning professionals to have no idea about statistics and risk. As our economies and societies grow ever bigger and more complex, policymakers will rely more and more on statistics. Inferences drawn from them need to be robust. My feeling is that the ignorance that Gigerenzer documents thrives in the compartmentalized, specialized policy environment we have today. Doctors, lawyers and politicians don't know very much about risk and neither does most of the public. There's very little monitoring of policies for effectiveness and there's very little incentive to get policies right. You can't (easily) legislate for effective risk communication and understanding, but what you can do is throw the achievement of social goals open to the market - by using Social Policy Bonds for example - so that errors of the sort that Gigerenzer documents do not persist and entrench themselves.