Hidden metrics

Hidden metrics, not to be confused with the hidden variables of quantum mechanics, are one possible way of preventing the gaming that could otherwise result from targeting specific quantitative outcomes.

Let me explain. Say we are targeting female literacy in Pakistan. We issue bonds that will be redeemed when the literacy level of 15-year old girls in Pakistan exceeds 95 percent each year over a ten-year period. Campbell's Law tells us, rightly I think, that:

The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. Source

What metrics could we deploy that measure, accurately and inevitably, that which we want to achieve: namely, near-universal literacy for 15-year old Pakistani girls?  We could subject every girl of that age to a standard reading test, but we know that academic tests and their results, even in developed countries, are manipulated. Girls could be taught to the test (by, for instance, being given the test in advance and taught to memorise it by rote). Or anyone up the ladder from the administrators of the test to the collators of the results could be bribed to alter the data at any stage.

So instead we could take a random sample of, say, 100 schoolgirls from 100 districts around Pakistan at ten yearly intervals, and test them in non-standard ways for their literacy. We don't reveal which districts, let alone which girls, will be chosen; nor do we release exact details of the test. That makes it much harder to game the outcome.

More complex is the goal of peace. Now, nuclear peace is relatively easy to target: we can issue bonds that will not be redeemed, say, until a thirty-year period has elapsed during which there has been no detonation (accidental or not) of a nuclear weapon that kills more than 100 people. But what about peace more generally? Hot wars are fairly easy to recognise and define and so would be correspondingly easy to deter using a bond regime. But there are more nebulous ways of fomenting conflict between states (Ukraine), or of ratcheting up tension to levels that severly curtail quality of life.  In these instances we could use an array of metrics such as: trans-border movement of weapons, numbers of people killed or fleeing their homes. Even the lack of information could also be an indicator of conflict that could find its way into our calculations. We could also use survey data, such as attitudes about potential enemies, or expressions of fear. Other potential indicators are disruptions to food, water or electricity supplies and other results of damage to infrastructure. 
 
The important point is that in this as in other goals, we need not and indeed should not specify in advance exactly which combination of metrics and indicators will be used to determine whether or not the bonds' redemption terms shall be deemed satisfied. In general, we try, as far as possible, to target metrics that are, or that are inextricably linked to, exactly what we want to achieve. Where that is difficult, we try to prevent gaming by not specifying too far in advance the exact redemption terms of the bonds. The aim, at all times, is for bondholders to comply with the spirit as well as the letter of the goals set by the backers of the bonds.