What exactly is the problem?

'When greenbacks are on offer, American schoolchildren seem to try harder': the Economist discusses how financial incentives can change test results:

[R]esearchers conducted an experiment in secondary schools in Shanghai and America. In each place pupils were split into two groups. The first answered 25 maths questions that had appeared in PISA. The second took the same test, but before the pupils did so, they were presented with an envelope with 25 dollar bills or the equivalent in yuan. The teens were told that for every wrong answer they would be docked a dollar. [The researchers] found that the ploy boosted scores among American students relative to their compatriots without a cash incentive, but not among the Chinese ones. Effort, not ability, may explain the gap between American and Chinese pupils, 'the Economist', 19 August. PISA is the Programme for International Student Assessment

The boost was significant: 'According to some rough calculations, if extrapolated to the main PISA test, the improvement in performance would have moved America from 36th to 19th in the ranking [of 69 countries], in which Shanghai came top.' It's an interesting result, and one that should make us question what the tests are trying to measure, and whether there are wider policy implications.

Test scores will most likely continue to play a big role in determining policy and the allocation of resources in education. That would be true whether or not we issue Social Policy Bonds that aim to achieve, say, universal literacy. So, in the light of the research results summarised above, should we offer cash incentives to pupils about to take literacy tests?

I'm inclined to think not, at least when it comes to measuring basic literacy. One reason is that motivation, or the lack of it, are significant in themselves. If children of school age find a basic reading test too burdensome to pass without a financial incentive, then that in itself can  be seen as a problem that needs to be solved. The nature of that problem might be a general cultural one, or one that's specific to certain classes of pupil. We might even interpret the difference between performance with a financial incentive and without as a social problem; and aim to narrow it.

There's no definitive answer though. Much depends on how we're going to use test scores, and whether there are other indicators that can usefully be targeted at the same time. As a society, we do need to think carefully about what we are trying to achieve. 'Teaching to the test' is problematic in itself and, while I do think universal literacy and numeracy are valid goals in themselves, and not very susceptible to the effects described above, we might do better to target for reduction as well as, or instead of, illiteracy, the social problems of unemployment and poverty.

When government should step back: higher education

Marina Warner writes:

[T]here is a central contradiction in the [UK] government’s business model for higher education: you can’t inspire the citizenry, open their eyes and ears, achieve international standing, fill the intellectual granary of the country and replenish it, attract students from this country and beyond, keep up the reputation of the universities, expect your educators and scholars to be public citizens and serve on all kinds of bodies, if you pin them down to one-size-fits-all contracts, inflexible timetables, overflowing workloads, overcrowded classes. Diary, Marina Warner, 'London Review of Books' dated 11 September

Quite. There are some easily quantifiable benefits of education, such as universal literacy and numeracy, but not many. Government can and should target such goals and it could also usefully target attendance at approved educational establishments for children up to the age of 16 or 18. But when it comes to higher education, government should perhaps step back; it could still fund institutions if there's a public will for it, but there is a strong case for making its funding conditional only on certain minimum standards, rather try to apply the narrow, accountancy-based, short-term goals that are a feature of the business world. The demand for tertiary education is relatively informed; students relatively mobile. Government in this, and other policy areas, needs to exercise some humility. Diversity of funding sources, as between government and other sources, and within government, would also be helpful. I've done a short piece here on how an outcome-focused Social Policy Bond regime could approach education.

An update: released today (8 September) is a report done by the charity Save the Children, which says, referring to the UK: "The most comprehensive study of pre-school and primary school-aged children in a generation found disadvantaged children are the worst affected, with four in ten not reading well by the age of 11."

Goals for education

A correspondent has kindly drawn my attention to President Obama's accreditation reforms for higher education, as outlined in his State of the Union address. There is much that seems of merit to me in these reforms, but these reforms have encouraged me to think more broadly about education.

My premise is that policymakers should target, above all, social and environmental outcomes. As with other variables, like income say or, more to the point, literacy, at the basic level there is a strong correlation between something measurable and human wellbeing. So, for instance, we can measure functional literacy quite well, and the outcome of 100 percent literacy is a worthwhile one to target. In this vein, I was pleased to see that President Obama proposed programs to provide early education for four-year olds from lower-income households. There are also measures to encourage higher graduation rates from high school. At these levels there is a strong correlation between attendance at an educational facility and real, meaningful outcomes.

But things are much more complicated at higher levels of education. We really need to think about what sort of outcomes we want. Some social outcomes result only indirectly from education and, under a Social Policy Bond regime, we'd do better to target those outcomes themselves. For example, it would be more efficient to target unemployment directly rather than indirectly through the educational system.
Under a Social Policy Bond regime we could do that explicitly, and investors in  Unemployment Reduction Bonds (or Employment Maximisation Bonds) might well decide that the school system needs some sort of overhaul to meet the targeted goal.

This indicates how I think about targeting the supposed means toward achieving an employment target via such things as "graduation rates, costs, average amount borrowed etc" that are the focus of President Obama's reforms. These are less ends in themselves than supposed means to an end (or various ends). All our experience tells us that such narrow, short-term, top-down, goals can - and will - easily be gamed or manipulated or will just not end up doing what they are supposed to. Obama's reforms concentrate heavily on strengthening the regulation of institutions. This might be praiseworthy, but I'd much prefer to see fewer administrative fixes and more targeting of specific broad outcomes, which, under a bond regime, would motivate investors in to make their own decisions about how best to achieve them. We need diverse, adaptive solutions of the sort that government just cannot manage. Government, under a bond regime scheme, would still ultimately subsidise or pay for the achievement of these goals, and raise the revenue for their achievement, but it would contract out the actual achievement to investors.

As happens so often with government everywhere, President Obama's proposals take the existing institutions as a given. They take the existing institutional framework as a given too. (I suspect Obama's healthcare proposals suffer from the same problem.) In the long run  I think we'd do better to think carefully about our real goals and target those specifically. I think these would include high employment, universal literacy, lower crime rates, better physical and mental health and a few others. Education is, in my view, a means to those ends - and others, less easily specified which perhaps should not be a government remit at all.

So...Obama's proposals for higher education reform are undoubtedly well meaning, and arguably positive, given the existing framework. But the existing framework should be challenged. Giving existing institutions more targets might well do some good, but I see them as entrenching the existing, and increasingly questionable, institutional structure at a time of rapid social and technological change. Instead of micro-managing the current system we need to clarify exactly what outcomes we, as a society, want to see, and which ones we think government can legitimately and usefully target, and which it can't.