Tuesday, February 25, 2014

The root of all value: a neural common currency for choice

Levy, D., & Glimcher, P. (2012). The root of all value: a neural common currency for choice Current Opinion in Neurobiology, 22 (6), 1027-1038 DOI: 10.1016/j.conb.2012.06.001


Introduction


Humans make enormous numbers of choices. Some of them are – at least in principle – relatively easy, for example choosing between one hundred, and ten thousand dollars or between one sip of water and ten sips. But others are more tricky, for example choosing between spending some free time watching a movie and playing tennis. In the latter case the two options vary in more than one way. Consequently a simple ‘more is better’ rule doesn’t determine what is best.

Given that we do, however, make large numbers of choices, including these superficially tricky ones, often without noticeable effort, we might wonder what it is that we do when we make them.

One idea, with a fairly long history, is that we do it by converting choices, including tricky ones, into easy ones, by expressing the worth of the different options in some abstract and general scale that is used for all choices. This more abstract scale is, of course, often referred to as a ‘common currency’ but might also be called ‘utility’ or ‘reward’, or ‘pleasure’ or just ‘value’. Once this conversion has been done, the idea is that all choices are simple ones: now we can take the one worth the most in the common currency.[1]

According to some, if our choices, including the tricky ones, are consistent, then we must be converting the options into values in a common currency. As discussed elsewhere on this blog, Shizgal and Conover have argued that:

“In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension” (Shizgal & Conover 1996).

This argument need not be decisive. That is, it is possible that some other mechanism which did not involve an internal common currency could explain ‘orderly choice’. So one question that arises here is a theoretical one, about what range of mechanisms could produce orderly choice. Perhaps the only viable options are mechanisms involving common currency. Independent of this theoretical question, is an empirical one: How, in fact, do humans achieve orderly choice?[2] Levy and Glimcher survey some evidence, supplemented by a new meta-analysis of previous research, to defend the conclusion that whether or not they have to, human brains do in fact implement a common currency.

Levy and Glimcher’s review of existing evidence


Levy and Glimcher endorse the same theoretical argument that convinces, among many others, Shizgal and Conover. Here is how they gloss the argument:

What we need to do is to take into consideration many different attributes of each option (like color, size, taste, health benefits, our metabolic state, etc.), assess the value of each of the attributes, and combine all of these attributes into one coherent value representation that allows comparison with any other possible option. What we need, at least in principle, is a single common currency of valuation for comparing options of many different kinds” (Levy & Glimcher 2012: p1027).

I emphasise that for Levy and Glimcher this argument functions more as an encouragement to seek a certain kind of evidence, rather than a proof that it will be found in advance of empirical investigation. To repeat, the main point of their paper is to review the evidence that there is in fact a common currency.

Speaking very generally, this is done by relating aspects of patterns in choice activity to aspects of patterns of neural activity. The word ‘aspects’ is not idle in either case: It is not every feature of choice activity, or of the options offered, that is of interest, and the same goes for neural activity during choice. Again speaking very generally, it is the economic aspects of patterns of choice that are of interest. Those are the ones that reveal strength of relative preference between options – in both easy cases and tricky ones. Sometimes it will take a large number of choices before a pattern – let us call this a behavioural pattern – can be characterised in suitable detail. Similarly, the neural activity that is of interest will be that fraction that is specific to the ways in which choices vary economically, which involves removing from consideration the parts which are involved with other aspects of the task (such as sensory discrimination and motor control). Again, sometimes a large number of neural measurements will be needed before a neural pattern is detectable. But, with decent data of both kinds, it is possible to ask whether there is a neural pattern that corresponds appropriately to a behavioural one.

Prior neureconomic research in humans has found that a number of brain regions are commonly engaged by tasks with an economic aspect:

“Over the course of the past decade there have been a wealth of studies suggesting that activity in small number of brain areas encodes reward quantities during decision-making tasks. […] Indeed, there is now broad consensus in the neuroscience of decision-making community that reward magnitude is represented in a small number of well-identified areas” (Levy & Glimcher 2012: p1027).

The focus of this review is specifically on the claim that fMRI studies in humans support the view that the ventromedial prefrontal cortext/orbital frontal cortex (vmPFC/OFC) represent the value of many reward types on a common scale. (The claim is not that this region is unique in this respect, merely that it is one area that serves that function.)

The evidence for this claim comes, in the first place, from studies of three broad kinds:

  • In some, subjects were offered purely monetary rewards (variously received, chosen, anticipated, lost, etc.), and the relationship between behavioural and neural pattern encouraged the view that vmPFC/OFC activity was correlated with strength of preference.
  • In others, extending the first category, complicate the monetary rewards by introducing delays as well as probabilistic and ambiguous payments, or by varying the ways in which choice is expressed or values communicated. Again, it was found repeatedly that behavioural pattern was correlated with levels of vmPFC/OFC activity.
  • In the third category, at least one reward type is non-monetary. Levy and Glimcher survey the results of ten studies, in which the non-monetary rewards included social rewards (including reputation), gustatory rewards (food, drink), pain (avoided or suffered), and aesthetic rewards (viewing attractive faces). In these studies too, behavioural and neural patterns correlated in the vmPFC/OFC.

This last category is the most encouraging for the view that the vmPFC/OFC represents a common currency for all choices, because it finds the same general kind of association for reward types other than money. And we want a common currency to represent non-monetary rewards. It is important to see, though, why the third category of empirical result above falls short of being conclusive evidence:

“But in order to demonstrate that these representations exist in a single common currency appropriate for computing the trade-offs that guide choice one must also show that the activity-level in these areas is equivalent whenever subjects report that offers of two different kinds of rewards are equally desirable” (Levy & Glimcher 2012: p1032).

What Levy and Glimcher are saying is that evidence for a common currency requires a more demanding fit between behavioural and neural patterns. Orderly choice between reward types should be consistent with a determinate exchange rate. And if there is a neutrally represented common currency, then it should be possible to find correlates of that exchange rate in patterns of brain activity. According to Levy and Glimcher, two studies (as of their 2012) provide evidence of the required kind. In both of them the behavioural pattern allowed the exchange rate between two reward modalities to be determined (in one case between money and time spent viewing images of attractive faces, and in the other certain and risky choices over money and food). In both cases appropriate correlations were found, which is very encouraging news for defenders of a neural common currency in humans. Additional methodologically similar studies may strengthen the case.

Levy and Glimcher’s meta-analysis


Over and above the review of previous studies, Levy and Glimcher include a novel meta-analysis in their paper. To do this they extracted the ‘peak voxel’ – that is, the most active voxel – for value-related activity from each of the thirteen studies in their review, and marked them on a single brain template. The result shows considerable agreement across designs and reward types:

Peak (value related activity) voxels across the 13 studies in the meta-analysis. A 5mm cubed sphere has been drawn around each voxel to make the figure more legible. From page 1034 of paper. See also Table 1 on page 1031.

This close association, across quite divergent experimental designs (with different measures of preference) and with a variety of different reward types, is impressive and interesting. Levy and Glimcher express the upshot of all of this as follows:

“From these data we think that a single conclusion seems at this point relatively straightforward. There is indeed a small subregion in the vmPFC/OFC that tracks subjective value on a common currency appropriate for guiding choices between different kinds of rewards” (Levy & Glimcher 2012: p1035).

Conclusions


Levy and Glimcher are clear that not all questions here are settled. Even if all reward types are represented on a common scale in the vmPFC/OFC, there are open questions about how this fits into wider networks subserving choice, learning and action in the brain. One possible view of the outlines of those networks is provided in the following figure (from the paper):

Figure 6 from Levy & Glimcher (2012).

At most, then, the evidence surveyed by Levy and Glimcher supports the claim that the region numbered (1) – which is the vmPFC/OFC – represents values of all options in a common currency.

But even the evidence for that is, as they note, not (yet) entirely conclusive. Not all reward modalities have been examined, and the experiments surveyed all have money in common as at least one modality. Future work, consolidating and extending the evidence, should widen the range of reward types offered in choice situations, and also include choices where neither option is monetary.

In this, it seems fair to say, that some kinds of variation will be more interesting than others. The value of a study with some new reward type that is for all that broadly similar to ones already studied (for example by using access to social media instead of some other social reward) will be lower than one which involves a more radical departure from established results. From this perspective perhaps the most interesting candidates for study are those that are the subject of explicit rejections of common currency claims. Two specific categories seem to me to be worth noting:

  • Some people claim, for example, that some of the things that humans value they do so ‘lexically’ so that any amount of the one (no matter how small) is worth more than any amount of the other (no matter how large). Such preferences, if anyone really has them, would involve discontinuities in a common currency, and these discontinuities should be empirically detectable – both behaviourally and neurally. One recent study has claimed (in my view running far, far ahead of what the evidence it musters can really support) to have found neural evidence that some humans have‘sacred’ values.
  • In addition, some people claim that some values are incommensurable, in the sense that for some combinations of them, there is no fact of the matter about whether one or the other is better, or the two are of equal value. (See 'Incommensurable values' in the Stanford Encyclopedia of Philosophy.) This is a strong claim, and if true it should have both behavioural consequences and neural consequences. As far as I know nobody has seriously attempted to asses the incommensurable values claim in an approximately neuroeconomic experiment.


It would not be trivial to investigate either possibility. (I hope to write about both the difficulties and possibilities in the future, on this blog.) But the results would certainly be very interesting.

Full Abstract (of Levy & Glimcher 2012):


How do humans make choices between different types of rewards? Economists have long argued on theoretical grounds that humans typically make these choices as if the values of the options they consider have been mapped to a single common scale for comparison. Neuroimaging studies in humans have recently begun to suggest the existence of a small group of specific brain sites that appear to encode the subjective values of different types of rewards on a neural common scale, almost exactly as predicted by theory. We have conducted a meta analysis using data from thirteen different functional magnetic resonance imaging studies published in recent years and we show that the principle brain area associated with this common representation is a subregion of the ventromedial prefrontal cortex (vmPFC)/orbitofrontal cortex (OFC). The data available today suggest that this common valuation path is a core system that participates in day-to-day decision making suggesting both a neurobiological foundation for standard economic theory and a tool for measuring preferences neurobiologically.
Perhaps even more exciting is the possibility that our emerging understanding of the neural mechanisms for valuation and choice may provide fundamental insights into pathological choice behaviors like addiction, obesity and gambling.

Reference

Levy, D., & Glimcher, P. (2012). The root of all value: a neural common currency for choice Current Opinion in Neurobiology, 22 (6), 1027-1038 DOI: 10.1016/j.conb.2012.06.001

Link to PDF on Glimcher Lab Website.

ResearchBlogging.org
  





[1] There’s a possible complication that I’m skipping over here. Even if we convert all choices into a common scale, we might apply some decision rule other than ‘choose the best’. We might, for example, allocate our actions (over time) between several options in proportion to their values.
[2] There’s another complication that I’m skipping here. There is room for disagreement over whether human choices are orderly at all (they’re clearly partly orderly). And among those who agree that there’s some kind of order, there’s disagreement over how best to describe the order. Levy and Glimcher are, furthermore, fully aware of the variety of models of the order in human choice.

No comments:

Post a Comment