Paul Glimcher - Video Lecture

Postings here have been a bit thin and slow lately. I've been teaching a lot. Work continues when it can, though. I'm working on adding something about neuroeconomics. In the interim, here's a very fine talk by Paul Glimcher of NYU, on the subject of "Neurobiological Foundations of Economic Choice". Glimcher has extraordinarily deep understanding of a variety of economic theories, and of brain science. This is a fairly accessible, and very clear and lucid, exposition of the outlines of his research programme. It's really well worth watching, and a useful introduction/companion to Glimcher's terrific 2011 book Foundations of Neuroeconomic Analysis.

Elsewhere on this blog you can find one discussion of a paper by Glimcher (and Levy) reviewing and extending some of the neural evidence for a common currency: The Root of All Value.

The root of all value: a neural common currency for choice

Levy, D., & Glimcher, P. (2012). The root of all value: a neural common currency for choice Current Opinion in Neurobiology, 22 (6), 1027-1038 DOI: 10.1016/j.conb.2012.06.001

Introduction

Humans make enormous numbers of choices. Some of them are – at least in principle – relatively easy, for example choosing between one hundred, and ten thousand dollars or between one sip of water and ten sips. But others are more tricky, for example choosing between spending some free time watching a movie and playing tennis. In the latter case the two options vary in more than one way. Consequently a simple ‘more is better’ rule doesn’t determine what is best.

Given that we do, however, make large numbers of choices, including these superficially tricky ones, often without noticeable effort, we might wonder what it is that we do when we make them.

One idea, with a fairly long history, is that we do it by converting choices, including tricky ones, into easy ones, by expressing the worth of the different options in some abstract and general scale that is used for all choices. This more abstract scale is, of course, often referred to as a ‘common currency’ but might also be called ‘utility’ or ‘reward’, or ‘pleasure’ or just ‘value’. Once this conversion has been done, the idea is that all choices are simple ones: now we can take the one worth the most in the common currency.[1]

According to some, if our choices, including the tricky ones, are consistent, then we must be converting the options into values in a common currency. As discussed elsewhere on this blog, Shizgal and Conover have argued that:

“In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension” (Shizgal & Conover 1996).

This argument need not be decisive. That is, it is possible that some other mechanism which did not involve an internal common currency could explain ‘orderly choice’. So one question that arises here is a theoretical one, about what range of mechanisms could produce orderly choice. Perhaps the only viable options are mechanisms involving common currency. Independent of this theoretical question, is an empirical one: How, in fact, do humans achieve orderly choice?[2] Levy and Glimcher survey some evidence, supplemented by a new meta-analysis of previous research, to defend the conclusion that whether or not they have to, human brains do in fact implement a common currency.

Levy and Glimcher’s review of existing evidence

Levy and Glimcher endorse the same theoretical argument that convinces, among many others, Shizgal and Conover. Here is how they gloss the argument:

“What we need to do is to take into consideration many different attributes of each option (like color, size, taste, health benefits, our metabolic state, etc.), assess the value of each of the attributes, and combine all of these attributes into one coherent value representation that allows comparison with any other possible option. What we need, at least in principle, is a single common currency of valuation for comparing options of many different kinds” (Levy & Glimcher 2012: p1027).

I emphasise that for Levy and Glimcher this argument functions more as an encouragement to seek a certain kind of evidence, rather than a proof that it will be found in advance of empirical investigation. To repeat, the main point of their paper is to review the evidence that there is in fact a common currency.

Speaking very generally, this is done by relating aspects of patterns in choice activity to aspects of patterns of neural activity. The word ‘aspects’ is not idle in either case: It is not every feature of choice activity, or of the options offered, that is of interest, and the same goes for neural activity during choice. Again speaking very generally, it is the economic aspects of patterns of choice that are of interest. Those are the ones that reveal strength of relative preference between options – in both easy cases and tricky ones. Sometimes it will take a large number of choices before a pattern – let us call this a behavioural pattern – can be characterised in suitable detail. Similarly, the neural activity that is of interest will be that fraction that is specific to the ways in which choices vary economically, which involves removing from consideration the parts which are involved with other aspects of the task (such as sensory discrimination and motor control). Again, sometimes a large number of neural measurements will be needed before a neural pattern is detectable. But, with decent data of both kinds, it is possible to ask whether there is a neural pattern that corresponds appropriately to a behavioural one.

Prior neureconomic research in humans has found that a number of brain regions are commonly engaged by tasks with an economic aspect:

“Over the course of the past decade there have been a wealth of studies suggesting that activity in small number of brain areas encodes reward quantities during decision-making tasks. […] Indeed, there is now broad consensus in the neuroscience of decision-making community that reward magnitude is represented in a small number of well-identified areas” (Levy & Glimcher 2012: p1027).

The focus of this review is specifically on the claim that fMRI studies in humans support the view that the ventromedial prefrontal cortext/orbital frontal cortex (vmPFC/OFC) represent the value of many reward types on a common scale. (The claim is not that this region is unique in this respect, merely that it is one area that serves that function.)

The evidence for this claim comes, in the first place, from studies of three broad kinds:

• In some, subjects were offered purely monetary rewards (variously received, chosen, anticipated, lost, etc.), and the relationship between behavioural and neural pattern encouraged the view that vmPFC/OFC activity was correlated with strength of preference.
• In others, extending the first category, complicate the monetary rewards by introducing delays as well as probabilistic and ambiguous payments, or by varying the ways in which choice is expressed or values communicated. Again, it was found repeatedly that behavioural pattern was correlated with levels of vmPFC/OFC activity.
• In the third category, at least one reward type is non-monetary. Levy and Glimcher survey the results of ten studies, in which the non-monetary rewards included social rewards (including reputation), gustatory rewards (food, drink), pain (avoided or suffered), and aesthetic rewards (viewing attractive faces). In these studies too, behavioural and neural patterns correlated in the vmPFC/OFC.

This last category is the most encouraging for the view that the vmPFC/OFC represents a common currency for all choices, because it finds the same general kind of association for reward types other than money. And we want a common currency to represent non-monetary rewards. It is important to see, though, why the third category of empirical result above falls short of being conclusive evidence:

“But in order to demonstrate that these representations exist in a single common currency appropriate for computing the trade-offs that guide choice one must also show that the activity-level in these areas is equivalent whenever subjects report that offers of two different kinds of rewards are equally desirable” (Levy & Glimcher 2012: p1032).

What Levy and Glimcher are saying is that evidence for a common currency requires a more demanding fit between behavioural and neural patterns. Orderly choice between reward types should be consistent with a determinate exchange rate. And if there is a neutrally represented common currency, then it should be possible to find correlates of that exchange rate in patterns of brain activity. According to Levy and Glimcher, two studies (as of their 2012) provide evidence of the required kind. In both of them the behavioural pattern allowed the exchange rate between two reward modalities to be determined (in one case between money and time spent viewing images of attractive faces, and in the other certain and risky choices over money and food). In both cases appropriate correlations were found, which is very encouraging news for defenders of a neural common currency in humans. Additional methodologically similar studies may strengthen the case.

Levy and Glimcher’s meta-analysis

Over and above the review of previous studies, Levy and Glimcher include a novel meta-analysis in their paper. To do this they extracted the ‘peak voxel’ – that is, the most active voxel – for value-related activity from each of the thirteen studies in their review, and marked them on a single brain template. The result shows considerable agreement across designs and reward types:

Peak (value related activity) voxels across the 13 studies in the meta-analysis. A 5mm cubed sphere has been drawn around each voxel to make the figure more legible. From page 1034 of paper. See also Table 1 on page 1031.

This close association, across quite divergent experimental designs (with different measures of preference) and with a variety of different reward types, is impressive and interesting. Levy and Glimcher express the upshot of all of this as follows:

“From these data we think that a single conclusion seems at this point relatively straightforward. There is indeed a small subregion in the vmPFC/OFC that tracks subjective value on a common currency appropriate for guiding choices between different kinds of rewards” (Levy & Glimcher 2012: p1035).

Conclusions

Levy and Glimcher are clear that not all questions here are settled. Even if all reward types are represented on a common scale in the vmPFC/OFC, there are open questions about how this fits into wider networks subserving choice, learning and action in the brain. One possible view of the outlines of those networks is provided in the following figure (from the paper):

Figure 6 from Levy & Glimcher (2012).

At most, then, the evidence surveyed by Levy and Glimcher supports the claim that the region numbered (1) – which is the vmPFC/OFC – represents values of all options in a common currency.

But even the evidence for that is, as they note, not (yet) entirely conclusive. Not all reward modalities have been examined, and the experiments surveyed all have money in common as at least one modality. Future work, consolidating and extending the evidence, should widen the range of reward types offered in choice situations, and also include choices where neither option is monetary.

In this, it seems fair to say, that some kinds of variation will be more interesting than others. The value of a study with some new reward type that is for all that broadly similar to ones already studied (for example by using access to social media instead of some other social reward) will be lower than one which involves a more radical departure from established results. From this perspective perhaps the most interesting candidates for study are those that are the subject of explicit rejections of common currency claims. Two specific categories seem to me to be worth noting:

• Some people claim, for example, that some of the things that humans value they do so ‘lexically’ so that any amount of the one (no matter how small) is worth more than any amount of the other (no matter how large). Such preferences, if anyone really has them, would involve discontinuities in a common currency, and these discontinuities should be empirically detectable – both behaviourally and neurally. One recent study has claimed (in my view running far, far ahead of what the evidence it musters can really support) to have found neural evidence that some humans have‘sacred’ values.
• In addition, some people claim that some values are incommensurable, in the sense that for some combinations of them, there is no fact of the matter about whether one or the other is better, or the two are of equal value. (See 'Incommensurable values' in the Stanford Encyclopedia of Philosophy.) This is a strong claim, and if true it should have both behavioural consequences and neural consequences. As far as I know nobody has seriously attempted to asses the incommensurable values claim in an approximately neuroeconomic experiment.

It would not be trivial to investigate either possibility. (I hope to write about both the difficulties and possibilities in the future, on this blog.) But the results would certainly be very interesting.

Full Abstract (of Levy & Glimcher 2012):

How do humans make choices between different types of rewards? Economists have long argued on theoretical grounds that humans typically make these choices as if the values of the options they consider have been mapped to a single common scale for comparison. Neuroimaging studies in humans have recently begun to suggest the existence of a small group of specific brain sites that appear to encode the subjective values of different types of rewards on a neural common scale, almost exactly as predicted by theory. We have conducted a meta analysis using data from thirteen different functional magnetic resonance imaging studies published in recent years and we show that the principle brain area associated with this common representation is a subregion of the ventromedial prefrontal cortex (vmPFC)/orbitofrontal cortex (OFC). The data available today suggest that this common valuation path is a core system that participates in day-to-day decision making suggesting both a neurobiological foundation for standard economic theory and a tool for measuring preferences neurobiologically.

Perhaps even more exciting is the possibility that our emerging understanding of the neural mechanisms for valuation and choice may provide fundamental insights into pathological choice behaviors like addiction, obesity and gambling.

Reference

Levy, D., & Glimcher, P. (2012). The root of all value: a neural common currency for choice Current Opinion in Neurobiology, 22 (6), 1027-1038 DOI: 10.1016/j.conb.2012.06.001

Link to PDF on Glimcher Lab Website.

  

---

[1] There’s a possible complication that I’m skipping over here. Even if we convert all choices into a common scale, we might apply some decision rule other than ‘choose the best’. We might, for example, allocate our actions (over time) between several options in proportion to their values.

[2] There’s another complication that I’m skipping here. There is room for disagreement over whether human choices are orderly at all (they’re clearly partly orderly). And among those who agree that there’s some kind of order, there’s disagreement over how best to describe the order. Levy and Glimcher are, furthermore, fully aware of the variety of models of the order in human choice.

Shizgal and Conover’s ‘Orderly choice’ argument and evidence

One of the clearest statements of an argument for a common currency thesis is found early in Peter Shizgal and Kent Conover’s 1996 paper ‘On the neural computation of utility.’ This is a really important paper for the common currency topic, and should be required reading for anyone in the area. It doesn’t only state a version of one of the key arguments, it also provides an exemplary kind of empirical evidence.

Here is how they state the argument:

In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension (Shizgal & Conover 1996).

The second sentence can be slightly reformulated (making one implicit premise explicit) as follows:

Premise 1: Orderly choice is possible.

Premise 2: For orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension.

So

Conclusion: The utility of all competing resources is represented on a single, common dimension.

This certainly looks like a valid argument. But what does it mean?

Well, the suggestion is clearly that (a) the rewards available to an organism might vary in lots of ways, across multiple dimensions (consider sex, rest, food of different kinds, grooming, water, avoiding predators, feeding young, etc.), and (b) making ‘orderly’ decisions between available rewards, the rewards all need to have a simple one-dimensional representation. (See The (very) Basic Big Idea on this site.)

Shizgal and Conover don’t claim that all choice is in fact orderly. But they clearly intend to say that when it is orderly, a common dimension of comparison is required. But what is it for choice to be orderly? Looking at Shizgal and Conover’s experiments will help clarify this.

The research Shizgal and Conover report on concerns ‘brain self-reward’. A famous paper by Olds and Milner (1954) reported that rats would work, including learning novel behaviours, for no more reinforcement than pulses of electrical stimulation to a part of the medial forebrain where an electrode terminated. Some early reports of brain-self reward – as it came to be called, or (BSR) for short – focused on ways in which it was unusual. The popular imagination got excited by the rat that pressed its lever nearly continually for around 20 days at about one press every two seconds (Valenstein & Beer 1964), or cases where food was apparently ignored over brain self-reward (Routtenberg & Lindy 1965). Shizgal and Conover sought, in part, to demonstrate that brain self-reward is a reinforcer like any other. In a series of experiments they had rats chose between trains of BSR pulses and infusions of sucrose solution.

The experimental design infused the juice directly into the rats’ mouths to give the sucrose solution a key property of BSR, which is that a single action leads to both procurement and consumption. In addition the swallowed fruit juice was drained away, reducing postingestive effects so that the infusions shared another property of BSR, which is absence of satiation.

Schematic illustration of experiment 1 from preprint version of the paper.

Shizgal and Conover manipulated the strength of reinforcement to each modality (by changing the duration or number of pulses in a train of BSR, or the size of the sucrose infusion) and measured the relative allocation of lever presses to each. Among other things they found that each individual reward modality was preferred over nothing, more of it preferred to less, and that increasing the opportunity cost of either (by increasing the reward available from the other lever) led to less consumption of that reward.

In a variation on the experiment they manipulated the magnitude of a BSR-only reward, while the alternative reward was a constant combination of BSR and sucrose solution. In this condition it took more BSR to make the rat forgo the compound reward than had been necessary for the sucrose part of the reward alone.

Schematic illustration of experiment 2 from preprint version of the paper.

Their conclusion is that the results of the first experiment imply, “that on a given trial, the rat selected the alternative that registered a larger value in a common system of measurement”, rather than following a categorical rule (such as ‘whenever the size of one reward is above some threshold, choose it’).

Regarding the second experiment with the combination rewards, they say it implies “that the electrical stimulation and the sucrose were subjected to a common evaluation”.

(The paper describes two further experiments, complementary to those I’ve just described. I’ll discuss them in a future posting.)

So, one way of glossing this is that Shizgal and Conover demonstrate that rat choices between BSR and sucrose solution is orderly in two senses:

• Choices between modalities are quantitatively sensitive to opportunity cost.
• Choices between rewards in one modality and two-modality combination rewards are similarly sensitive to the combined opportunity cost.

These results are important. They show (in terminology that I’ve described elsewhere on this site, but which isn’t used by Shizgal and Conover) that at least some rat choices have a certain patterning, conforming to an ultimate common currency.

Shizgal and Conover also clearly intend the conclusion that the results about an ultimate common currency support the view that there is a proximal one:

“We speculate that this common ability arises from a common action of the gustatory and electrical stimuli on a neural system that determines goal selection by signaling the utility of competing goals.”

The other two experiments, and their ongoing research programme, provide further support for this speculation. But that’s all I’ve got time for now.

References

Olds, J. & Milner, P. 1954. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47, pp. 419-427.

Routtenberg, A. & Lindy, J. 1965. Effects of the availability of rewarding septal and hypothalamic stimulation on bar pressing for food under conditions of deprivation. Journal of Comparative and Physiological Psychology, 60, pp. 158-161.

Shizgal, P. & Conover, K. 1998. On the neural computation of utility, Current Directions in Psychological Science, 5(2), pp. 37-43. [Publisher’s site –may be behind a paywall] [Preprint version at CogPrints] [Google Scholar Citations]

Valenstein, E.S. & Beer, B. 1964. Continuous opportunity for reinforcing brain stimulation. Journal for the Experimental Analysis of Behavior, 7, pp. 183-184.

Forthcoming attractions

Discussion of Shizgal (1999).

See also

Currencies can be 'ultimate' or 'proximal'

Shizgal, P., & Conover, K. (1996). On the Neural Computation of Utility. Current Directions in Psychological Science, 5 (2), 37-43 DOI: 10.1111/1467-8721.ep10772715