Friday, July 11, 2014

Reward Prediction Error Signals are Meta-Representational

“Research on humans and other animals has produced an impressive body of converging evidence that midbrain dopamine neurons produce a reward prediction error signal (RPE) that is causally involved in choice behaviour […]. RPEs are found in humans, primates, rodents and perhaps even insects […]. This paper argues that RPEs carry metarepresentational contents.” (Shea 2014.)

Introduction


Scientifically interesting questions about common currencies fall largely into three groups:
  1. About whether the behaviour of some type of agent (including people) is consistent with a single preference or value ordering? (Related questions here concern what kinds and degrees of order there are, and what theories - especially biological and economic ones - make sense of the order.
  2. About whether the processes that produce the behaviour of some type of agent (including people) consult or ‘consume’ a representation of values or preferences?
  3. Finally, about the relationships between answers to the questions in groups (1) and (2).

Questions in the second group are about the existence and nature of internal, cognitive, representations of value. If there are comprehensive internal value representations, then they are, in terms discussed elsewhere on this blog, instances of proximal common currencies. The point of a proximal common currency is that one way, at least, to produce behaviour that is sensitive to the values (whether understood as reward, utility, pleasure,…) of the results of actions, is to represent those values internally. Then the behaviour selection process can consult the value representation, and the sensitivity of behaviour to value is both made possible and explained.

I recently ran into an elegant and interesting paper by Nicholas Shea, at the time at Oxford but now back at King’s College, London. The paper, like it says on the box, argues that reward prediction error signals are meta-representational. On this blog so far, I’ve mostly left discussion of representation as a topic in a black box, and focused on the arguments about whether or not there is, or must be, a represented common currency in order to explain order in behaviour. But that’s not because of my thinking that representation - especially of values - wasn’t an interesting topic.

Representation has, of course, an enduring fascination for the philosophy of cognitive science. Back in the olden days this fascination was partly expressed in big picture yelling about whether cognition in general was essentially representational, or was possible without representations (e.g. Rodney Brooks’ famous 1991 paper Intelligence without Representation [PDF]).

There’s also a tradition of more interesting - to my mind, anyway, - work focused on developing detailed theories of various kinds of representation, and working out how they might apply to far more specific cognitive phenomena and cognitive mechanisms. Shea’s paper is a fine example of that latter approach. It reminds me a bit of Dan Lloyd’s wonderful book Simple Minds (1989), in focusing on simple systems that, because they can be exhaustively described, can explicitly be shown to instantiate representations by the lights of a specific theory.

The topic of value, or reward, representation cries out for serious philosophical treatment. Neuroscientists explicitly and regularly make claims about what this or that brain network, or brain activity, represents, but rarely develop or invoke specific theories of representation while they do so. Most of the work goes on experimental design and execution aimed at showing suitably determinate relationships between the brain network or activity and something else that can be independently measured. A theory of representation can help work out how to relate the discoveries from neuroscience - in this case including neuroeconomics - to theories from other relevant areas, and to theories at other scales of granularity including, perhaps and eventually, whole agent theories about the rational explanation of actions in terms of desires or preferences (and beliefs).

So here’s a sketch of the highlights - given my rather specific interests - of Shea’s argument for the thesis that reward prediction error signals (RPEs) are meta-representational. To turn this into something a bit more specific and comprehensible, we need to get clear on what some of those notions mean. (All quotations are, unless otherwise noted, from Shea’s paper.)

Metarepresentation


A representation says something about something. For example, a price tag on some object says of it that it can be bought for some sum of money.

Sometimes what is represented is itself a representation. Only some of these representations are meta-representations, though. For example, I might point to a price tag and say of it that the tag is printed with a dot-matrix printer. Then I’m saying something about the form of a representation, but not the content. If, instead, I said of the very same price tag that “that’s more than I was planning on spending”, or “that’s less than it was last week”, I’d be saying saying something about the content of the representation. In this sort of case I’m metarepresenting.

The kinds of representation, and metarepresentation, we’re concerned with here are non-conceptual. This means that they don’t require concept possession. If that sounds perplexing, cling to this: What’s important for present purposes about a non-conceptual representation is that it has correctness, or satisfaction, conditions. So we establish that something is a representation by specifying these conditions. In the case of metarepresentations, Shea adopts the following criterion. Consider a putative metarepresentation (‘M’):

“ […] M’s having a correctness condition or satisfaction condition that concerns the content of another representation is taken to be a sufficient condition for M to be a metarepresentation. That is a reasonably stringent test. It is not enough that M concerns another representation. A representational property must figure in M’s correctness condition or satisfaction condition.”

Reward Prediction Errors (RPEs)


Very generally and informally, a prediction error occurs in a class of ‘temporal difference’ learning algorithms. It is the difference between an expected or predicted and an actual value, and is used to modify the expectations in future cases. The expectations are averages, and the details of the modification vary from case to case, but the general idea is intuitively clear: If the expected value was too low, raise it a little; if too high, lower it a little. In the case of reward prediction error, or RPE, the specific thing predicted is reward. But the fact that it’s reward that is predicted isn’t an intrinsic property of the algorithm, it depends on how an implementation of the algorithm is hooked up (to the world, and other bits of an information processing system).

As Shea notes, temporal difference learning and prediction errors came out of computer science, and to begin with weren’t intended to say much about brains or animals:

“But then Wolfram Schultz and colleagues discovered that midbrain dopamine neurons broadcast an RPE signal […]. Relative to a background tonic level of firing, there is a transitory phasic increase when a unpredicted reward is delivered, and a phasic decrease in firing when a predicted reward is not delivered. This finding brought the computational modelling rapidly back into contact with real psychology, galvanised the cognitive neuroscience of decision-making and launched the science of neuroeconomics.”

To get an idea of what Schultz and colleagues discovered, consider the figure below,  adapted from Schultz, Dayan & Montague (1997). I think it’s fair now to call it famous - most people with anything more than passing knowledge of neuroeconomics will recognise it and be able to explain it. Apologies if you’re already familiar with it:

Figure from Schultz, Dayan & Montague (1997)

In each of the three panels, time is represented horizontally, with earlier times to the left of later ones. The rest of the each panel represents the activity of some dopamine neurons, with dots representing spikes, aggregated into a bar graph along the top. ‘CS’ is a point in time at which a conditioned stimulus is presented to an awake monkey (bottom two plots only).  R’ is a point at which a reward (such as squirt of fruit juice into the mouth) is delivered to the monkey (top two plots only). The CS isn’t rewarding in itself (it’s typically a sound, or a flash of light). But R is rewarding.

The data in all three plots is from a subject familiar with CS preceding R by around one second.

In the top plot, there is no CS, but R is still delivered. In that case there is a flare up in neural activity shortly after R. In the middle plot the presentation of CS is followed by an increase in dopamine neuron activity, but those neurons show no response to the subsequent delivery of R. In the bottom image, CS is the same and receives the same neural response, but R is absent, and in that case there’s a drop in activity when R was expected.

These experiments were decisive against the notion that dopamine is a ‘pleasure’ molecule, or otherwise primarily linked to experienced or occurrent reward. (That was a popular view in science for a while, and is still sometimes asserted in night clubs.) If dopamine was simply linked to reward, then it wouldn’t be associated with unexpected unrewarding CS events, and it would be associated with rewards, even if they were expected.

The point is not that dopamine is unrelated to reward, it’s rather that it’s involved in reward prediction. More specifically, the consensus is now, it is a reward prediction error, signalling when there’s more or less reward (or reward cue) than expected at any time. That’s why there’s more of it for unexpected R and unexpected CS, and less of it for unexpected absence of R.

There’s been much more detailed and specific research into the neural implementation of prediction errors and reward prediction errors since the work just described. There are also some outstanding questions and controversies over some issues of detail. Nonetheless, the general point that some brains (including human ones) implement temporal difference learning, and that reward learning involves RPEs is very widely accepted. As Shea puts it: “… the current state of the art is as strong a scientific consensus as a philosopher could possibly hope for.”

Shea’s paper (sections 2 and 3) gives a really clear and useful account of reward prediction errors, and a detailed explanation of a simplified model in which reward prediction error is immediate (as opposed to delayed, for example if rewards come after a series of actions). In the model that Shea describes the predicted rewards are associated with actions. This is important. The monkey subjects in the experiment described above didn’t have to do anything. Not all rewards are contingent on action, and many important experiments about reward expectancy and learning don’t require subjects to make choices. The same class of learning algorithms can, though, be applied to action selection cases. Then the expected reward is contingent on the actually selected action, and any prediction error modifies the reward expectations for that action.  

Putting the bits Together


Now that we have an idea of what metarepresentation is, and also what a reward prediction error is, it shouldn’t be difficult to see how they relate.

The reward prediction error is metarepresentational in the sense that it represents something about the content of one or more other representations. It says whether actual incoming reward is greater, or smaller, or equal to what was expected. Put slightly differently, it says whether the expectation was correct, or too low, or too high. Either way, the  RPE is about a relationship between the content of some other representations (of expected reward, and actual reward).

Here’s Shea:

“[…] RPEs have metarepresentational contents. They have both indicative and imperative contents (they are so-called pushmi-pullyus). The indicative content is that the content of another representation—the agent’s (first-order) representation of the reward that will be delivered on average for performing a given action—differs from the current feedback, and by how much. The imperative content instructs that it be revised upwards or downwards proportionately.”

So RPEs non-conceptually metarepresent content about the content of other representations, and the RPEs are processed as instructions by other cognitive systems, in ways that lead to modifying the content of the basic representations that the RPEs are about.

This all seems spot on to me. In fact, once an appropriate account of metarepresentation, and a careful description of RPEs are laid out side by side, as Shea does in the first few sections of his paper, the conclusion he is urging seems very difficult to resist. (Later stages of Shea’s paper are partly devoted to considering alternative approaches and deflationary strategies. I’m not going to attempt an account of those sections here. The whole paper is, though, well worth careful attention.)

So what?


I think anyone interested in making philosophical sense of neuroeconomics should read Shea’s paper. I’m going to close with a few specific remarks about how it’s relevant to my preoccupation with common currencies.

When scientists (and some philosophers) refer to a proximal common currency, they often focus on the structure of the common currency as a scale. For example, Levy and Glimcher (discussed in an earlier posting say, when discussing how the things we choose between vary along many different dimensions, and don’t always have the same dimensions in common:

“What we need to do is to take into consideration many different attributes of each option (like color, size, taste, health benefits, our metabolic state, etc.), assess the value of each of the attributes, and combine all of these attributes into one coherent value representation that allows comparison with any other possible option. What we need, at least in principle, is a single common currency of valuation for comparing options of many different kinds” (Levy & Glimcher 2012: p1027).

A fairly strong claim is being here: The brain must encode or represent options (including actions) that might differ in a wide range of modalities on a single unidimensional scale. And a lot of work goes into describing properties of the scale, such as whether in this or that brain process it represents expected, or relative expected utility, or what resolution it has.

But we should also be thinking about the representational structure of the value scale itself. This is an interesting topic even if it doesn’t constitute a completely common, or completely consistent, currency. (Even weakened or approximate theses about proximal common currencies are theses about value representation.) The value representation can’t just be a scale, it has - somehow - to be indexed to representations of what it is about. Just as RPEs are indexed to specific expectations, so the values have to be indexed to actions, and perceptual cues, and other information.   

Speaking very speculatively, it seems unlikely that a represented value scale would be anything like a simple ordered list. It might be tempting to envisage an image of a great big ruler with outcomes inscribed next to their corresponding number of ‘hedons’, or ‘utiles’. But such an image is hardly credible. The list would prohibitively large if it actually stored expected values for all discriminable quantities of all consumption types (one beer, two beers, three beers; one dollar, two dollars, three dollars, let alone the sips and cents). It’s more likely, then, for a value representation to be encoded in a mixture of procedural and model-like ways, that allow reward expectancies to be generated in response to specific option sets, including drawing on our capacities to simulate and imagine. But in that sort of case the details of credit assignment, and what to change in response to RPEs, would be more complicated. Perhaps surprisingly, it’s clearer that RPEs are metarepresentational than it is what the representational structure of the reward expectancies themselves are.

(ASIDE: Some, including Andy Clark, have recently support the view that the whole brain is in the business of using prediction errors to refine expectations, ultimately aiming for a state where nothing is surprising. In that case many prediction errors would be about processes other than reward, including perception and motor control. The thought that reducing prediction error is in some sense the aim of cognition rather than a means leads to some rather odd worrying about why creatures with brains don’t just seek out and stay in dark rooms. (In dark rooms there are no surprises, and so no prediction errors.) I hope to write about the ‘dark room problem’ here in the future. For more, including commentaries, see ‘Whatever Next?’ (may be behind a paywall). Commentaries continued at the Open Access Frontiers in Theoretical and Philosophical Psychology including my own commentary.)

Related postings on this site:


References

Brooks, R.A. (1991). Intelligence without representation, Artificial Intelligence, 47: 139–159.
Clark, A. (2013) Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3), pp181-204.
Lloyd, D. (1989). Simple Minds. Cambridge, MA: MIT Press.
Shea, N. (2014). Reward Prediction Error Signals are Meta-Representational, Noûs, 48(2): 314-341. [LINK]
Schultz, W., Dayan, P., and Montague, R. (1997), ‘A neural substrate of prediction and reward’, Science, 275 (5306), 1593.

Research Blogging Citation:


Shea, N. (2014). Reward Prediction Error Signals are Meta-Representational Noûs, 48 (2), 314-341 DOI: 10.1111/j.1468-0068.2012.00863.x ResearchBlogging.org

Friday, June 27, 2014

Stephen C. Stearns - Economic Decisions for the Foraging Individual

Here's another useful video lecture. It's from the Open Yale Course 'EB122: Principles of Ecology, Evolution and Behavior', taught by Stephen C. Stearns, Edward P. Bass Professor of Ecology and Evolutionary Biology.

Here is his overview of the lecture:
There are several ways to examine the behaviors of organisms when they forage or hunt for food or mates. These behaviors become more complex in higher organisms, such as primates and whales, which can hunt in groups. Foragers and hunters have been shown to examine the marginal cost and marginal benefit of continuing an action and then adjust their behaviors accordingly. They are also able to handle risk by hoarding resources.
Here is the course home page on Yale Open Courses.

This material is of interest because of the importance of economic analyses of decisions by living animals, and foraging is an area where there's been considerable progress. Behavioural ecology is one of the areas in science where you'll regularly run into talk of common currencies. Behavioural ecologists are sometimes relatively agnostic about the mechanisms producing behaviour, but sometimes intensely interested in them. (The same course includes a useful session - number 10 - on genomic conflict.)


Tuesday, June 24, 2014

Ultimate currencies can be subjective or evolutionary

Ultimate currencies can be evolutionary or subjective

Claims about common currencies are offered as explanations of one or both of two putative facts:
  • The currency represents the fundamental principle in some pattern in the choices made by some agent. These are ultimate currencies. OR
  • The currency is a psychologically real characteristic of the processes by which choice is produced. These are proximal currencies.
Any single ranking of options on a scale such that the behaviour of an agent can be described as – perhaps approximately – consistent with that ordering counts as an ‘ultimate’ common currency. An ultimate currency relates values to options, or to what selecting those options achieves or perhaps have the function of achieving. It is easy to imagine possible instantiations of an ultimate currency (‘all of Jim’s actions are efficiently ordered to contribute to the greater glory of the Flying Spaghetti Monster’) but among the scientifically interesting forms of consistency, two families stand out. One of these relate to fitness, and the other to some of other form of utility.

Here’s Don Ross describing Paul Samuelson’s work on what came to be called revealed preference theory:
“Paul Samuelson (1938) […] set out to define utility in such a way that it becomes a purely technical concept. Since Samuelson's re-definition became standard in the 1950s, when we say that an agent acts so as to maximize her utility, we mean by ‘utility’ simply whatever it is that the agent's behavior suggests her to consistently act so as to make more probable.” (Ross, 2005)
It would be difficult to find a clearer statement of the basic idea of an ultimate common currency. As I said, the two main scientifically interesting variants of claims about ultimate currencies relate to fitness, or to utility.

I therefore say that an ultimate currency can be evolutionary or subjective. Here I’m knowingly, albeit slightly, departing from standard usage, insofar as the usual way of distinguishing proximal from ultimate has the latter taken to be synonymous with fitness promoting. (The standard sense is partly preserved here, because the values in an evolutionary ultimate currency are a function of contribution to fitness.) A guiding presumption of behavioural ecology is that behavioural dispositions make contributions to fitness, and that to the extent that the dispositions have a heritable basis, selection will tend to drive them towards making (constrained) optimal contributions. The following statement by McNamara and Houston is an exemplary (and frequently quoted) assertion about an evolutionary ultimate currency:
“Any attempt to understand behavior in terms of the evolutionary advantage that it might confer has to find a "common currency" for comparing the costs and benefits of various alternative courses of action” (McNamara and Houston 1986: 358).
There are compelling reasons for thinking that natural selection will operate on at least some behavioural tendencies, and so a strong general justification for the project of behavioural ecology. Nonetheless, the question of whether the behaviour allocation of the individuals in any particular species does indeed tend to optimise fitness (or would have in historical selective environments) is an empirical one. Behavioural ecologists have studied many species and types of behaviour and achieved striking successes in restricted domains such as foraging and mate selection. These successes have often relied on focusing on a simplified and more empirically tractable currency such as net rate of calorie intake (in the case of foraging), or the health of the selected mate (in mate selection). These proxy currencies plausibly contribute to fitness. Even so, success relating behaviour patterns to the proxies falls short of establishing relationships between all behavioural dispositions and overall fitness in any species. Two of the most serious shortfalls are in the area of relative allocation between significantly different modalities (such as calorie intake versus pursuit of mating opportunities versus predator avoidance) and variation in allocation over extended periods of time, including the full life-history.

(I note as an aside here that evolutionary psychologists tend to share the general theoretical orientation of behavioural ecologists, but not to make claims about common currencies. I’m not sure why this is. Perhaps they’re mostly more interested in attempting to establish the existence of biases or preferences that fit with their theoretical orientation, and being interested in their relative strengths is the sort of thing that comes later. Those evolutionary psychologists who are most committed to modularity and opposed to central systems, though, seem to be committed - if only by implication - to denying the possibility of a proximal common currency.)

Anyway, enough about ultimate common currencies.

A subjective ultimate currency, in contrast, attributes values that are a function of the revealed preferences or inferred utilities of the individual agent, without requiring any relationship to fitness. Ross’s description of Samuelson, quoted above, states the key idea. The paradigmatic sciences of subjective ultimate currencies are microeconomics (with variously regimented notions of utility functions revealed through consumption) and behavioural psychology (where strength of reinforcement is defined in terms of effect on patterns of behaviour allocation). Evolutionary ultimate currencies, then, have stronger empirical conditions than subjective ones, because the latter require ‘mere’ consistency in behaviour, whereas the former require consistency in contributing to fitness.

The distinction between evolutionary and subjective ultimate currencies, as I’ve sketched it here, conceals considerable technical detail. There are different and competing fitness concepts, debates over the level (especially gene or individual) at which selection operates, and differing positions over the correct (if any) individuation principles for genes, genomes, species and other relevant categories. There are also competing utility concepts, offering different explanations of the same empirical data.

Related postings:



References:


McNamara, J.M. and Houston, A.I. (1986) The Common Currency for Behavioral Decisions, The American Naturalist, 127(3), pp358-378.

Ross, D. (2005). Economic Theory and Cognitive Science, Volume One: Microexplanation. Cambridge, MA: MIT Press.

Monday, June 23, 2014

The Google Scholar ‘Common Currency Ratio’

I’m a big fan of Google Scholar (and Google Scholar Citations). I use them both regularly to identify academic literature on a wide range of topics, and to trace, at least in outline, the responses to specific papers. I was fiddling about with it the other day, and wondered whether I could use it to try to develop a measure, even if a rough one, of the extent to which reference to common currencies occurs in a person’s published research.

My first attempt is the following:

  • An author’s Common Currency Score (CCS) is the number of results on Google Scholar for the author’s name (in quotation marks), plus “common currency” (also in quotation marks).
  • An author’s Common Currency Ratio (CCR) is their CCS divided by the number of results for their name alone.

This is rougher than a warthog’s shins, and will be hugely unsatisfactory in cases of names corresponding to multiple authors. (It’s pretty bad for authors whose names are indexed in more than one way - for example “PW Glimcher” returns about 3 times as many correct results as “PW Glimcher”. But searching for “DJ McFarland” picks up irrelevant McFarlands including false positives.)

Rough and ready though this is, it isn’t devoid of interest. George Ainslie has the highest CCS and CCR of anyone I thought to search for. I might update this in the future - as I apply it to other authors, or refine it in some way or other.

George Ainslie (22 June):
Common Currency Score (CCS) = 117
“George Ainslie” = 1680
George Ainslie CCR: 0.07

John McNamara (23 June)
Common Currency Score (CCS) = 151
“JM McNamara” = 3560
John McNamara CCR = 0.04

Paul Glimcher (22 June)
Common Currency Score (CCS) = 92
“PW Glimcher” = 1510
CCR: 0.06

David McFarland (22 June)
Common Currency Score (CCS) = 61 (for “DJ McFarland”)
“David McFarland” = 1130
David McFarland CCR = 0.05

David Spurrett (22 June)
Common Currency Score (CCS) = 25
“David Spurrett” = 455
David Spurrett CCR = 0.05

Peter Shizgal (23 June)
Common Currency Score (CCS) = 36
“P Shizgal” = 1070
Peter Shizgal CCR = 0.03

Michel Cabanac (23 June)
Common Currency Score (CCS) = 109
“M Cabanac” = 2320
Michel Cabanac CCR = 0.04




Friday, March 14, 2014

Paul Glimcher - Video Lecture

Postings here have been a bit thin and slow lately. I've been teaching a lot. Work continues when it can, though. I'm working on adding something about neuroeconomics. In the interim, here's a very fine talk by Paul Glimcher of NYU, on the subject of "Neurobiological Foundations of Economic Choice". Glimcher has extraordinarily deep understanding of a variety of economic theories, and of brain science. This is a fairly accessible, and very clear and lucid, exposition of the outlines of his research programme. It's really well worth watching, and a useful introduction/companion to Glimcher's terrific 2011 book Foundations of Neuroeconomic Analysis.

Elsewhere on this blog you can find one discussion of a paper by Glimcher (and Levy) reviewing and extending some of the neural evidence for a common currency: The Root of All Value.


Tuesday, February 25, 2014

The root of all value: a neural common currency for choice

Levy, D., & Glimcher, P. (2012). The root of all value: a neural common currency for choice Current Opinion in Neurobiology, 22 (6), 1027-1038 DOI: 10.1016/j.conb.2012.06.001


Introduction


Humans make enormous numbers of choices. Some of them are – at least in principle – relatively easy, for example choosing between one hundred, and ten thousand dollars or between one sip of water and ten sips. But others are more tricky, for example choosing between spending some free time watching a movie and playing tennis. In the latter case the two options vary in more than one way. Consequently a simple ‘more is better’ rule doesn’t determine what is best.

Given that we do, however, make large numbers of choices, including these superficially tricky ones, often without noticeable effort, we might wonder what it is that we do when we make them.

One idea, with a fairly long history, is that we do it by converting choices, including tricky ones, into easy ones, by expressing the worth of the different options in some abstract and general scale that is used for all choices. This more abstract scale is, of course, often referred to as a ‘common currency’ but might also be called ‘utility’ or ‘reward’, or ‘pleasure’ or just ‘value’. Once this conversion has been done, the idea is that all choices are simple ones: now we can take the one worth the most in the common currency.[1]

According to some, if our choices, including the tricky ones, are consistent, then we must be converting the options into values in a common currency. As discussed elsewhere on this blog, Shizgal and Conover have argued that:

“In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension” (Shizgal & Conover 1996).

This argument need not be decisive. That is, it is possible that some other mechanism which did not involve an internal common currency could explain ‘orderly choice’. So one question that arises here is a theoretical one, about what range of mechanisms could produce orderly choice. Perhaps the only viable options are mechanisms involving common currency. Independent of this theoretical question, is an empirical one: How, in fact, do humans achieve orderly choice?[2] Levy and Glimcher survey some evidence, supplemented by a new meta-analysis of previous research, to defend the conclusion that whether or not they have to, human brains do in fact implement a common currency.

Levy and Glimcher’s review of existing evidence


Levy and Glimcher endorse the same theoretical argument that convinces, among many others, Shizgal and Conover. Here is how they gloss the argument:

What we need to do is to take into consideration many different attributes of each option (like color, size, taste, health benefits, our metabolic state, etc.), assess the value of each of the attributes, and combine all of these attributes into one coherent value representation that allows comparison with any other possible option. What we need, at least in principle, is a single common currency of valuation for comparing options of many different kinds” (Levy & Glimcher 2012: p1027).

I emphasise that for Levy and Glimcher this argument functions more as an encouragement to seek a certain kind of evidence, rather than a proof that it will be found in advance of empirical investigation. To repeat, the main point of their paper is to review the evidence that there is in fact a common currency.

Speaking very generally, this is done by relating aspects of patterns in choice activity to aspects of patterns of neural activity. The word ‘aspects’ is not idle in either case: It is not every feature of choice activity, or of the options offered, that is of interest, and the same goes for neural activity during choice. Again speaking very generally, it is the economic aspects of patterns of choice that are of interest. Those are the ones that reveal strength of relative preference between options – in both easy cases and tricky ones. Sometimes it will take a large number of choices before a pattern – let us call this a behavioural pattern – can be characterised in suitable detail. Similarly, the neural activity that is of interest will be that fraction that is specific to the ways in which choices vary economically, which involves removing from consideration the parts which are involved with other aspects of the task (such as sensory discrimination and motor control). Again, sometimes a large number of neural measurements will be needed before a neural pattern is detectable. But, with decent data of both kinds, it is possible to ask whether there is a neural pattern that corresponds appropriately to a behavioural one.

Prior neureconomic research in humans has found that a number of brain regions are commonly engaged by tasks with an economic aspect:

“Over the course of the past decade there have been a wealth of studies suggesting that activity in small number of brain areas encodes reward quantities during decision-making tasks. […] Indeed, there is now broad consensus in the neuroscience of decision-making community that reward magnitude is represented in a small number of well-identified areas” (Levy & Glimcher 2012: p1027).

The focus of this review is specifically on the claim that fMRI studies in humans support the view that the ventromedial prefrontal cortext/orbital frontal cortex (vmPFC/OFC) represent the value of many reward types on a common scale. (The claim is not that this region is unique in this respect, merely that it is one area that serves that function.)

The evidence for this claim comes, in the first place, from studies of three broad kinds:

  • In some, subjects were offered purely monetary rewards (variously received, chosen, anticipated, lost, etc.), and the relationship between behavioural and neural pattern encouraged the view that vmPFC/OFC activity was correlated with strength of preference.
  • In others, extending the first category, complicate the monetary rewards by introducing delays as well as probabilistic and ambiguous payments, or by varying the ways in which choice is expressed or values communicated. Again, it was found repeatedly that behavioural pattern was correlated with levels of vmPFC/OFC activity.
  • In the third category, at least one reward type is non-monetary. Levy and Glimcher survey the results of ten studies, in which the non-monetary rewards included social rewards (including reputation), gustatory rewards (food, drink), pain (avoided or suffered), and aesthetic rewards (viewing attractive faces). In these studies too, behavioural and neural patterns correlated in the vmPFC/OFC.

This last category is the most encouraging for the view that the vmPFC/OFC represents a common currency for all choices, because it finds the same general kind of association for reward types other than money. And we want a common currency to represent non-monetary rewards. It is important to see, though, why the third category of empirical result above falls short of being conclusive evidence:

“But in order to demonstrate that these representations exist in a single common currency appropriate for computing the trade-offs that guide choice one must also show that the activity-level in these areas is equivalent whenever subjects report that offers of two different kinds of rewards are equally desirable” (Levy & Glimcher 2012: p1032).

What Levy and Glimcher are saying is that evidence for a common currency requires a more demanding fit between behavioural and neural patterns. Orderly choice between reward types should be consistent with a determinate exchange rate. And if there is a neutrally represented common currency, then it should be possible to find correlates of that exchange rate in patterns of brain activity. According to Levy and Glimcher, two studies (as of their 2012) provide evidence of the required kind. In both of them the behavioural pattern allowed the exchange rate between two reward modalities to be determined (in one case between money and time spent viewing images of attractive faces, and in the other certain and risky choices over money and food). In both cases appropriate correlations were found, which is very encouraging news for defenders of a neural common currency in humans. Additional methodologically similar studies may strengthen the case.

Levy and Glimcher’s meta-analysis


Over and above the review of previous studies, Levy and Glimcher include a novel meta-analysis in their paper. To do this they extracted the ‘peak voxel’ – that is, the most active voxel – for value-related activity from each of the thirteen studies in their review, and marked them on a single brain template. The result shows considerable agreement across designs and reward types:

Peak (value related activity) voxels across the 13 studies in the meta-analysis. A 5mm cubed sphere has been drawn around each voxel to make the figure more legible. From page 1034 of paper. See also Table 1 on page 1031.

This close association, across quite divergent experimental designs (with different measures of preference) and with a variety of different reward types, is impressive and interesting. Levy and Glimcher express the upshot of all of this as follows:

“From these data we think that a single conclusion seems at this point relatively straightforward. There is indeed a small subregion in the vmPFC/OFC that tracks subjective value on a common currency appropriate for guiding choices between different kinds of rewards” (Levy & Glimcher 2012: p1035).

Conclusions


Levy and Glimcher are clear that not all questions here are settled. Even if all reward types are represented on a common scale in the vmPFC/OFC, there are open questions about how this fits into wider networks subserving choice, learning and action in the brain. One possible view of the outlines of those networks is provided in the following figure (from the paper):

Figure 6 from Levy & Glimcher (2012).

At most, then, the evidence surveyed by Levy and Glimcher supports the claim that the region numbered (1) – which is the vmPFC/OFC – represents values of all options in a common currency.

But even the evidence for that is, as they note, not (yet) entirely conclusive. Not all reward modalities have been examined, and the experiments surveyed all have money in common as at least one modality. Future work, consolidating and extending the evidence, should widen the range of reward types offered in choice situations, and also include choices where neither option is monetary.

In this, it seems fair to say, that some kinds of variation will be more interesting than others. The value of a study with some new reward type that is for all that broadly similar to ones already studied (for example by using access to social media instead of some other social reward) will be lower than one which involves a more radical departure from established results. From this perspective perhaps the most interesting candidates for study are those that are the subject of explicit rejections of common currency claims. Two specific categories seem to me to be worth noting:

  • Some people claim, for example, that some of the things that humans value they do so ‘lexically’ so that any amount of the one (no matter how small) is worth more than any amount of the other (no matter how large). Such preferences, if anyone really has them, would involve discontinuities in a common currency, and these discontinuities should be empirically detectable – both behaviourally and neurally. One recent study has claimed (in my view running far, far ahead of what the evidence it musters can really support) to have found neural evidence that some humans have‘sacred’ values.
  • In addition, some people claim that some values are incommensurable, in the sense that for some combinations of them, there is no fact of the matter about whether one or the other is better, or the two are of equal value. (See 'Incommensurable values' in the Stanford Encyclopedia of Philosophy.) This is a strong claim, and if true it should have both behavioural consequences and neural consequences. As far as I know nobody has seriously attempted to asses the incommensurable values claim in an approximately neuroeconomic experiment.


It would not be trivial to investigate either possibility. (I hope to write about both the difficulties and possibilities in the future, on this blog.) But the results would certainly be very interesting.

Full Abstract (of Levy & Glimcher 2012):


How do humans make choices between different types of rewards? Economists have long argued on theoretical grounds that humans typically make these choices as if the values of the options they consider have been mapped to a single common scale for comparison. Neuroimaging studies in humans have recently begun to suggest the existence of a small group of specific brain sites that appear to encode the subjective values of different types of rewards on a neural common scale, almost exactly as predicted by theory. We have conducted a meta analysis using data from thirteen different functional magnetic resonance imaging studies published in recent years and we show that the principle brain area associated with this common representation is a subregion of the ventromedial prefrontal cortex (vmPFC)/orbitofrontal cortex (OFC). The data available today suggest that this common valuation path is a core system that participates in day-to-day decision making suggesting both a neurobiological foundation for standard economic theory and a tool for measuring preferences neurobiologically.
Perhaps even more exciting is the possibility that our emerging understanding of the neural mechanisms for valuation and choice may provide fundamental insights into pathological choice behaviors like addiction, obesity and gambling.

Reference

Levy, D., & Glimcher, P. (2012). The root of all value: a neural common currency for choice Current Opinion in Neurobiology, 22 (6), 1027-1038 DOI: 10.1016/j.conb.2012.06.001

Link to PDF on Glimcher Lab Website.

ResearchBlogging.org
  





[1] There’s a possible complication that I’m skipping over here. Even if we convert all choices into a common scale, we might apply some decision rule other than ‘choose the best’. We might, for example, allocate our actions (over time) between several options in proportion to their values.
[2] There’s another complication that I’m skipping here. There is room for disagreement over whether human choices are orderly at all (they’re clearly partly orderly). And among those who agree that there’s some kind of order, there’s disagreement over how best to describe the order. Levy and Glimcher are, furthermore, fully aware of the variety of models of the order in human choice.

Monday, February 3, 2014

Inner Conflict - TEDxUmhlanga

Back on 7 September of last year, I did a talk at TEDxUmhlanga, on the topic of 'Inner Conflict'. Some of what I spoke about is relevant to the common currency topic, in the sense that it concerns a reward-based explanation for inconsistency in choice patterns. (Inconsistency might otherwise be taken as an argument against a common currency.) The talk is embedded below.



(If you've seen the video of my talk at 'Thinking Things Through' in December, you might be wondering whether I ever wear anything else. Yes. Yes, I do.) I've also been doing some more serious research work on this topic, and hope to have more things to post really soon.


Draft: Philosophers should be interested in ‘common currency’ claims in the cognitive and behavioural sciences

[This is the working text of my paper from the Philosophical Society of Southern Africa (PSSA) conference in 2014. I've had to cut the paper down drastically to get the length into the ballpark required for consideration for the conference proceedings volume. So expect a much longer and slower version of this in the future and some point. Sorry that some of the formatting is a bit irregular. Many a slip twixt clipboard and the next application.]


Philosophers should be interested in ‘common currency’ claims in the cognitive and behavioural sciences.

David Spurrett (UKZN) for PSSA 2014

spurrett@ukzn.ac.za

Abstract

A recurring claim made in a number of behavioural, cognitive and neuro-scientific literatures is that there is, or must be, a unidimensional ‘common currency’ in which the values of different available options are represented.

There is striking variety in the quantities or properties that have been proposed as determinants of the ordering in motivational strength. Among those seriously suggested are pain and pleasure, biological fitness, reward and reinforcement, and utility among economists, who have regimented the notion of utility in a variety of ways, some of them incompatible.

This topic deserves philosophical attention for at least the following reasons: (1) Repeated invocation of the ‘common currency’ idiom isn’t merely terminological coincidence because most of the claims are competing explanations for one or the other of two putative kinds of fact. In one case the currency represents a principle of manifest pattern in choices. In the other, it is a functional part of the processes which produce choice. (2) We can’t suppose that the different currency claims within each area are compatible, because there are significant obstacles to identifying pairs of members of either the ‘pattern’ or ‘process’ group. (3) There are, finally, seriously opposed positions about the relationships (generally, and in specific cases including that of humans) between the pattern facts and the process facts.

Philosophical positions both favouring and opposing a common currency exist. Philosophers who incline to view their positions as at least partly empirical, should be more interested in the issues outlined here than they are.



1. Introduction

A recurring claim made in a number of behavioural, cognitive and neuro-scientific literatures is that there is in fact, or must be, a unidimensional ‘common currency’ in which the values (actual, or expected) of different available options are represented. These currency metaphors partly succeed older, less overtly economic, yet similarly quantitative ways of speaking of decision-making as a kind of ‘weighing’, or of options attracting with varying ‘force’. What these images share is doing approximate justice to the pre-theoretical notion that the value – to the choosing agent – of two options may be equal, or unequal, and when unequal that the relative difference between them can differ in magnitude.[1] When all options are taken to stand in these relations, we are on the way to a common currency thesis.

The theses get fleshed out in a variety of ways as the pre-theoretical notion is regimented in specific scientific and philosophical contexts. Different kinds of argument and evidence are appropriate to defending the more specific formulations. Here are two fairly typical and widely cited examples:

“In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension” (Shizgal & Conover 1996).

“Any attempt to understand behavior in terms of the evolutionary advantage that it might confer has to find a "common currency" for comparing the costs and benefits of various alternative courses of action” (McNamara and Houston 1986: 358).

These arguments are at least superficially similar enough that they might be taken as variants of a single argument, or as complementary arguments for the same thesis. Both seem to demand a single currency, encouraging viewing them as complementary. Indeed, the prospect that they, or combinations of similar seeming arguments, might collectively justify multiple and fundamentally distinct currencies (as opposed to superficially distinct ones that turned out to be equivalent) is prima facie perplexing. At the very least, the claim of one currency to be ‘common’ would be undermined by the existence of others.

One claim I defend here is that this superficial appearance – that we have complementary arguments for one claim – is misleading. Significantly different claims about common currencies can and should be recognized. The remark by Shizgal and Conover occurs in discussion of experiments demonstrating that behaviour allocation in rats given choices between natural (gustatory)[2] rewards and brain stimulation reward is sensitive to opportunity cost in the un-chosen reward, and that combination rewards with components in both modalities are (approximately) valued as sums of their components. The ‘common currency’ in question is reward magnitude for the individual rat. And Shizgal and Conover claim that this kind of pattern in allocation – the sensitivity to opportunity cost and disposition to value combination rewards as sums of their components that they call ‘orderly choice’ – is empirically observable and warrants an inference to a common currency that contributes to the choice process.

McNamara and Houston, on the other hand, are describing the project of behavioural ecology. They observe that this project presupposes that behaviours have determinate fitness consequences, and seeks to determine what those consequences are. One of their central claims is, furthermore, that the main successes of behavioural ecology up to that time had focused on restricted currencies, such as net rate of energy intake in optimal foraging models (e.g. Pyke et al 1977), and made relatively little headway with interpreting more comprehensive behavioural repertoires and life histories from the perspective of fitness. Such a comprehensive mapping, though, would express the relative values of all behaviours in an evolutionary common currency.

The two currency theses (Shizgal and Conover’s on the one hand, and McNamara and Houston’s on the other) are clearly not equivalent. Among the differences, one is centrally concerned with fitness, the other with something closer to subjective (expected) utility. Only one of them seems concerned with the choice-making process. Perhaps the repeated occurrence of ‘common currency’ talk is simply an un-interesting coincidence of terminology: researchers talking about distinct phenomena happen to use a superficially similar idiom. Perhaps, though, there is a tangle worth trying to unravel.

I favour the latter view, and in this brief paper offer a compressed survey of the terrain in which common currency theses occur, along with a preliminary defence of the claims that there is a tangle at all, and that the tangle is philosophically interesting. Identifying all of the interestingly distinct claims about common currencies, and assessing the bodies of evidence relevant to each, is far too large a project for a single paper. My aims here are therefore more modest. In what follows I introduce a classification of common currency theses (section 2), and explain why some of the thus classified theses should be understood as competitors (section 3). Not all currency theses conflict, but where combinations of them are possibly complementary, we find additional disputes and disagreements (section 4), and this set of issues is philosophically interesting (section 5).

2. Common currency claims distinguished

Distinct claims about common currencies or value scales occur in various scientific settings. Generically, and perforce vaguely – to begin with – because different more precise formulations pull in partly incompatible directions, a common currency is a unidimensional quantity that different options have in varying amounts. A common currency is a representation of what an agent maximises, or, more weakly, it is some value ordering with which the agent’s behaviour is consistent.

The two main ways of clarifying currency claims are, first, to specify in greater detail the characteristics of the value scale (whether ordinal, interval, etc.), and, second, to describe groupings within the range of currency theses and the types of explanations in which they feature. The latter task commands priority because the specific regimentation of the notion of scale that is appropriate depends on what purportedly is being measured or described and how. Here I focus mostly on classification, making only passing remarks about scale.[3]

Claims about common currencies are offered as explanations of one or both of two putative facts:

The currency represents the fundamental principle in some pattern in the choices made by some agent. OR

The currency is a characteristic of the processes by which choice is produced.

Scientifically interesting claims about common currencies, that is, can be divided into two groups. To mark this division I follow, but slightly adapt, an established distinction, and refer to ‘ultimate’ and ‘proximal’ currencies. An ultimate currency is a construct in a descriptive or explanatory theory of the behaviour of some agent. A proximal currency, on the other hand, is supposed to play some role in the processes by which options are selected by an agent.

Any single ranking of options on a scale such that behaviour can be described as – perhaps approximately – consistent with that ordering counts as an instance of what I will call an ‘ultimate’ common currency. An ultimate currency relates values to options, or to what selecting those options achieves or perhaps have the function of achieving. It is easy to imagine possible instantiations of an ultimate currency (‘all of Jim’s actions are efficiently ordered to contribute to the greater glory of the Flying Spaghetti Monster’) but among the scientifically interesting forms of consistency, two families stand out. One of these relate to fitness, and the other to some of other form of utility.

To mark this distinction I say that an ultimate currency can be evolutionary or subjective. Here I’m knowingly, albeit slightly, departing from standard usage, insofar as the usual way of distinguishing proximal from ultimate has the latter taken to be synonymous with fitness promoting. The standard sense is partly preserved here, because what I call an evolutionary ultimate currency attributes values that are a function of contribution to fitness. A guiding presumption of behavioural ecology is that behavioural dispositions make contributions to fitness, and that to the extent that the dispositions have a heritable basis, selection will tend to drive them towards making (constrained) optimal contributions. The statement by McNamara and Houston briefly discussed above is an exemplary assertion about an evolutionary ultimate currency.

There are compelling reasons for thinking that natural selection will operate on at least some behavioural tendencies, and so a strong general justification for the project of behavioural ecology. Nonetheless, the question of whether the behaviour allocation of the individuals in any particular species does indeed tend to optimise fitness (or would have in historical selective environments) is an empirical one. Behavioural ecologists have studied many species and types of behaviour and achieved striking successes in restricted domains such as foraging and mate selection. These successes have often relied on focusing on a simplified and more empirically tractable currency such as net rate of calorie intake (in the case of foraging), or the health of the selected mate (in mate selection). These proxy currencies plausibly contribute to fitness. Even so, success relating behaviour patterns to the proxies falls short of establishing relationships between all behavioural dispositions and overall fitness in any species. Two of the most serious shortfalls are in the area of relative allocation between significantly different modalities (such as calorie intake versus pursuit of mating opportunities versus predator avoidance) and variation in allocation over extended periods of time, including the full life-history.

A subjective ultimate currency, in contrast, attributes values that are a function of the revealed preferences or inferred utilities of the individual agent, without requiring any relationship to fitness. The paradigmatic sciences of subjective ultimate currencies are microeconomics (with variously regimented notions of utility functions revealed through consumption) and behavioural psychology (where strength of reinforcement is defined in terms of effect on patterns of behaviour allocation). Evolutionary ultimate currencies, then, have stronger empirical conditions than subjective ones, because the latter require ‘mere’ consistency in behaviour, whereas the former require consistency in contributing to fitness.

The distinction between evolutionary and subjective ultimate currencies, as I’ve sketched it, conceals important technical variation. There are different and competing fitness concepts, debates over the level (especially gene or individual) at which selection operates, and differing positions over the correct (if any) individuation principles for genes, genomes, species and other relevant categories. There are also different and competing utility concepts, offering different explanations of the same empirical data. (For example, some make randomness a feature of the utility representation itself, while others hypothesise randomness in the form of ‘trembling hands’ in the process of expressing the preferences.) Surveying this terrain is beyond the scope of this brief paper, but one further complication must be noted. It concerns two different ways of understanding ‘consistency’.

Economists tend to favour a strict notion of consistency because they recognise that predictable inconsistency makes agents vulnerable to systematic exploitation, so such agents would lose an influence on markets. Perhaps the best-known specific version of this worry is the argument that an agent with cyclical preferences[4] could be used as a ‘money pump’ (See Ross 2005, Chapter 5). Such an agent would freely pay for a series of trades that eventually left it with no effective money or stock. This convinces most economists that viable agency requires acyclicity (‘transitivity’), among other criteria for consistency. Behavioural psychologists, on the other hand, seek phenomenological fits of functions to empirical data, and have recognised patterns in behaviour objectionable to micro-economists. The clearest example of this is their accepting that the generalised Matching Law applies to delayed rewards (Chung and Herrnstein 1967). This implies that rewards are valued in inverse proportion to delay (i.e. by a hyperbolic function), and the relative desirability of incentives at different times can change simply with the passage of time. Given appropriate repeated choices, cyclical preferences follow.[5] That should mean vulnerability to money pumping, and non-behavioural economists have favoured delay discounting according to exponential functions largely for this reason (again, see Ross 2005, Chapter 5). The hyperbolic delay discounter is inconsistent insofar as she temporarily prefers smaller rewards that are immanently available. But to the behavioural psychologist her choices are all consistently reward seeking, in the sense that once some empirical parameters have been determined, relative rates of behaviour are predictable.

The other main role for currency claims is to explain the processes by which options are selected. Such theses assert the existence of a ‘proximal’ common currency. Here are some examples. A realist about desires who holds that for any pair of desires there is a fact of the matter about whether they are of equal strength or one is stronger, is committed to a common currency thesis. Shizgal and Conover’s inference regarding a “single, common dimension” is a scientifically motivated claim about the cognitive requirements of producing “orderly choice”. A conventional chess-playing programme generating a tree of possible game-states, then attaching values to them on the basis of some alogorithm, in order to select a best move (among the options explored in the available time) implements a common currency. Finally, the leading current scientific research programme focused on a proximal common currency is neuroeconomics, which seeks to determine how utilities are represented in brains, and how these representations are processed in choosing and learning (e.g. Levy & Glimcher 2012).

Not everyone who thinks that behaviour is consistent, and that there is a mechanical process explaining behaviour selection, is committed to a proximal common currency. This is because not all views about how behaviour is caused involve representations, including value representations. One reason is respect for what is now sometimes known as ‘Morgan’s cannon’:

In no case may we interpret an action as the outcome of the exercise of a higher psychical faculty, if it can be interpreted as the outcome of one which stands lower on the psychological scale. (Morgan, 1894)

Giant sea slugs (Pleurobranchaea), for example, are carnivorous and typically eat any animal matter they run into, “including other sea-slugs and their eggs” (Manning and Dawkins 1998: 226). They do not, however, eat their own eggs during egg laying. This disposition is obviously fitness enhancing: creatures that routinely consume their own offspring leave fewer viable descendants. The mechanism which stops sea slugs from eating their own eggs, though, while it could be regarded as in some very broad sense ‘cognitive’ is not one in which values are represented (e.g. Godfrey-Smith 2002).[6] When sea slugs lay eggs, they release a hormone that inhibits movement of the mouth (Davis et al 1977). This simple override exemplifies the ‘subsumption’ relationship between control layers championed by Rodney Brooks (e.g. 1991), who famously maintained that ‘intelligent’ behaviour could be achieved in the absence of representation.

A claim asserting a proximal common currency, then, is not any assertion about mechanisms producing behaviour, even consistent behaviour. A proximal common currency is, rather, a single, structured and integrated set of states that represents values, in at least the weak sense that there is a supportable mapping between the states and values in an ultimate currency.

There are three generic and complementary forms of argument in favour of a proximal currency. The first attempts an inference to best explanation for observed order in behaviour. The second considers abstract features of a control system, and argues that the control bottlenecks make it more likely that some unified value representation is playing a role. This is sometimes called the ‘final common path’ argument (e.g. McFarland & Sibley 1975). Finally, there are cases where it is claimed that a proximal currency has effectively been observed in action, through study of the behaviour control system at work (e.g. in neuroeconomics). All three are, furthermore, contested.

I maintain that scientifically interesting talk about common currencies can almost entirely be organised into arguments (i) in favour of or against one kind of currency in this taxonomy, and (ii) in favour of or against inferences from ultimate to proximal currencies. I regard this as a significant payoff of the taxonomy just outlined. Let me now argue briefly that the set of currency theses to be found in current science warrant serious philosophical examination.

3. Competitors, not complements

I discuss attempted inferences between distinct currency thesis types, especially from ultimate to proximal, in the next section. Here I consider relationships between ultimate (pattern) and proximal (process) common currency theses. The number of at least superficially distinct common currency theses is striking. As we’ve seen, behavioural ecologists seek to relate patterns of behaviour to their contribution to fitness. Early utilitarians, and some contemporary theorists about pain and analgesia claim that pain and pleasure provide a common scale (e.g. Bentham 1789, Cabanac 1971, Leknes & Tracey 2010). Behaviourist psychologists refer to reward or reinforcement, while contemporary economists are more likely to advance a currency thesis about utility. In addition some of the central concepts which characterise the scales have been theoretically elaborated in varied ways. This is most striking in the case of utility, both within economics, including behavioural economics, and in neighbouring areas such as decision theory.

This mere proliferation is not intrinsically interesting, of course. Perhaps the various theses are complementary. In the case of ultimate currencies, there is, prima facie, a prospect of pluralistic harmony, because such theses are claims about pattern. And, surely, more than one kind of pattern can be discerned in the same data. It is, though, an observation that is familiar to the point of banality that subjective preferences – for example for sex with contraceptives – don’t always coincide with what is fitness promoting. More generally, and not only in humans what is fitness promoting is not always motivating, and what is motivating is not always fitness promoting. The fact that animal subjects in behavioural experiments would work for non-nutritive sweeteners, for example, was recognised decades ago as an obstacle to identifying reward with evolutionary interest (see, e.g., Rachlin 1991, Chapter 3).

It is not unusual, in fact, to find biologists expressing scepticism about the chances of a currency thesis outside biology, even while being optimistic within it. Maynard-Smith, for example, suggested that “it has turned out that game theory […] is more readily applied to biology than to the field of economic behavior for which it was originally designed”. And part of his justification for this is that he finds utility to be a “a somewhat artificial and uncomfortable concept”, whereas in biology “Darwinian fitness provides a natural and genuinely one-dimensional scale” (Maynard Smith 1982, quoted in Glimcher 2002, p323).

Many economists and behaviourists dispute this, on the grounds that the utility concept is in much better shape than Maynard-Smith allows. My point is not to endorse either view, but to observe that the lack of consensus suggests work for applied philosophy of science. One impediment to taking sides is, in any event, that economists disagree with each other. There are competing formulations of utility and disputes over which do best justice to the evidence, or best suit what theoretical purposes. I noted above that economists’ concern with consistency leads them to favour exponential delay discount functions, while behavioural psychologists are more inclined to hyperbola-like functions. This is just one instance of a wider pattern. To give one more example, defenders of prospect theory (e.g. Kahneman & Tversky 1979) claim that their model, built to account for phenomena including apparent violations of the independence axiom, and asymmetries of risk sensitivity in gains and losses, does better justice to data about real human choices. Sceptics, though, express frustration that the sheer number of free parameters in prospect theory undermines the empirical value of such fits (see Glimcher 2011, Chapter 5).

A further incompatibility between the currency claims of behavioural ecologists and psychologists arises because they seek to explain somewhat different things, even while both calling it ‘behaviour’. We’ve noted the giant sea slug’s apparent restraint over eating its own eggs. My point there was to explain why behavioural ecologists are often carefully agnostic about proximal common currencies. Similar cases can also be used to make a different point. For example, a textbook case in in behavioural ecology is clutch sizes in oviparous species (on birds, see e.g. Lack 1966). There’s substantial evidence that clutch sizes in many bird species are close to what would maximise lifetime reproductive success of parent birds, whose situation involves trade-offs between current and future clutches, and between members of individual clutches, given that larger (better fed) young generally do better.

Some of the determinants of clutch size plausibly respond only to phylogenetic rewards, where successful offspring are the payoffs in repeated games with genotypes as the players or strategies. There is little reason to think, though, that clutch size is modifiable by reward or punishment directed at the individual bird, any more than one might bribe a giant sea slug into eating its own eggs. But responsiveness to reinforcement is what would make it behaviour for a behavioural psychologists or economist.

What about proximal common currency theses? It seems that multiple distinct proximal theses raise the prospect of over-determination. That is, if the mechanisms of choice include more than one structured and integrated set of states representing values and involved in causing behaviour, then we’d have more explanation than we needed. (‘I did it because it had to most expected utility, and it was pleasurable and I preferred it…’) This worry could be dissolved if it turned out that the different proximal currency theses expressed substantially the same claim, so that what some kind of agent wanted (or desired) was also what it liked (or gave it pleasure) and what promoted its fitness.

The prospects for such an outcome, though, are slim. Among other reasons, behavioural economists have drawn attention to various examples of apparently motivated behaviour involving considerable pain, for example mountaineering (e.g. Loewenstein 1999). Brain scientists studying the learning, enjoying and choosing brain now mostly maintain that ‘wanting’ and ‘liking’ (including enjoying and suffering) are neurally and functionally dissociated and that only the former promises to provide a perspective from which behaviour can be understood as consistent with some value representation (e.g. Berridge 2004). These arguments conflict directly with the claim defended by some researchers on hedonic experience that pain and pleasure provide a common currency for behaviour selection (e.g. Leknes and Tracey 2010). Whatever consensus eventually emerges, it seems clear that no more than one of the current proximal common currency theses can be correct.

4. Inferences from ultimate to proximal currency theses

Within categories – ultimate and proximal – different common currency theses, then, are competitors, but there are obvious ways that one thesis of each type could be complementary. If the order in behaviour warrants attributing an ultimate common currency, then this might justify hypothesising a proximal currency involved in producing the pattern. Conversely, if we had grounds to suppose that some agent’s cognitive processes implemented a proximal common currency, we might expect its behaviour to exhibit pattern consistent with an ultimate currency. Versions of both inferences have been defended, and contested.

The range of options for relations between proximal and ultimate currency theses is approximately analogous with those regarding the status of folk psychological kinds in in the philosophy of cognitive science. (Analogous might not be the best term, because folk psychology includes desires and beliefs. But most of the debates over folk psychology focused on epistemic states – like belief, perception, and memory – to the relative neglect of motivation states, like desires.) The major options are realism (beliefs are scientifically respectable), eliminativism (cognitive processes include nothing sufficiently like beliefs for belief talk to pass scientific muster) and attributionism, for example Dennett’s intentional stance (the conditions for belief attribution exclusively concern pattern in behaviour). Approximately corresponding to this, we find the following positions regarding proximal common currencies:

First there are realists committed to the existence of cognitive states standing for the motivational strength of different courses of action in humans, and at least some other agents. Examples include typical realists such as Fodor (1983), some scientists specialising in hedonic experience (e.g. Cabanac 1971) and neuroeconomists identifying preferences with brain states. A paradigmatic example of an inference from pattern to proximal representation is, of course, the syllogism stated by Shizgal and Conover (1996) quoted above, linking ‘orderly choice’ with the requirement of ‘representation’ on a common scale.

Second, are eliminativist positions, denying – to varying degrees – the reality or necessity of cognitive states corresponding to desires. Brooks (1991), for example, argues against the need for representations of any kind to produce ‘intelligence’. For him the world is ‘its own best representation’. Brooks directs most of his fire against representations in the sense of models of the external environment, but his arguments clearly imply rejection of representations of values.[7] Brooks’ work remains an inspiration to some philosophers. Clark (e.g. 1997, chapter 9) claims that pattern in human economic choices depends heavily on highly scaffolded choice environments. Sterelny (2003), on the other hand, suggests that the cognitive implementation of preferences in humans is “incomplete”, and that pattern in the behaviour of most non-humans can be explained without hypothesising preference like states as parts of their cognitive architectures.

Finally, we find attributionists, emphasising the view that the work of those who trade in ultimate common currencies is independent from proximal considerations. The exemplary contemporary attributionist about folk psychology is Dennett, and scientific attributionists share behaviourist inspiration. Micro-economists mostly regiment their notion of utility so that it makes no psychological or hedonic commitments, in favour of specifying different degrees of consistency that can be empirically manifest in behaviour (e.g. Samuelson 1938, see Ross 2005). Behaviourist psychologists are similarly suspicious of – or hostile to – claims about hedonic experience, and favour more empirically tractable notions such as reinforcement and reward (Thorndike 1927).

The task of assessing the strengths and weaknesses of realism, eliminativism and attributionism in the case of the more epistemic aspects of folk psychology (believing, perceiving, remembering, inferring…) attracted wide and deep philosophical activity over several decades. Over this period considerably less attention was paid to the motivational aspects of folk psychology (wanting, desiring, choosing). In the final section of this brief survey I attempt to say a little more about why these topics deserve more attention.

5. This is philosophically interesting

Suppose that I’ve convinced a reader that there is a tangle of variously competing and (possibly) complementary theses about common currencies across a number of cognitive and behavioural sciences. Why think that this tangle is of any philosophical significance? Among the reasons I could offer, let me single out three.

First, there’s an image of philosophers associated with Locke as a kind of conceptual janitor for empirical science (Locke’s image, in the Epistle to the Reader of his Essay concerning human understanding, was of an ‘under-labourer’). Such conceptual work is more clearly indicated, perhaps, in cases where there is conflict within and between empirical sciences. To illustrate this, consider the different situations of behaviourism in psychology, and economics. Behaviourist psychologists thought of themselves, to state the obvious, as being in the psychology business, although driven by a distinctive vision of what it meant to do that business in a methodologically serious way. Their rejection of introspection was methodological, motivated by the consideration that scientific evidence should be inter-subjectively available. Consistently with that commitment, some more recent behaviourists have argued that reports of subjective experience can be a kind of data (e.g. Dennett’s ‘heterophenomenology’ in his 1991). Furthermore, advances in measuring devices have made previously unobservable brain processes amenable to empirical study, and so allowed different kinds of data to pass muster by behaviourist lights. Neuroeconomics now promises to provide an empirical basis for a theory of reinforcement or reward that explains observed pattern in what behaviour can be reinforced, and to what degree. While some individual behaviourists might reject specific theoretical suggestions in this area, the lack of some theory commanding wide acceptance has long been recognised.

Given the noted inspiration behaviourism provided for contemporary formulations of the concept of utility in economics, one might expect similar enthusiasm for neuroeconomics among working economists. Instead, one finds lively and sometimes intensely polarised debate. This should not be especially surprising. Unlike behaviourists, who had a distinctive view about how psychology should be done, economists (who used to view their discipline as at least closely linked to psychology – e.g. Jevons 1911) view their discipline as having made progress by getting out of the psychology business entirely. While some behavioural economists have more recently sought to move in the opposite direction, they by no means dominate the profession.

This helps explain some of the response to an enthusiastic early manifesto for neuroeconomics (Camerer, Loewenstein and Prelec 2005). Here, and in related publications around the time, it was claimed that a new and importantly psychological research programme held out the hope of providing new ‘foundations’ for microeconomics. While the experimental techniques would have amazed Mill, Bentham, Jevons and their colleagues, the generic view of the relationship between economics and psychology probably would not. But 2005 is not the eighteenth century, and one of the most spirited responses to neuroeconomics was called ‘The case for mindless economics’, and argued forcefully (whether or not correctly) that economics and psychology were separated by a ‘logical’ gulf, that they addressed “different questions”, used “different abstractions” and relied on “different types of empirical evidence”, with the consequence that neuroscience could have no bearing on economics whatsoever (Gul & Pesendorfer 2005). It should not need pointing out that this dispute is, among other things, a special case of a wider and older debate about the status of folk-psychology.[8]

Second, the specific issues relating to the status of folk-psychology that arise in the case of motivation, and with respect to the credibility of this or that common currency thesis, are not simply ‘more of the same’ in the sense of being mere repetitions or generalisations of arguments about beliefs or representations. Were that the case, there would still be philosophical work to do, because applying whatever lessons emerged from the earlier debates would depend partly on specific details that science has revealed in the domain of preference and decision. Brooks’ general case against representation, for example, is widely recognised as failing in the case of belief except for very simple control systems, and may also fail in the case of preference, but perhaps not for the same reasons.

But there are also important ways in which things are different here. The ‘old’ debate over folk psychology was, as noted, disproportionately focused on epistemic matters, and so unsurprisingly much of the traffic with philosophy was in areas relating to epistemology and language. But those aren’t the most obvious or promising sources of exchange in the case of desires. There, in fact, the fruitful exchange is more likely to be in areas engaged with thinking about practical reasoning and decision, and thinking about value.

Third, and finally, philosophers already defend, assume or deny common currency theses or positions implying such commitments. A common opening move among decision theorists, for example, is to assume that values – whatever their ‘content’ – can be represented with the real numbers (e.g. Briggs 2010). More generally what one might call ‘empiricist’ views of motivation suppose that there is a central economy of desires competing on the basis of their strength, however that strength is understood. On the other hand more ‘rationalist’ views favour the view that in at least some cases motivation is not simply a matter of relative strength of desires, but is rather rule-based, or otherwise based on reason in opposition to desire.

Whether or not directly defending rationalism in this very loose sense, a number of lines of philosophical thinking oppose, directly or by implication, the thought that all options are valued on a single scale. Among the examples of this are: arguments that some options or values might be incommensurable, in the sense that there is no fact of the matter about which is more valuable, or whether they are equal (e.g. Raz 1986, Williams 1981); positions maintaining that some values are ranked ‘lexically’ in the sense that any amount of some – no matter how small - is worth more than any amount of others – no matter how large (e.g. Rawls 1971). In addition, in some recent empirical literature we find arguments that some values are ‘protected’ or ‘sacred’ and somehow isolated from trade-offs with others (e.g. Baron & Spranca 1997, Atran & Ginges 2012).

A number of long-standing philosophical questions about desire, decision, preference and practical reasoning, then, (and this survey is woefully short of comprehensive) are partly questions about the degree to which motivational systems are, or can be, unified, and about the kinds of ranking and hierarchy systems of value or motivational strength can or should have. Common currency theses – whether affirmed or denied – provide a useful level of abstraction that permits comparison of otherwise mutually isolated theses and theories, and a substantial and tangled scientific terrain is debating aspects of these very questions.


References

Ainslie, G. (1992).  Picoeconomics:  The Strategic Interac­tion of Successive Motivational States within the Person.  Cam­bridge: Cambridge University Press.
Ainslie, G. (2001). Breakdown of Will.  New York: Cambridge University Press.
Atran, S. and Ginges, J. 2012. Religious and sacred imperatives in human conflict. Science, 336: 855-857.
Baron, J. and Spranca, M. 1997. Protected values. Organizational behavior and human decision processes, 70(1): 1-16.
Bentham, J. (1789) Introduction to the principles of morals and legislation (1823 edition), Oxford: Clarendon Press.
Berridge, K.C. (2004) Motivation concepts in behavioral neuroscience, Physiology and Behavior, 81, 179-209.
Brooks, R.A. (1991) Intelligence without representation, Artificial Intelligence, 47: 139–159.
Cabanac, M. (1971) Physiological role of pleasure, Science, 173: 1103-7.
Camerer, C.F., Loewenstein, G. & Prelec, D. (2005) ‘Neuroeconomics: How neuroscience can inform economics’, Journal of Economic Literature, 34(1): 9-64.
Chung, S. & Herrnstein, R. J. (1967). Choice and delay of rein­forcement. Journal of the Experimental Analysis of Behavior, 10, 67-74.
Clark, A. (1997) Being There, Cambridge, Mass.: MIT Press.
Davis, W. J., Mpitsos, G. J., Pinneo, J. M., & Ram, J. L. (1977) Modification of the behavioral hierarchy of Pleurobranchaea. I. Satiation and feeding motivation, Journal of Computational Physiology. 117: 99-125.
Dennett, D. (1991) Consciousness Explained, Boston: Little, Brown.
Fodor, J. A. (1983). The Modularity of Mind. Cambridge, MA: MIT Press.
Glimcher, P. (2002) Decisions, Decisions, Decisions: Choosing a Biological Science of Choice. Neuron, 36: 323-332.
Glimcher, P. (2011) Foundations of Neuroeconomic Analysis, Oxford: Oxford University Press.
Godfrey-Smith, P. (2002) Environmental Complexity and The Evolution of Cognition, in R. Sternberg and J. Kaufman (eds.) The Evolution of Intelligence. Mahwah: Lawrence Erlbaum, pp. 233-249.
Gul, F. and Pesendorfer, W. (2008) The case for mindless economics. In Foundations of positive and normative economics, ed. A. Caplin and A. Schotter. New York: Oxford University Press.
Haig, D. (2006) Intrapersonal conflict. Pages 8-22, in M.K. Jones and A.C. Fabian (eds.) Conflict. Cambridge University Press, Cambridge.
Hamilton, W.D. (1987) Discriminating Nepotism: expectable, common and overlooked. In D.J.C. Fletcher and C.D. Michener (eds.) Kin recognition in animals. Chichester, U.K.: John Wiley and Sons.
Kahneman, D., and Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2): 263-291.
Kaye, B. and Spurrett, D. (in preparation) Subsumption architectures and common currencies.
Lack, D. (1966) Population studies of birds. Oxford: Oxford University Press.
Leknes, S. and Tracey, I. (2010). Pain and pleasure: masters of mankind. In, M. L. Kringelbach, and K.C. Berridge (eds). Pleasures of the brain. Oxford: Oxford University Press. 320-336.
Levy, D. J. and Glimcher, P. W. (2012) The root of all value: a neural common currency for choice, Current Opinion in Neurobiology, 22: 1027-1038.
Loewenstein, G. (1999) Because It Is There: The Challenge of Mountaineering . . . for Utility Theory, Kyklos, 52(3), pp. 315-43.
Luce, R.D. & Suppes, P. (1965) Preference, Utility and Subjective Probability, in R.D. Luce, R.R. Bush, & E.H. Galanter (eds), Handbook of Mathematical Psychology, vol 3., New York: Wiley: 249-410.
Manning, A. & Dawkins, M. S. (1998) An Introduction to Animal Behaviour (3rd edition). Cambridge: Cambridge University Press.
Maynard-Smith, J. (1982) Evolution and the theory of games, Cambridge: Cambridge University Press.
McFarland, D.J. and Sibley, R.M. 1971. The Behavioural Final Common Path, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 270(907), pp. 265-293.
McNamara, J.M. and Houston, A.I. (1986) The Common Currency for Behavioral Decisions, The American Naturalist, 127(3), pp358-378.
Morgan, C. L. (1894). An introduction to comparative psychology. London: W. Scott.
Pyke, G. H., Pulliam, H. R., & Charnov, E. L. (1977) Optimal foraging: a selective review of theory and tests. The Quarterly Review of Biology, 52, 137-154.
Rachlin, H. (1991) Introduction to Modern Behaviorism (3rd edition), New York: Freeman.
Rawls, J. (1971) A Theory of Justice. Cambridge: Harvard University Press.
Raz, J. (1986) The Morality of Freedom, Oxford: Clarendon Press.
Ross, D. (2005). Economic Theory and Cognitive Science, Volume One: Microexplanation. Cambridge, MA: MIT Press.
Samuelson (1938) A note on the pure theory of consumer’s behaviour. Economica, New Series, 5(17): 61-71.
Shizgal, P. & Conover, K. 1998. On the neural computation of utility, Current Directions in Psychological Science, 5(2), pp. 37-43.
Sterelny, K. (2003) Thought in a Hostile World, Oxford: Blackwell.
Suppes, P. and Zinnes, J.L. (1963) Basic measurement theory, In R. D. Luce, R. R. Bush,
E. Galanter (Eds.). Handbook of Mathentatical Psyhology, vol 1. New York: Wiley, 1-76.
Thorndike, E. L. (1927) The law of effect. American Journal of Psychology. 39:212–22.
Williams, B. (1981) “Conflicts of Values”, in Moral Luck, Cambridge: Cambridge University Press.






[1]         Some have an opposed hunch that there is no fact of the matter about the relative ranking of at least some options, perhaps because they are incommensurable (e.g. Williams 1981, Raz 1986).
[2]         The ‘natural’ rewards were modified in various ways in order to manipulate their similarities to and differences from brain stimulation reward (BSR). Shizgal and Conover make further inferences about the neural representation of value on the basis of how BSR and gustatory reinforcement respond differently to these manipulations.
[3]         For a classic discussion of the formal properties of some notions of preference see Luce and Suppes (1965), and for more general remarks on scales and measurement see Suppes and Zinnes (1963).
[4]         In the sense of preferring bundle a to b, bundle b to c, and also bundle c to a.
[5]         Nobody has done more than George Ainslie when it comes to thinking seriously about what it would mean for humans if we value rewards approximately in inverse proportion to their delay. See Ainslie (1992, 2001).
[6]         I follow Godfrey-Smith’s suggested policy regarding understanding ‘cognitive’ in a broad way to embrace the processes controlling behaviour, even in systems (e.g. some plants and fungi) that lack central nervous systems.
[7]         See Kaye and Spurrett (in preparation).
[8]         A striking feature of much of this specific version is repeated invocation of the notion of a common currency (e.g. Levy & Glimcher 2012).