Thursday, May 16, 2013

On the Motivational Commensurability of Pleasure and Pain


'On the Motivational Commensurability of Pleasure and Pain' is the working title of a talk/paper that is part of this sprawling ‘common currency’ project.

When the project started, I wasn’t especially interested in pleasure and pain. (That is, I wasn’t especially academically interested in them.) That was because I partly shared the behaviourist tendency in both the psychological and economic sense: introspection and affect weren’t much scientific use, and what we needed to know about motivation could be revealed in behaviour, or perhaps in behaviour and cognitive neuroscience.

One thing that changed my mind about this was finding out that pleasure and pain themselves get mooted as being the ‘common currency’. So, as a comparative investigator of common currencies, I had to pay some attention.

Another thing that changed my mind was paying a bit more attention to the pre-behaviourist history of some of the disciplines that are relevant to the project. For early utilitarians (like Bentham, and Mill) what was motivating coincided with what was pleasant or painful. Behaviourist psychology and most economics (since around the 1930s) have studiously avoided reference to affect (in favour of reinforcement, reward, and a de-psychologised notion of ‘utility’). But it’s an interesting question how what is discovered about motivation and what is discovered about pleasure and pain relate.

This represents what I somewhat pretentiously called (in version 1 of the talk) the ‘Optimistic Correspondence Hypothesis’, and which is fairly clearly false some of the time. This is my own diagram, but you will find equivalent images in some theoretically-minded discussions of pain.


Finally, I saw a call for papers for a future special issue of the Review of Philosophy and Psychology that got me thinking. I don’t know if I’ll have something I’m happy with ready in time for the deadline, but I started collecting some thoughts, and put together a working talk.

Presentations

15 May 2013, at UKZN philosophy. This was the first outing. I think it went pretty well, in the sense that I was reasonably clear, and (as I often find) immediately after thought of a number of improvements to the presentation and ways of describing the issues. So I’ll update the thing, and look for another audience I can pester with it. I'll post the slides here after I've refined them a little.


Monday, May 6, 2013

Cui bono? Selfish goals need to pay their way


The text below is the un-edited preprint of a commentary I've just submitted to BBS (Behavioral and Brain Sciences). The commentary is about a target article by Julie Y. Huang and John A. Bargh called "The Selfish Goal: Autonomously Operating Motivational Structures as the Proximate Cause of Human Judgment and Behavior".

I have no idea when the commentary might see print. It's a standard joke to say that 'BBS' stands for 'Badly Behind Schedule', and my commentary on Andy Clark's forthcoming target article (see 'My Own Efforts') appeared in another journal before the article it was commenting on. (There was a great flood of people wanting to say something, and some of those who didn't make the first cut had a second change through Frontiers in Theoretical and Philosophical and Psychology.)

As you'll see, I think that Huang and Bargh's proposal faces some serious difficulties. And some of them relate to common currencies.

Abstract

The target article risks not explaining the phenomena, including motivational conflict, that it claims to. The two mains reasons for this are: (1) it is unclear in what sense goals are‘selfish’. (2) We need an account of how selfish goals motivate people. If selfish goals aren’t in the replication business, then what is in it for them? And if they don’t offer people something that they want, how do they ever influence what people do?


Main Text

The proposal in the target article risks not explaining the phenomena, including motivational conflict, that it claims to. The two mains reasons for this are: (1) it is not clear enough in what sense goals might be ‘selfish’, or what incentives they respond to. (2) We need an account of how selfish goals motivate individual people, including how they compete with other incentives to which people respond.

Goals are apparently selfish in a way ‘analogous’ to Dawkins’ (1976) use with reference to genes. Dawkins argued that to understand much of biology it was necessary to take the perspective of the units of heredity. These are ‘selfish’ in the sense that their only interest is in being replicated. Serving this interest sometimes makes demands that oppose the well being of the individual carrying them. Dawkins focuses mostly on units of biological inheritance – genes – but also speculates that there may be analogously selfish cultural replicators or ‘memes’. In the case of memes, as with genes, selection processes favour efficient replicators, irrespective of whether this serves the interest of the ‘host’. In both cases it is clear in what sense the replicators are selfish, and what is ‘in it for them’. The game of life pays replicators with copies of themselves, and the various individuals in which they occur are means to that end. The interests of these replicators are not appreciated by the replicators themselves, but well defined in evolutionary game theory where payoffs are numbers of descendants (Maynard-Smith 1982).

The target article carefully avoids using the term ‘meme’, and makes almost no reference to replication (except in the context of glossing Dawkins’ popularisation of gene-centric selection). This suggests that the sense in which goals are selfish analogously with genes is not the same as Dawkins’ existing meme analogy. But if selfish goals don’t have a primary interest in replication, what are their interests? Goals, we are told, represent ‘end states’, and succeed by being pursued and/or by being completed.

This proposal needs to answer two crucial questions. First, what is ‘in it’ for goals such that pursuit and completion constitutes an incentive? Second, how might selfish goals compete for control of an individual?

It is difficult to discern an answer to the first question, or even clues as to what it might be, in the target article. If selfish goals aren’t replicators and are furthermore in some sense separate from the person they occupy, then they can neither be rewarded with copies of themselves, nor in whatever subjective utility the person responds to. (In any event, in the latter case they would not be selfish at all – they would simply be the person’s preferences.)

The second question is, if anything, even more important. The various (finite) degrees of freedom of any human represent a scarce resource that has alternative uses. That is to say the problem of behaviour allocation is essentially economic (Shizgal, 2012). Some of the processes that implement allocation are peripheral and relatively encapsulated, but most are not. The behaviour allocations of people are undoubtedly sensitive to costs and payoffs, even if it is a matter of controversy what specific economic model humans instantiate. In addition, there is mounting evidence that contemplating or selecting both desirable and aversive options in a wide range of modalities (including money, delayed money, risky money, food, drink, pain, looking at attractive faces, and social reputation) is consistently associated with activity proportional to behaviourally inferred desirability in a single brain region (for a recent review see Levy & Glimcher 2012). Independent of this specific evidence, the motor areas of the brain constitute a final common path for control processes, plausibly requiring any candidate deployment of the agent’s capacities to compete on the same terms as the others.

These considerations apply to selfish goals. The options available to a person (including end-states of goals) are often composed of complex mixtures of components (money, food, sex, status…) and in different modalities. Their availability could be immediate or delayed, and they can be subject to risk. The costs of options also vary in magnitude and type (effort, money, pain, delay…), and may themselves be multi-modal (e.g. including both monetary cost and delay). Making choices in even an approximately efficient way requires trading off these multi-modal options by reference to their net costs and benefits. Arguably, solving that problem is a significant part of what brains are for. Unless selfish goals are to be epiphenomenal, therefore, they need to compete along with the already recognized sources of subjective utility (or reward, or reinforcement), and they need to do so somewhere along the recognised pathways for behavioural control. 

These factors appear to make almost no impact on the argument in the target article, where there is no mention whatsoever of utility, incentive, consumption, pleasure or risk, and merely solitary and passing references to pain, reinforcement, and reward. But selfish goals need to motivate the agents that they occupy, and to do so they need to pay their way in some kind of incentive to which those agents are responsive.

Finally, it is worth pointing out that even non-selfish goals can come into conflict, and that people motivated purely by their own incentives can exhibit inconsistency over time. The most banal of objectives can be mutually exclusive (resting and working, specialising and generalising). All that is needed to explain some conflict, that is, is that not all desires (even those of a self) can be satisfied. In addition, the ways we price in the costs of various ways of being separated from a reward (by delay, risk, effort, etc.) can themselves be a source of inconsistency. This has been most extensively studied in the case of delayed rewards, where there is considerable evidence that humans discount rewards that will be received later (Ainslie 1992, Kable & Glimcher 2007) in a manner that corresponds more closely to a hyperbolic than exponential function. The mere passage of time, that is, can change the ranking of preferences within a single person.

References

Ainslie, G. (1992) Picoeconomics: The strategic interaction of successive motivational states within the person. Cambridge University Press.

Kable , J. W. , and Glimcher , P. W. ( 2007). The neural correlates of subjective value during intertemporal choice . Nature Neuroscience, 10(12): 1625-1633.

Levy, D. J. and Glimcher, P. W. (2012) The root of all value: a neural common currency for choice, Current Opinion in Neurobiology, 22: 1027-1038.

Maynard Smith, J. (1982) Evolution and the Theory of Games. Cambridge University Press.

Shizgal P (2012) Scarce means with alternative uses: Robbins’ definition of economics and its extension to the behavioral and neurobiological study of animal decision making. Frontiers in Neuroscience. 6(20). 


Wednesday, May 1, 2013

Shizgal and Conover’s ‘Orderly choice’ argument and evidence


ResearchBlogging.org One of the clearest statements of an argument for a common currency thesis is found early in Peter Shizgal and Kent Conover’s 1996 paper ‘On the neural computation of utility.’ This is a really important paper for the common currency topic, and should be required reading for anyone in the area. It doesn’t only state a version of one of the key arguments, it also provides an exemplary kind of empirical evidence.

Here is how they state the argument:

In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension (Shizgal & Conover 1996).

The second sentence can be slightly reformulated (making one implicit premise explicit) as follows:

Premise 1: Orderly choice is possible.
Premise 2: For orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension.
So
Conclusion: The utility of all competing resources is represented on a single, common dimension.

This certainly looks like a valid argument. But what does it mean?

Well, the suggestion is clearly that (a) the rewards available to an organism might vary in lots of ways, across multiple dimensions (consider sex, rest, food of different kinds, grooming, water, avoiding predators, feeding young, etc.), and (b) making ‘orderly’ decisions between available rewards, the rewards all need to have a simple one-dimensional representation. (See The (very) Basic Big Idea on this site.)

Shizgal and Conover don’t claim that all choice is in fact orderly. But they clearly intend to say that when it is orderly, a common dimension of comparison is required. But what is it for choice to be orderly? Looking at Shizgal and Conover’s experiments will help clarify this.

The research Shizgal and Conover report on concerns ‘brain self-reward’. A famous paper by Olds and Milner (1954) reported that rats would work, including learning novel behaviours, for no more reinforcement than pulses of electrical stimulation to a part of the medial forebrain where an electrode terminated. Some early reports of brain-self reward – as it came to be called, or (BSR) for short – focused on ways in which it was unusual. The popular imagination got excited by the rat that pressed its lever nearly continually for around 20 days at about one press every two seconds (Valenstein & Beer 1964), or cases where food was apparently ignored over brain self-reward (Routtenberg & Lindy 1965). Shizgal and Conover sought, in part, to demonstrate that brain self-reward is a reinforcer like any other. In a series of experiments they had rats chose between trains of BSR pulses and infusions of sucrose solution.

The experimental design infused the juice directly into the rats’ mouths to give the sucrose solution a key property of BSR, which is that a single action leads to both procurement and consumption. In addition the swallowed fruit juice was drained away, reducing postingestive effects so that the infusions shared another property of BSR, which is absence of satiation.

Schematic illustration of experiment 1 from preprint version of the paper.

Shizgal and Conover manipulated the strength of reinforcement to each modality (by changing the duration or number of pulses in a train of BSR, or the size of the sucrose infusion) and measured the relative allocation of lever presses to each. Among other things they found that each individual reward modality was preferred over nothing, more of it preferred to less, and that increasing the opportunity cost of either (by increasing the reward available from the other lever) led to less consumption of that reward.

In a variation on the experiment they manipulated the magnitude of a BSR-only reward, while the alternative reward was a constant combination of BSR and sucrose solution. In this condition it took more BSR to make the rat forgo the compound reward than had been necessary for the sucrose part of the reward alone.

Schematic illustration of experiment 2 from preprint version of the paper.
Their conclusion is that the results of the first experiment imply, “that on a given trial, the rat selected the alternative that registered a larger value in a common system of measurement”, rather than following a categorical rule (such as ‘whenever the size of one reward is above some threshold, choose it’).

Regarding the second experiment with the combination rewards, they say it implies “that the electrical stimulation and the sucrose were subjected to a common evaluation”.

(The paper describes two further experiments, complementary to those I’ve just described. I’ll discuss them in a future posting.)

So, one way of glossing this is that Shizgal and Conover demonstrate that rat choices between BSR and sucrose solution is orderly in two senses:

  • Choices between modalities are quantitatively sensitive to opportunity cost.
  • Choices between rewards in one modality and two-modality combination rewards are similarly sensitive to the combined opportunity cost.

These results are important. They show (in terminology that I’ve described elsewhere on this site, but which isn’t used by Shizgal and Conover) that at least some rat choices have a certain patterning, conforming to an ultimate common currency.

Shizgal and Conover also clearly intend the conclusion that the results about an ultimate common currency support the view that there is a proximal one:
“We speculate that this common ability arises from a common action of the gustatory and electrical stimuli on a neural system that determines goal selection by signaling the utility of competing goals.”
The other two experiments, and their ongoing research programme, provide further support for this speculation. But that’s all I’ve got time for now.

References

Olds, J. & Milner, P. 1954. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47, pp. 419-427.
Routtenberg, A. & Lindy, J. 1965. Effects of the availability of rewarding septal and hypothalamic stimulation on bar pressing for food under conditions of deprivation. Journal of Comparative and Physiological Psychology, 60, pp. 158-161.
Shizgal, P. & Conover, K. 1998. On the neural computation of utility, Current Directions in Psychological Science, 5(2), pp. 37-43. [Publisher’s site –may be behind a paywall] [Preprint version at CogPrints] [Google Scholar Citations]
Valenstein, E.S. & Beer, B. 1964. Continuous opportunity for reinforcing brain stimulation. Journal for the Experimental Analysis of Behavior, 7, pp. 183-184.

Forthcoming attractions

Discussion of Shizgal (1999).

See also

Currencies can be 'ultimate' or 'proximal'




Shizgal, P., & Conover, K. (1996). On the Neural Computation of Utility. Current Directions in Psychological Science, 5 (2), 37-43 DOI: 10.1111/1467-8721.ep10772715

Friday, April 26, 2013

The invisible blog

So, the term 'common currency' is mostly used to talk about currency unions. This is, of course, where more than one political entity - usually a state - use the same currency in the sense of money.

The term used much less frequently in the sense relevant to this blog, which is with reference to decision, pattern in choices, and the processes by which choices are produced. As a result this blog is, so far, barely visible. Except for social traffic from people who pay attention to my personal activity on Facebook and Twitter, there's almost no traffic at all. Part of the problem is the dominance of what for me are irrelevant uses of 'common currency'. Even searching for "common currency brain science blog" doesn't (yet) return results where this site is anywhere near the top.

I'm going to try to take a few steps to improve matters. One of those is signing the blog up on Research Blogging, so that appropriate posts can be digested and appear in their feeds. I'll also see what I can do to generate some more inbound links.

Wednesday, April 24, 2013

Image collection: Brain dollars, dollar brains


The field of neuroeconomics has been attracting a certain amount of popular press, and also generating a growing number of book and special issue publications. Both of these drive appetite for illustrations. And people sometimes seem very keen to distinguish their illustrations from the general sorts of “pictures of brains” that go with other publications about neuroscience. So what are they to do?

It turns out that a fairly common approach is to throw together something that superimposes images of brains and images of cash, or that show brains and cash in some kind of interaction.

It takes only a little thought to see that some of the images are actually pretty silly. My reading (about common currencies) keeps taking me to places where I see these things, and so I’m going to start collecting them and posting them here. If you run into an image that you think belongs in the gallery, please either comment below or email me. (I’d also really welcome pointers to good images relating to the idea of value representation in the brain, such as the Society for Neuroeconomics logo below. This gallery is not just a Hall of Shame.)

Category 1: Brains or skulls with money on or in them


It is common to think that different motives can compete, and that that individual ones can differ in how motivating they are. At different times people have been tempted to use different metaphors to express this. When Newton was a relatively fresh and salient inspiration, it was more common to use the term ‘force’ to describe what motives had, and what some of them had more of. I’m not sure when it became popular to describe the process of deciding between competing motives as a kind of ‘weighing’, but I’d guess it was when balance scales were part of everyday experience. These days the fashionable metaphors are economic, and specifically related to cash. This is part, I think, of why ‘common currency’ talk is increasingly popular.

Exhibit 1.1: A skull with a hand holding a fan of cash in it.


Source: http://historiesofthingstocome.blogspot.com/2011/12/neuroeconomics.html
It's not clear what to make of this. What controls a hand sticking out of the spinal column? Who or what might see or interact with the money?

Exhibit 1.2: Synapses forming a suggested dollar sign


Source:  http://www.neuroeconomics.org/

This is the rather cool logo of the Society for Neuroeconomics. I like that the dollar sign is suggested and entangled rather than superimposed.

Exhibit 1.3: Clunky dollar signs on a murky brain


Source: http://www.apa.org/monitor/2009/01/neuroeconomics.aspx

This one is a 'bottom of the barrel' FAIL on the APA website. It's pictures like this that made me think of posting this gallery.


Exhibit 1.4: Single clunky dollar sign on a murky brain


Source:  http://xvatit.com/school/sch-online/compet/120473-urok-keys-dlya-chogo-potriben-internet-banking.html
This is remarkably similar to exhibit (1.3) above. The page it's from is in Russian, and I have no idea what it's about, but it turns up on searches for 'neuroeconomics'.

Exhibit 1.5: Skull looking at a dollar bill


So this is basically exhibit 1.1 above, but with the hand and the bill outside the head.

Source: http://financelongrun.blogspot.com/2013/10/neuroeconomics.html

Category 2: Money with brains on or in it


Showing cash in or on the brain can suggest the idea of brains processing representation states related to value or utility. So some category 1 pictures can sort of make sense. The converse idea, that there might in any sense be a coherent metaphor where there are little brains in the cash, is rather farcical. This does not prevent some designers from adopting it, and some editors from approving it. Here are a few examples.

Exhibit 2.1: Dollar bills with heads and brains on them


Source: http://cas.umkc.edu/psychology/brain/current.html 

This (the logo for a project on the "Neuroeconomics of controversial food technologies") really speaks for itself. But what is it saying?


Exhibit 2.2: One Dollar bills with one brain on it

Source: http://disinfo.com/2011/11/the-neuroeconomics-revolution/#sthash.x321F4oj.dpbs

The brain in this money is glowing. Maybe it was bitten by a radioactive banker, or something.


Category 3: Brains and money in some kind of interaction


Exhibit 3.1: The weighing metaphor, with brains and cash

Source: http://www.cns.nyu.edu/symposia/sympo2008.html

This is pretty odd if you think about it. Unless your concern is the price of brains. Which I suppose it might be, if you were an economically active zombie. But if the brain is in the scale, then it isn't doing the weighing, or reading off the values. So what or who is? Maybe there are little brains inside brains, and the little brains do all the processing. (The same picture also appears on the cover of the important collection "Neuroeconomics: Decision Making and the Brain" edited by Glimcher, Fehr, Camerer and Poldrack. [Publisher link.])

Category 4: Miscellaneous


Exhibit 4.1: Head chart


So here's a quite fetching little image. There's no cash in the skull, or bits of brain somehow stuck onto the cash. But there's a data line of some kind that looks price-like, and with an arrowhead suggesting a projection beyond where the data runs out. It makes much more sense to think that brains represent costs and/or returns, and projects trends, than to think that they've got little banknotes in them.

Source: http://blogs.cfainstitute.org/investor/2012/02/27/bankable-insights-eight-lessons-from-neuroeconomics-for-money-managers/





Monday, April 22, 2013

The (very) basic big idea

Buridan' Ass
Old illustration of Buridan's Ass (famously indifferent between two food options).

If you’re struggling for a rough sense of what the hell I’m talking about on this site, then this post is supposed to help. It’s deliberately non-technical and informal. My aim is to give a fairly general and intuitive sense of the idea of a common currency, and avoid the technical details that go with how that concept is regimented in various sciences.

Consider any two things that you want. Here are some possibilities:
  1. You want each of them to about the same degree.
  2. You want one of them more than you want the other.
  3. There’s no fact of the matter about how(or whether) the degree to which you want them differs.
Suppose, further, that in this case possibility number (3) does not apply. (This isn’t a trivial thing to suppose – and I’ll discuss it in future postings.)

Now suppose that either option (1) or (2) applies not just to the particular two things that you initially imagined, but to every possible pair of things taken from the whole set (possibly infinite) of things that you want.

Then your wants would form a kind of list (possibly infinite), where every item on the list had a definite ranking (perhaps with some on the same rankings).

Those rankings would form a kind of common currency, like positions in a race (first, second, third, etc.). It would be common because everything you want would be somewhere on it.

It’s important that the ranking should include things that you don’t want too, because (a) you should want them less than things you positively want (possibility 1 or 2 above should apply to each of them), and (b) because they’re not all equally horrid. This means that we can (if we want) imagine the ranking to represent the net desirability of the things on it. (So a cupcake that costs ten dollars, or ten minutes of effort, could come lower than the very same cupcake for one dollar, or one minute of effort.) But we could also represent the costs as negative wants on their own, and suppose that net desirability is calculated (by subtracting costs from outcomes). Those are different ways of representing the same information.

We might imagine that the strengths of our wants are a little more quantitative. So, I might say not merely that I want a cup of coffee more than I want a glass of water, but that I want a cup of coffee twice as much as a glass of water. If every pair of our wants stood in this kind of relationship, then they would all have definite relative degrees of desirability. This would give a similarly common scale, but one that includes information about the relative sizes of the gaps between positions.

There are at least two ways of thinking about this ‘common currency’ scale, or dimension:
  • In one case, it is a set of facts about the pattern in the actual choices you make. (One way of thinking of this is to suppose that an economist or behavioural researcher had observed a lot your activity and inferred the relative value of different outcomes from your choices. Such researchers might at least be agnostic about is be going on inside you. Many economists and behaviourists are officially agnostic about the inside, even though they’re committed to common currencies.) I’ve explained elsewhere on this blog that I find it useful to call this kind of “pattern in choices” currency ultimate.
  • In the other case, it is a set of facts about the processes by which your choices are generated. One way to think about this is to think that your wants, or desires, or preferences, are real psychological states of yours. Another (which need not be the same thing) is to suppose that the strengths of your wants are represented in neural activity in your brain. I’ve explained elsewhere on this blog that I find it useful to call this kind of “process producing choices” currency proximal.

It turns out that scientists of various kinds (including behavioural ecologists, economists, behavioural psychologists, and neuroeconomists) argue that there is a common currency for at least some kinds of agent. But they sometimes argue for ultimate currencies, and sometimes proximal ones, and they use different kinds of evidence and theory to defend their positions.

It’s also not difficult to find examples of people denying that there is a common currency. Some deny it for humans, partly because they find it offensive to suppose that our most noble wants might appear on the same scale as our most base ones. Some deny it for some classes of animal or robot, usually saying that internal representations (including of strength of preference) aren’t necessary for orderly behaviour.

I’m interested in the tangle of claim and counter-claim that this combination of arguments in favour and against produces, when looked at comparatively. To get an idea of what I’m working on see My OwnEfforts and the links there.

See also:

Important Sources: Rodney Brooks – Denies representations for at least some robots, so denies proximal currency.
Important Sources: Kim Sterelny – Seems to hold a position sharing some key features with Brooks.
Important Sources: George Ainslie – Staunch defender of common currency thesis.

Thursday, April 18, 2013

Intragenomic Conflict and Common Currencies


‘Intragenomic Conflict and Common Currencies’ is the working title of one of the papers that is part of this common currency project. It grew out of my reading of an essay by David Haig called ‘Intrapersonal Conflict’ which is the first piece in an anthology simply called Conflict (Cambridge, 2006, edited by M Jones and AC Fabian).

Picture showing differences between embryos of mice where genetic imprinting had been manipulated. (See the powerpoint slides below.) Image from Haig (citation below) in turn from Keverne et al.

I presented a very preliminary version of the argument at the 2013 conference of the PSSA, at the Salt Rock Hotel, in KwaZulu-Natal, South Africa. Because I’d also been organizing the conference, the talk wasn’t as worked out as it might have been.

More recently (April 17, 2013) I presented an updated version to the senior seminar at UKZN, and I’ll be taking the same talk to the University of Johannesburg around the end of April.

The written draft of the paper is still something of a construction site, but my presentation slides may be of some interest, and so I’m posting them here.

I’ll post updated slides from future presentations (right here on this page), and eventually the working draft of the paper when it’s ready for comment and feedback.

References

Haig D (2006) Intrapersonal conflict. In: Jones M, Fabian AC (eds) Conflict. Cambridge University Press, Cambridge, pp 8–22.