Wednesday, May 1, 2013

Shizgal and Conover’s ‘Orderly choice’ argument and evidence One of the clearest statements of an argument for a common currency thesis is found early in Peter Shizgal and Kent Conover’s 1996 paper ‘On the neural computation of utility.’ This is a really important paper for the common currency topic, and should be required reading for anyone in the area. It doesn’t only state a version of one of the key arguments, it also provides an exemplary kind of empirical evidence.

Here is how they state the argument:

In natural settings, the goals competing for behavior are complex, multidimensional objects and outcomes. Yet, for orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension (Shizgal & Conover 1996).

The second sentence can be slightly reformulated (making one implicit premise explicit) as follows:

Premise 1: Orderly choice is possible.
Premise 2: For orderly choice to be possible, the utility of all competing resources must be represented on a single, common dimension.
Conclusion: The utility of all competing resources is represented on a single, common dimension.

This certainly looks like a valid argument. But what does it mean?

Well, the suggestion is clearly that (a) the rewards available to an organism might vary in lots of ways, across multiple dimensions (consider sex, rest, food of different kinds, grooming, water, avoiding predators, feeding young, etc.), and (b) making ‘orderly’ decisions between available rewards, the rewards all need to have a simple one-dimensional representation. (See The (very) Basic Big Idea on this site.)

Shizgal and Conover don’t claim that all choice is in fact orderly. But they clearly intend to say that when it is orderly, a common dimension of comparison is required. But what is it for choice to be orderly? Looking at Shizgal and Conover’s experiments will help clarify this.

The research Shizgal and Conover report on concerns ‘brain self-reward’. A famous paper by Olds and Milner (1954) reported that rats would work, including learning novel behaviours, for no more reinforcement than pulses of electrical stimulation to a part of the medial forebrain where an electrode terminated. Some early reports of brain-self reward – as it came to be called, or (BSR) for short – focused on ways in which it was unusual. The popular imagination got excited by the rat that pressed its lever nearly continually for around 20 days at about one press every two seconds (Valenstein & Beer 1964), or cases where food was apparently ignored over brain self-reward (Routtenberg & Lindy 1965). Shizgal and Conover sought, in part, to demonstrate that brain self-reward is a reinforcer like any other. In a series of experiments they had rats chose between trains of BSR pulses and infusions of sucrose solution.

The experimental design infused the juice directly into the rats’ mouths to give the sucrose solution a key property of BSR, which is that a single action leads to both procurement and consumption. In addition the swallowed fruit juice was drained away, reducing postingestive effects so that the infusions shared another property of BSR, which is absence of satiation.

Schematic illustration of experiment 1 from preprint version of the paper.

Shizgal and Conover manipulated the strength of reinforcement to each modality (by changing the duration or number of pulses in a train of BSR, or the size of the sucrose infusion) and measured the relative allocation of lever presses to each. Among other things they found that each individual reward modality was preferred over nothing, more of it preferred to less, and that increasing the opportunity cost of either (by increasing the reward available from the other lever) led to less consumption of that reward.

In a variation on the experiment they manipulated the magnitude of a BSR-only reward, while the alternative reward was a constant combination of BSR and sucrose solution. In this condition it took more BSR to make the rat forgo the compound reward than had been necessary for the sucrose part of the reward alone.

Schematic illustration of experiment 2 from preprint version of the paper.
Their conclusion is that the results of the first experiment imply, “that on a given trial, the rat selected the alternative that registered a larger value in a common system of measurement”, rather than following a categorical rule (such as ‘whenever the size of one reward is above some threshold, choose it’).

Regarding the second experiment with the combination rewards, they say it implies “that the electrical stimulation and the sucrose were subjected to a common evaluation”.

(The paper describes two further experiments, complementary to those I’ve just described. I’ll discuss them in a future posting.)

So, one way of glossing this is that Shizgal and Conover demonstrate that rat choices between BSR and sucrose solution is orderly in two senses:

  • Choices between modalities are quantitatively sensitive to opportunity cost.
  • Choices between rewards in one modality and two-modality combination rewards are similarly sensitive to the combined opportunity cost.

These results are important. They show (in terminology that I’ve described elsewhere on this site, but which isn’t used by Shizgal and Conover) that at least some rat choices have a certain patterning, conforming to an ultimate common currency.

Shizgal and Conover also clearly intend the conclusion that the results about an ultimate common currency support the view that there is a proximal one:
We speculate that this common ability arises from a common action of the gustatory and electrical stimuli on a neural system that determines goal selection by signaling the utility of competing goals.”
The other two experiments, and their ongoing research programme, provide further support for this speculation. But that’s all I’ve got time for now.


Olds, J. & Milner, P. 1954. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47, pp. 419-427.
Routtenberg, A. & Lindy, J. 1965. Effects of the availability of rewarding septal and hypothalamic stimulation on bar pressing for food under conditions of deprivation. Journal of Comparative and Physiological Psychology, 60, pp. 158-161.
Shizgal, P. & Conover, K. 1998. On the neural computation of utility, Current Directions in Psychological Science, 5(2), pp. 37-43. [Publisher’s site –may be behind a paywall] [Preprint version at CogPrints] [Google Scholar Citations]
Valenstein, E.S. & Beer, B. 1964. Continuous opportunity for reinforcing brain stimulation. Journal for the Experimental Analysis of Behavior, 7, pp. 183-184.

Forthcoming attractions

Discussion of Shizgal (1999).

See also

Currencies can be 'ultimate' or 'proximal'

Shizgal, P., & Conover, K. (1996). On the Neural Computation of Utility. Current Directions in Psychological Science, 5 (2), 37-43 DOI: 10.1111/1467-8721.ep10772715

No comments:

Post a Comment