Current Zoology 61 (2): 368–381, 2015

From foraging to autonoetic consciousness: The primal self

as a consequence of embodied prospective foraging
Thomas T. HILLS1*, Stephen BUTTERFILL2
1
Department of Psychology, University of Warwick, CV4 7AL, Coventry, UK
2
Department of Philosophy, University of Warwick, CV4 7AL, Coventry, UK

Abstract The capacity to adapt to resource distributions by modulating the frequency of exploratory and exploitative behaviors
is common across metazoans and is arguably a principal selective force in the evolution of cognition. Here we (1) review recent
work investigating behavioral and biological commonalities between external foraging in space and internal foraging over envi-
ronments specified by cognitive representations, and (2) explore the implications of these commonalities for understanding the
origins of the self. Behavioural commonalities include the capacity for what is known as area-restricted search in the ecological
literature: this is search focussed around locations where resources have been found in the past, but moving away from locations
where few resources are found, and capable of producing movement patterns mimicking Lévy flights. Area-restricted search
shares a neural basis across metazoans, and these biological commonalities in vertebrates suggest an evolutionary homology be-
tween external and internal foraging. Internal foraging, and in particular a form we call embodied prospective foraging, makes
available additional capacities for prediction based on search through a cognitive representation of the external environment, and
allows predictions about outcomes of possible future actions. We demonstrate that cognitive systems that use embodied prospec-
tive foraging require a primitive sense of self, needed to distinguish actual from simulated action. This relationship has implica-
tions for understanding the evolution of autonoetic consciousness and self-awareness [Current Zoology 61 (2): 368–381, 2015].
Keywords Self-projection, Mental time travel, Episodic future thinking, Foraging, Consciousness

While studying the abilities of rats to navigate mazes, for working out how to get from one place to another.
Tolman and colleagues observed that in many cases rats But it also allows another kind of behaviour—delibera-
took routes that suggested they could piece together tion. That is, cognitive representations that can be in-
memories in a way that allowed them to make novel ternally searched should provide organisms with the
inferences (Tolman, 1948; Tolman and Gleitman, 1949). capacity to explore alternatives, via internal foraging,
As opposed to simple cue-response relationships that prior to committing themselves to one course of action.
might be sufficient to learn a maze, the animals demon- Meunzinger and Fletcher (1938) called behavioural
strated the ability to make the kind of inferences that evidence consistent with internal foraging—that is, ap-
would require search over some form of internal repre- parent deliberation at a choice point—vicarious trial-
sentation—akin to realizing that one could go to India and-error and learning, and showed that it correlated
by taking a novel route around the earth. Tolman (1948) with subsequent performance (see also Hu and Amsel,
called these representations “cognitive maps” and ar- 1995). More recent evidence from neuroscience indi-
gued that they provided a vast repertoire of behaviour cates that animals at choice points activate areas of the
not conceivable under the standard behaviorist's stimu- brain associated with the outcomes of past decisions at
lus-response theories. What is meant by a cognitive map that point. One goal of this article is to provide a basis
and which species, if any, have them is highly debated for understanding why this kind of internal foraging
(Cheung et al., 2014; Cruse and Wehner, 2011; Tsoar et may have been pre-adapted for by the evolution of ex-
al., 2011). For our purposes it is necessary only that ternal foraging.
these “maps” be cognitive representations of the world A second goal is to explore the consequences of in-
and involve a sense of place such that some things can ternal foraging for the existence of a precursor to self-
be nearer to one another than others. The capacity to awareness, or what Tulving (1985) called autonoetic
search through internal representations provides a means consciousness: “Autonoetic (self-knowing) conscious-

Received Nov. 1, 2014; accepted Feb. 12, 2015.

 Corresponding author. E-mail: [email protected]
© 2015 Current Zoology
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 369

ness is the name given to the kind of consciousness that strategies (Banks et al., 2008; Stephens and Krebs, 1987).
mediates an individual’s awareness of his or her exis- Among these, one strategy for mediating the trade-off
tence and identity in subjective time extending from the between exploration and exploitation—called area-re-
personal past through the present to the personal future” stricted or area-concentrated search—is extremely com-
(p. 1, 1985). More specifically, in relation to distinctions mon. Animals that use area-restricted search localize
made by William James (1890), the “I” is conscious and search around areas where resources have been found in
the “Me” is one of many things the “I” can be conscious the past (Kareiva and Odell, 1987), but move away
of. We are interested in the “Me,” the potential object of from locations where resources are encountered less
self-awareness. Our aim will be to describe how a pri- frequently. A classic example is houseflies increasing
mal version of this self is a consequence of internal their turning angle when they encounter a drop of su-
foraging. However, readers should not be too worried, crose, such that they continue searching nearby, but
as we promise to arrive far short of the kind of self- reducing their turning angle as the time since the last
awareness commonly attributed to humans (Marko- encounter grows, so that they move away to explore
witsch and Staniloiu, 2011). Nonetheless, autonoetic new locations (Bell, 1991). Area-restricted search is ob-
consciousness has been tied to behaviors such as mental served throughout metazoans (i.e., animals), including
time travel (Schacter et al., 2007; Suddendorf and Cor- nematodes, moths, leeches, rodents, and humans (see
ballis, 2007), self-projection (Buckner and Carroll, 2007) Fig. 1; Hills, 2006). Area-restricted search is also found
and episodic future thinking (Atance and O'Neill, 2001), among animals that are well known for other, more
and these have often been proposed as exclusively hu- idiosyncratic search behaviors, such as the desert ant
man capacities. This naturally raises the question of Cataglyphis bicolor, which is renowned for its path-
where these abilities may have originated. Our proposal integration abilities, but quickly engages in area-
entails that self-awareness and its associated cognitive restricted search when path-integration fails (Wehner
consequences are unlikely to be exclusively human ca- and Srinivasan, 1981). Moreover, increasingly research
pacities. is suggesting that the power-law distributed movement
Connecting the dots between foraging and autonoetic patterns associated with Lévy flights may also be pro-
consciousness requires understanding the selective duced by area-restricted search (Hills et al., 2013; Plank
forces most likely to have driven the evolution of cogni- and James, 2008).
tion, specifically the trade-off between two opposing The commonality of area-restricted search is ex-
behavioral motivators—exploiting known resources and plained by its adaptation to locating clustered resources.
exploring for unknown resources. In what follows, we Area-restricted search is a useful strategy for tracking
will first discuss the comparative behavioral and bio- resources that are spatially-autocorrelated, especially
logical commonalities underlying adaptations for re- when patch boundaries are not well defined. When an
solving this trade-off. Then we will provide experimen- encounter with a resource indicates other resources
tal evidence that this capacity is conserved to search nearby, an animal’s turning angle should be a function
internal representations, and that this is indeed a core of resource encounters. This has been demonstrated in
function of what cognitive psychologists mean by ex- numerous studies (Grünbaum, 1998; Hills, 2006; Ka-
ecutive (or effortful) cognition. Finally, we will provide reiva and Odell, 1987). Moreover, in direct comparisons
evidence that these capacities are necessary for fore- with memoryless strategies, such as Lévy flights, memo-
sight, or rather, looking ahead, and that, in turn, cogni- ry-based strategies like area-restricted search have a
tive systems that use embodied prospective foraging distinct performance advantage when resources are
must distinguish between real and simulated events and clustered (Ferreira et al., 2012; Plank and James, 2008).
thus require a precursor to autonoetic consciousness— Further, species-area distributions are consistently found
recognition of the self. to be more aggregated than random placement models
would predict, indicating spatial-autocorrelation (Harte
1 External Foraging et al., 2008), and seed dispersal distributions of primary
Among metazoans, the capacity to forage—that is, to producers also tend to fall-off with distance from the
adaptively search for and exploit resources—is ubiqui- parent (Janzen, 1970); both suggest a strong tendency
tous. The strategies associated with foraging are varied, for resource clustering in the natural world, though
ranging from the relatively simple, such as sit-and-wait whether foragers experience this will depend on the
or random walks, through to more systematic search scale at which they forage.
 370 Current Zoology Vol. 61 No. 2

Fig. 1 Examples of area-restricted search in nematodes and humans

A. Paths taken by the nematode Caenorhabditis elegans immediately after removal from food (0–5 minutes) and 30 minutes later (30–35 minutes).
The white arrow shows a period of roughly straight, forward motion; the dark arrow shows a high-angled turn. The bottom panel of A shows the
frequency of high-angled turns over the two periods of observation. Reproduced from Hills et al. (2003). B. Paths of humans foraging for invisible
resources in a 200 m diameter virtual foraging arena with diffuse and clustered resources. The lower panel shows that when encountering resources
(indicted by a tone) humans in clustered environments take sharper turns, indicating area-restricted search. Reproduced from Hills, Kalff, and Wie-
ner (2013).

In some cases, the ubiquity of area-restricted search is such as spatial foraging and active exploration. Though
certainly the outcome of convergent evolution. One can not without exceptions, the dominant pattern of dopa-
reasonably argue that bacteria engage in area-restricted minergic signalling facilitating persistent or stereotypic
search, because the second-messenger systems they use behaviors is found across vertebrates (Barron et al., 2010;
to track resources have dephosphorylation times on the Winstanley et al., 2012).
order of seconds (Macnab and Koshland, 1972). This
2 The Evidence for Structure in Internal
allows them to detect differences in resource concentra-
tions larger than their own body length, which in turn Foraging
allows them to “remember” that a position they were in Area-restricted search represents a behavioral solu-
a moment ago is better than a position they are in now. tion to the exploration-exploitation trade-off necessary
They can then make a random turn in an effort to return to forage successfully for resources clustered in space.
to the general vicinity of their previous location. However, the exploration-exploitation trade-off is in the
In vertebrates, on the other hand, there is strong rea- same class as other evolutionary trade-offs known to
son to believe that area-restricted search is symplesio- influence evolution across a variety of domains—such
morphic—a trait shared across a variety of extant taxa as the surface-area-to-volume trade-off or the size-num-
that was also possessed by their remote common ances- ber trade-off. As a consequence, solutions to the explo-
tor. The principal reason for this inference is the shared ration-exploitation trade-off are found across natural
biological basis of area-restricted search (reviewed in living systems, from stress induced mutagenesis at the
Hills, 2006; Hills and Dukas, 2012). In invertebrates, genomic level to DNA sharing at the population level
behaviors modulated by dopaminergic signalling in- (Claverys et al., 2006; Galhardo et al., 2007). Impor-
volve feeding or foraging related behaviors, while in tantly for our present purposes, the exploration-exploi-
vertebrates the behaviors mediated by dopaminergic tation trade-off is also commonplace in cognitive do-
signalling are more consistent with area-restricted search, mains, such as in memory search, information search,
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 371

problem solving, and visual search (Hills et al., 2015). els of memory also make the central assumption that
In each of these domains, cognitive systems must de- memory is a high-dimensional manifold that has struc-
cide between continuing with one course of action that ture, such that items in memory can be near or far from
has reasonably certain consequences, versus exploring one another (Austerweil et al., 2012; Thompson and
other possible courses of action for which the outcomes Kello, 2014).
are less certain. Problem solving is another domain that provides
Memory search provides one domain where internal evidence of internal foraging. Cognitive models of pro-
search shares features common to animals foraging in blem solving often use quasi-spatial representations to
space. In particular, both internal and external search predict how people will arrive at solutions. For example,
appear to involve dynamic patterns of search over struc- in an anagram-like task where people are asked to pro-
tured domains; items can be “near” or “far” from the duce multiple four-letter words from one letter set, such
position of search. One approach to understanding the as RNTPOE, people produce solutions that are more
structure of memory has been to develop quantitative similar by string-edit distance to one another the more
representations that can then be used to predict and proximate they are in the series of produced solutions
measure behavior. For example, semantic representa- (Hills et al., 2008; 2010). In the above example, a series
tions of memory derived from statistical models that of solutions might look as follows: ROPE, ROTE,
read through Wikipedia and compute quantitative mea- NOTE, RENT, PENT, etc. The first three and last two
sures of word similarity have been used with success to solutions can each be arrived at in series by changing
predict the order in which people recall lists of items only one letter, whereas the third and fourth solutions
from memory (see Fig. 2A; Hills et al., 2012; Hills et al., (NOTE to RENT) suggest a larger transition across the
2013; Thompson and Kello, 2014). These structures solution space. Using a computational model called the
also allow for testing alternative search algorithms. In executive search process, Hills et al. (2010) demon-
model comparisons, the best fitting models of iterated strated that search in this task could be explained by
memory search are two-stage models that assume local assuming a local search process that made iterative
foraging (using one item in memory to activate the next changes to the prior solution, but occasionally emptied
item in memory) with occasional non-local, long-range the memory buffer and started again by sampling from
transitions to richer regions of the memory representa- the original letter set.
tion (Hills et al., 2012; Hills et al., 2013; Hills and The Remote Associates Test is another problem solv-
Pachur, 2012). This is similar to area-restricted search, ing domain that is well modeled by local search strate-
and some of the earliest mathematical models of serial gies. The task is to find, as quickly as possible, a word
recall from memory assume area-restricted search-like that is common to three other words, such as MOON,
dynamics (Raaijmakers and Shiffrin, 1981). Other mod- DEW, and COMB. Here the answer is HONEY. In a

Fig. 2 Examples of structure in internal foraging

A. A typical sequence of items produced in the animal fluency task in which participants are asked to “name all the animals you can think of.” Par-
ticipants often produce items in clusters of semantically similar items (solid lines) with infrequent transitions between clusters (dotted line). B. A
typical sequence of items produced in the remote associates task, in which participants are asked to produce a word that is common to each of three
other words. Here, the words are COMB, MOON, and DEW. In Smith et al. (2013), participants were also asked to produce prospective solutions
that came to mind as they searched for the final solution. Participants appeared to produce prospective solutions in clusters (solid lines) in associa-
tion with a cue word, and transitioned between these (dotted lines) as they homed in on their final solution.
 372 Current Zoology Vol. 61 No. 2

recent study asking participants to produce the incorrect of performance measures such as SAT scores and meas-
solutions they thought of while hunting for the correct ures of general fluid intelligence (Unsworth and Engle,
solution, responses were well characterized by a process 2007). People with higher working memory spans also
that involved sequential dependencies between adjacent tend show longer periods of local foraging in internal
responses (see Fig. 2B; Smith et al., 2013). memory representations before making long-distance
The evidence of sequential dependencies across these jumps to new regions of the memory space (Hills et al.,
domains reveals two things. First, internal search ap- 2013; Hills and Pachur, 2012; see also Rosen and Engle,
pears to share with external search the property of in- 1997). Thus, features of internal memory search appear
volving search in structured spaces. And, second, quan- to share general control processes known to be involved
titative representations of internal spaces (e.g., memory in other behaviors associated with the control of atten-
or symbolic problem spaces) can be used to predict tion.
cognitive navigation of internal spaces in much the Finally, the capacity for behavioral focus (i.e., atten-
same way as Euclidean distances can be used to predict tion), like the capacity for spatial foraging, appears to
navigation in external spaces. be a conserved neuromolecular process across verte-
brates. At the synaptic level, this is characterized by the
3 Evidence for an Evolutionary dopaminergic modulation of G-protein linked receptors,
Relationship between Internal inducing a cascade of intracellular events that modulate
cellular responses to other neurotransmitters—most
and External Foraging
commonly, glutamate (Nicola et al., 2000). Dopamine is
Are the same mechanisms used to search internal and both excitatory and inhibitory, depending on the recep-
external spaces? In humans, there are three kinds of evi- tor subtype found at the membrane: the D1 subtype is
dence for such a generalized cognitive search process: typically excitatory while the D2 subtype is typically
priming between domains, the general nature of execu- inhibitory. Together, activation of the dopamine receptor
tive control, and the neurobiology of cognitive control. subtypes enhances the signal to noise ratio of a neural
Evidence of priming between domains is provided in message. This facilitates activity locally while inhibiting
a study aimed at priming search behavior between a activity more globally (Floresco et al., 1996; Seamans
visuospatial search task and an internal lexical search and Floresco, 1998). These neuromodulatory microcir-
task (Hills et al., 2008). People who first foraged in a cuits are found throughout the midbrain (Dani and Zhou,
clustered spatial environment (versus a diffuse spatial 2004)—a region associated with the control of attention
environment) subsequently searched as if there were and directly connected with the prefrontal cortex, a re-
more (or fewer) words in a group of letters in a lexical gion typically associated with goal maintenance (Miller
search task. This priming required search—simply har- and Cohen, 2001). In vertebrates, dopaminergic neurons
vesting resources from known locations was insufficient in the midbrain are required for learning the relations
to produce priming (Hills et al., 2010). Thus, the search between unconditioned and conditioned stimuli, and
process used to forage in space appeared to be subse- thus for forming associative chains between actions and
quently used to forage in an abstract lexical space. outcomes (Daw and Doya, 2006). They are also in-
A second kind of evidence for shared processes is volved in enhancing hippocampus-dependent memory
provided by a cognitive construct called executive ca- persistence (McNamara et al., 2014), which we will
pacity, which has been shown to be involved in both show in the next section is a central component of em-
external and internal search (Longstaffe et al., 2014; bodied prospective foraging by vicarious trial-and-error.
Hills and Pachur, 2012). One aspect of executive capaci- The above provides evidence at three different levels
ty can be measured by working memory span. Working —behavioral priming, cognitive modeling, and neuro-
memory span is a measure of the amount of independ- biology—for a relation between external and internal
ent pieces of information that can be held in short term foraging. Though not decisive, the evidence provides
memory, which is hypothesized to be a measure of peo- substantial support for the view that goal-directed cog-
ple’s ability to maintain top-down (‘executive’) focus nition and cognitive control more generally are evolu-
on specific goals while inhibiting distracting informa- tionary descendants of spatial foraging (Hills, 2006).
tion (Engle and Kane, 1999). Working memory span is Indeed, dopaminergic modulation facilitates persistent
seen as a general cognitive capacity associated with top- or stereotypic behaviors across species (Barron et al.,
down control of behavior, and associated with a variety 2010), for example, modulating area-restricted search in
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 373

Caenorhabditis elegans (Hills et al., 2004), response to area of the midbrain purported to be involved in epi-
cocaine in Drosophila melanogaster (Bainton et al., sodic memories in humans and requiring dopaminergic
2000), exploratory behavior in the rat (Fink and Smith, modulation for both flexibility and encoding of long-
1980), and cognitive flexibility in humans (Chermahini term memories (Shohamy and Adcock, 2010; see also
and Hommel, 2010; Cools and D'Esposito, 2009; Win- McNamara et al., 2014). The hippocampus is also well
stanley et al., 2012). Furthermore, both external and understood as the vertebrate brain’s spatial representa-
internal foraging require balancing the trade-off be- tion system (e.g., at the level of place cells and grid
tween exploration and exploitation, and appear to share cells, see Moser, Kropff, and Moser, 2008).
similar behavioral features and biological mechanisms Animals replay neural activation associated with past
for doing so (e.g., area-restricted search, (Hills et al., events, at least in part by reactivating neurons in the
2015). The capacity to use these mechanisms for more hippocampus. This appears to be associated with both
general goal-directed behavior is likely to be a derived memory consolidation and goal-directed planning (re-
trait borrowed (i.e., exapted) from its more spatial, an- viewed in Pezzulo et al., 2014). This was initially ob-
cestral trait; this would explain observations that dopa- served during periods of sleep, when rats exposed to
mine is strongly associated with “reward,” “novelty,” mazes during the day were later found to replay se-
and “information” (e.g., Costa et al., 2014; Bromberg- quences of activation in their hippocampus concordant
Martin and Hikosaka, 2009). This may also explain do- with those observed during their waking experience
pamine’s involvement in a wide range of goal-directed (Skaggs and McNaughton, 1996; Wilson and McNaugh-
behaviors (especially in vertebrates), but involvement in ton, 1994). These sequences of replay could cover dis-
only a narrower subset of foraging-related behaviors tances greater than 10-m (Davidson et al., 2009), and
across species (e.g., Barron et al., 2010). From an evo- disruption of them impaired spatial learning (Ego-
lutionary perspective, few behaviors are as widely Stengel and Wilson, 2010). Similar patterns of activa-
shared as foraging with as clear an indicator of common
tion have subsequently been found in animals during
descent as that provided by dopaminergic modulation.
awake states (Carr et al., 2011; Diba and Buzsáki, 2007;
Though the evidence that internal and external search
Foster and Wilson, 2006), and these are associated with
share a common evolutionary pathway is not incontro-
choice points prior to decision making, consistent with
vertible, we know of no more plausible hypothesis nor
vicarious trial-and-error learning. Here, rats show acti-
of any proposed alternatives.
vation patterns in their hippocampus that are associated
4 From Internal Foraging to Embo- with points in the maze in front of their current position
(Johnson and Redish, 2007). These patterns or “sweeps”
died Prospective Foraging
progress in series ahead of the animal’s current location,
Thus far, we have provided evidence that foraging in and terminate with activation in reward centers of the
internal and external spaces shares common behavioral brain, such as the ventral striatum (Johnson et al., 2007;
and biological features. However, internal foraging pro- Pennartz et al., 2011; Pezzulo et al., 2014).
vides something that external foraging does not: a ca- The most parsimonious explanation for the observed
pacity to deliberate. As noted in the introduction, even data prior to decision making is that animals use their
short range, stimulus-response relationships provide current context to activate sequences of memory that
animals with the ability to predict future outcomes that provide information necessary for future planning. For
increase the probability of short-term rewards. Richer example, in an experiment by Tolman and Gleitman
internal representations, such as those that support in- (1949), a rat was allowed to freely explore a T-maze,
ternal foraging, should provide better predictive capaci- which had a light and a dark region (both containing
ties. In this section we focus on a kind of internal for- food) at the goal boxes at the end of each passage. After
aging for which there is evidence in nonhuman animals, exploring, the animal was moved to a dark chamber,
which we call embodied prospective foraging. This is separate from the original T-maze, where it experienced
the form of internal foraging in which searching in- electric foot shocks. Finally, the animal was placed once
volves much the same processes, and draws on much again in the T-maze, where it moved to the light goal
the same abilities, that would be involved in actually box. One explanation for this behavior is that the rat
searching on the ground, with memory standing in for could look ahead in the T-maze down the arm associ-
experience. In identifying the evidence for embodied ated with the dark goal box, recall the dark goal box that
prospective foraging, we focus on the hippocampus, an then activated memories of the foot shock, and avoid
 374 Current Zoology Vol. 61 No. 2

that arm (Hesslow, 2012). Recent evidence from Gupta next section, damage to the hippocampus is also associ-
et al. (2010) found that animals who experienced two ated with a degraded sense of self (Hassabis et al., 2007;
routes, one with high frequency and another with low Corkin, 2002).
frequency, did not preferentially recall the more experi- The observation that humans and non-human animals
enced sequence, but showed activation more evenly can, without actually taking an action, fire-up neural
across all possible future routes prior to making a deci- activation patterns that resemble the patterns of activa-
sion—as if they were exploring alternatives. tion they would have when actually taking that action is
In addition, forward looking activation patterns in the the basis of the simulation theory of cognition (re-
hippocampus appear to take one route at a time (John- viewed in Hesslow, 2012). This theory is based on two
son and Redish, 2007). In other words, the patterns of key observations, for which there is ample experimental
activation do not indicate a general spreading of parallel support: 1) That activation of the motor and sensory
activation across all possible future routes, but are iso- cortex during non-action resembles that during real ac-
lated to plausible routes, each in turn (Pezzulo et al., tion, except that the execution of motor actions are sup-
2014). These forward looking activation patterns are pressed, and 2) that this activation tends to produce
most frequent when the animals have limited experience representations of outcomes in proportion to the real
with a particular choice; as they gain experience with a likelihood of occurrence. Evidence that the action-like
choice, and especially when one of the choices is fa- processes associated with non-action have much in
vored over another, the rats engage in fewer forward common with real actions has been observed both with
looking patterns of activation (Hu and Amsel, 1995; respect to the relative time it takes to imagine actions—
Johnson and Redish, 2007). The forward looking acti- for example, in mental rotation tasks (Shepard and
vation patterns also predict goal-directed future action Metzler, 1971)—as well as in relation to the neural ac-
in environments with multiple alternative routes, and tivation patterns associated with particular actions—
they can generate novel routes to known goals (Pfeiffer for example, when piano players play or imagine play-
and Foster, 2013). This further indicates that these for- ing music (Meister et al., 2004).
ward sweeps of activation are not simply recall of past In humans, research on the ability to predict the fu-
experience, but patterns of activation that foreshadow ture often focuses on episodic future thinking (Atance
actions in a goal-directed manner. and O'Neill, 2001), which is understood as the fu-
The above evidence suggests that the animals studied ture-directed counterpart of episodic memory and
can predict the outcomes of possible future actions by sometimes also referred to as ‘mental time travel’
making use of an internal representation to search for (Schacter et al., 2007; Suddendorf and Corballis, 2007)
exploitable resources in embodied prospective foraging. or ‘self-projection’ (Buckner and Carroll, 2007). Epi-
It also suggests that animals actively explore internal sodic future thinking is held to require metacognition
representations with continuous movement through a (i.e., thinking about thinking), autonoetic consciousness,
metric ‘cognitive’ space in much the same way that we concepts of self and time, abilities to identify with one’s
described for external and internal foraging in the pre- future self, and mindreading in the form of an under-
vious sections. standing that your future self may have needs and men-
Though more difficult to study, there is considerable tal states different from those you now have (Sudden-
evidence that humans use a similar process involving dorf and Corballis, 2007; see also Perner et al, 2007).
hippocampal-cortical interactions—along with a num- Perhaps unsurprisingly, evidence for episodic future
ber of other areas (see work on imagination networks in thinking so conceived is largely or entirely confined to
Hassabis et al., 2007)—to imagine future events (Buc- humans. Some have argued that there are forms of recall
kner, 2010; Conway et al., 2003). For example, patients that, although demanding less than full blown episodic
with damage to the hippocampus, like H.M. (Scoville, memory, do involve re-experiencing events and places
1968) and D.B. (Klein, Loftus and Kihlstrom, 2002), rather than merely retrieving information about them
have difficulty imagining future events even though (Clayton and Russell, 2009; see also Conway, 2005).
they may retain general knowledge about the future. The existence of embodied prospective foraging in hu-
Further, episodic future thinking in humans is associ- man and non-human animals implies, relatedly, that
ated with the hippocampus—and other brain regions— capacities to use memory to guide actions and to predict
and directly influences decision making (Peters and their outcomes do not always require episodic future
Büchel, 2010). Importantly for what we discuss in the thinking at the level observed in humans.
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 375

In sum, evidence from vertebrate brain activity sug- specifies a route that could plausibly be taken and use
gests a clear relationship between internal foraging and the subjective gains associated with such combinations
future action. The evidence further suggests that this in deciding which route to take. But this, we assume,
internal foraging can involve simulation of alternative does not happen because—in this case—it would be
routes prior to motor execution of choice. In the next adaptively useless. Rather, each prospective foraging
section we investigate potential consequences of this trip is treated as a single unit because it represents a
relationship for understanding the origin of the self as it coherent set of possible actions and the associated likely
features in autonoetic consciousness. subjective gains. Deciding between two routes by means
of embodied prospective foraging therefore involves an
5 The Primal Self as A Consequence
ability to segment sequences of points of view in such a
of Embodied Prospective Foraging way that each sequence represents an interval of a pos-
We have just seen evidence that a variety of animals sible future life. Each of these coherent sequences of
are capable of embodied prospective foraging. As we points of view we will call a q-self, because each repre-
stressed, embodied prospective foraging is likely to de- sents a query in the internal foraging space that could
mand far less cognitive sophistication than full blown represent a future self (see Fig. 3).
episodic future thinking, at least as often understood. How could embodied prospective foraging enable an
What embodied prospective foraging does demand, animal to decide which of several routes to take? Con-
however, are the ingredients for the construction of a sider what we will call the naive inhibition model (Fig.
primal self. Or so we shall argue in this section. 4): When the animal is unable to select a route, the last
Because embodied prospective foraging involves phases of action preparation are inhibited so that cogni-
simulating actions not on the basis of actual perceptual tion no longer leads to bodily movements, but only to
inputs but rather on the basis of memories associated their simulation. A decision is made at random and em-
with places, objects, and events, this kind of internal bodied prospective foraging occurs. This leads to reac-
foraging involves re-activating experiences of things tivation of past associations in response to stimuli in the
which enable at least some of the actions that actually current context, which can then be input back into the
experiencing those things would enable. We assume for cognitive system to generate further associations (e.g.,
now that these memories must specify a point of view Hesslow, 2002) until a plausible sequence of actions
from which to act in much the way that perceptual ex- and outcomes is generated. After some time the simula-
periences do. (We consider how our view would change tion terminates. If the animal remains unable to select a
if this assumption were false in the next section.) Em- route, another decision is made at random and simula-
bodied prospective foraging therefore involves an abili- tion occurs again—further sampling possible sequences
ty to switch between the animal’s actual point of view from memory (Pezzulo et al., 2014). Once sufficient
and points of view associated with the animal’s memo- information is accumulated to make a decision, motor
ries. and sensory areas are disinhibited and an action is taken.
In addition, embodied prospective foraging involves
an ability to internally generate temporally extended
sequences of points of view. To see why, consider an
individual who uses embodied prospective foraging to
decide which of two routes to take by mentally explor-
ing (or, as we shall sometimes say, ‘querying’) the first
and then mentally exploring the second. The decision of
which route to actually take depends on which route
yields the greatest subjective gain during embodied
prospective foraging. It thus involves distinguishing Fig. 3 During internal foraging, the animal inhibits ac-
between two sequences of simulated actions, each in- tion and queries the outcome of potential choices by acti-
volving a point of view that moves continuously throu- vating memories in a way that reveals structure
These queries appear to be associated with coherent points of view,
gh space. In principle it would be possible to combine
labelled here as q-selves, which each involve experiencing a stream of
arbitrary pieces from the two prospective sequences of events (dotted lines). Here, the p-self is located at the actual position
points of view irrespective of whether their combination of the animal, as further explained in the text.
 376 Current Zoology Vol. 61 No. 2

Figure 5 introduces the crude self-actuating model,

which is an attempt to improve on the naive inhibition
model. This improved model states that embodied pro-
spective foraging involves three processes. First, there
is a clone process that produces some form of copy or
marking of the animal’s current mental states, action
capacities, and memories. The clone process need not
Fig. 4 The naive inhibition model of embodied prospec- be taken literally—rather, it is a process that marks a
tive foraging separation between the simulated and actual animal.
The q-self represents a query into memory, leading to internally gene- Second, there is a simulation process in which the ani-
rated sequences that provide predictions about the outcomes of future mal internally forages, creating a coherent sequence of
actions. Inhibition prevents sensory input and motor output, but still
allows for updating during embodied prospective foraging.
possible experiences. As noted above, this sequence is a
q-self. During the simulation, experiences accrete to the
This naive inhibition model makes few assumptions q-self: the animal itself does not become sated, or
in that it requires no novel representational capacities on falsely believe, as a consequence of the process of in-
the part of the animal, only simulated action guided by ternal foraging, that resources have been depleted at one
selective activation of memories associated with the location relative to another. Third, there is a merge
present context. This is sufficient to explain the data process that occurs when a q-self terminates. This in-
described above (e.g., Johnson and Redish, 2007; Gupta volves the animal updating some beliefs and prefer-
et al., 2014; Pfieffer and Foster, 2013; Pezzulo et al., ences, but not others, based on the outcomes of the ac-
2014). However, the naive inhibition model is funda- tions associated with the q-self. For example, the animal
mentally maladaptive for behavior. This is because, should update with information about the availability of
according to the naive inhibition model, whenever an resources, but should not update with information that
action results in an outcome, the animal’s behaviours may have changed during the internal foraging process
and representations of its needs and environment are (such as resource depletion, mating encounters, or the
updated irrespective of whether the action is simulated presence of an aerial versus a ground predator).
or actual. But outcomes can occur in simulated action The crude self-actuating model may be incorrect in
without it being beneficial for the animal to update its many respects, but it exemplifies something we take to
actual behaviors and representations. This is true even be a necessary feature of any adequate model: it re-
though the same outcomes occurring in actual action quires the animal to distinguish between those se-
would require updating behaviors and representations. quences of points of view associated with embodied
For example, resources can become depleted, the ani- prospective foraging (the q-selves) and those sequences
mal’s position can change, predators can appear or dis- of points of view associated with actually foraging. As
appear, mating can occur, and so on. If the naive inhibi- we stressed above, this requirement appears to be nece-
tion model were correct, embodied prospective foraging
would cause the animal to acquire incorrect representa-
tions about its environment.
The failure of the naive inhibition model suggests
that if embodied prospective foraging is to enable an
animal to decide which of several routes to take, then
the animal must, in updating behaviours and representa-
tions, distinguish simulated actions and outcomes from
their actual counterparts. Let us call any model involv- Fig. 5 The crude self-actuating model of embodied pro-
spective foraging
ing such a distinction a self-actuating model. Although
Clone indicates a process where some separation is made between
nothing in our account hinges on which self-actuating simulated and actual events, such that the animal can simulate action-
model is correct, it is helpful to consider a particular event sequences to inform decision making. This need not be a literal
model in order to make it clear that distinguishing copying. The merge process indicates the place where information
from the simulation is used to update the state of the actual animal’s
simulated actions and outcomes from actual ones does
beliefs. The p-self does not necessarily change its state during the
not require sophisticated perspective taking, temporal simulation, but is used to guide appropriate updating in the merge
concepts or theory of mind. process.
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 377

ssary given that some but not all of what happens in of subjective rewards or a subject of experiences. This
embodied prospective foraging should result in updates follows from the fact that, as noted earlier, effectively
to the animal’s behaviours and representations of its merging the animal’s actual and simulated behaviors
needs and environment. cannot simply be a matter of copying information asso-
Where this requirement is met, as in the crude self- ciated with the q-self back onto the p-self, for this
actuating model, embodied prospective foraging entails would produce errors in the animal’s representation of
two things. First, the animal, either simultaneously or in the state of the surrounding world. The merging opera-
close temporal proximity, activates multiple coherent tion therefore involves distinguishing either what the
sequences of points of view. Some of these sequences q-self experiences from what the p-self experiences or
are q-selves, which are associated with simulated ac- the q-self’s rewards from the p-self’s rewards. The
tions and experiences. But one point of view is an- p-self—in that it has short-lived persistence, a point of
chored in reality in a way that the others are not: it view, and mental states (in some minimal sense)—pro-
specifies the actual animal. This matters: the animal’s vides the primal version of the self, Self 0.1.
actually eating is linked to its survival in ways that its This distinction points to a further interesting prob-
simulated eating is not. Second, in updating its behav- lem in identifying what the q-self actually returns to the
iours and representations of its needs and environment, animal, via the merge process. Numerous cognitive
the animal distinguishes one of these sequences of models of decision making hypothesize that exploratory
points of view as its actual point of view from those that processes return values that summarize the world along
are merely q-selves. some set of dimensions (e.g., risk and reward, Hau et al.,
In treating the sequences of points of view that are 2008; Tversky and Shafir, 1992). For cognitive level
merely q-selves differently from the way it treats the explanations in the laboratory, this is perfectly plausible
sequence of points of view associated with the actual and is a powerful predictive tool. However, natural en-
animal, the animal is manifesting a precursor of self- vironments vary along countless dimensions (e.g., effort
awareness. The animal is treating one of the sequences required, predator presence, kinds of predators, escape
of points of view as itself: distinguishing the points of routes, kinds of resources, shortcuts to other resources,
view associated with actual actions and actual conse- number and quality of competitors, and so on) and may
quences from other points of view just is distinguishing require more sophisticated expectations. A merge proc-
the self from things that are not the self. This gives rise ess that returned a single value would not prepare an
to a primal notion of a self or self-like entity, which we animal for possible action-event outcomes in ways that
shall call the p-self. The p-self is created when the ani- a more richly valued system would—nor would it ap-
mal determines which sequence of points of view pear to fit with the evidence. For example, imagined
should be treated as the actual animal. actions can influence mental and physiological states
The existence of the crude self-actuating model of such as anxiety and heart rate (Taylor et al., 1998; De-
embodied prospective foraging as a hypothetical possi- cety, 1996). However, taken to its extreme, a merge
bility offers additional insight into what is meant by the process that included all relevant information would not
p-self. The p-self is unlike the notion of self often be any different than a process that returns the simula-
thought to be necessary for mental time travel (Sud- tion. Here, we cannot offer a solution to this problem,
dendorf and Corballis, 2007), nor is it a pre-reflective but simply note that the results of the merge process
account of the self often proposed to lie at the heart of determine what the q-self returns to the animal. It can-
first person experience (Block, 1995; Ghallager, 2000; not simply replace the p-self—as it would in the naive
Metzinger, 2004; Panksepp, 1998; Vandekerckhove and inhibition model—but it may also be maladaptive to
Panksepp, 2009; see also Tagini and Raffone, 2010). In reduce the q-self experiences to a single value.
particular, the p-self is temporally short-lived relative to Embodied prospective foraging and the crude self-
the animal’s lifespan, because it is only required during actuating model offer an explanation of how foraging
activation of q-selves. The p-selves created across in- leads to the requirement for a primal self, thus poten-
ternal foraging events separated in time need not be tially providing clues to the origins of autonoetic con-
cognitively represented as the same p-self. Invoking a sciousness or self-awareness. Moreover, consistent with
p-self therefore does not require that the animal sud- the evidence provided in previous sections, embodied
denly envision itself as having a life. prospective foraging allows the animal to look ahead of
Second, the p-self is in some minimal sense a bearer its current position, to navigate over past experiences
 378 Current Zoology Vol. 61 No. 2

and potentially chain these together in novel ways, and campal damage. Thus, autonoetic consciousness might
to deliberate over possible outcomes in a way that offers be no more than an evolutionary consequence of em-
behavioral flexibility. In addition, the p-self/q-self dis- bodied prospective foraging: whatever mechanisms
tinction allows the animal to do this in a way that does enable animals to distinguish actual from simulated ac-
not confuse imagined experience with real experience tions in prospective foraging may also be needed for
(e.g., see Hassabis et al., 2007). If the q-self finds and autonoetic consciousness.
eats all the strawberries, the p-self allows the animal to The present work also raises a number of questions.
forage as if the strawberries are still there. First, to say that an animal is capable of embodied pro-
Since the p-self has minimal cognitive requirements, spective foraging implies that it can recall past memo-
it is reasonable to expect many kinds of agents, includ- ries sufficient to guide (simulated) actions, which sug-
ing artificial agents such as robots, could have a p-self gests that the recalled information may specify a point
(see Holland and Goodman, 2003). The core ingredients of view. Progress in determining whether, for various
are an ability for embodied prospective foraging and an animals, recalled information can specify a point of
ability, in making decisions based on such foraging, to view may be made by measuring their abilities in per-
distinguish sequences of points of view which are me- spective taking tasks (e.g. Vauclair et al., 1993; Mauck
rely q-selves from its actual points of view. Any cogni- and Denhardt, 1997; Burmann et al, 2005). Though we
tive system with these properties has a p-self, whatever see the reactivation of points of view as parsimonious
the mechanism used to make the distinction. with past research, the degree to which this is true is an
Finally, we note that the notion of self produced dur- open question. Animals could encode information in
ing embodied prospective foraging solves a problem in ways that are degraded or symbolic. However, we do
relation to the adaptive function of the self. In particular, not think this would alter the need for a p-self. Even if
if we are willing to entertain the idea that the earliest embodied prospective foraging were entirely symbolic,
versions of self-awareness, however humble, arise out if the symbolic updating that occurs during internal
of a process of embodied prospective foraging, then this foraging alters the animal’s correct representation of its
suggests that the primal function of the self was to mark current state, then it must distinguish the real from the
the distinction between actual and simulated action. imagined. How it makes this distinction is not relevant
to our argument, but it is nonetheless an open question.
6 Discussion For example, reality monitoring is the capacity to dis-
The aim of the present work has to been to explore tinguish internally generated information from exter-
the relationship between external foraging and internal nally derived information, and has been shown to in-
foraging, and the potential consequences of the latter for volve comparisons between the quality of the informa-
understanding the evolution of autonoetic consciousness tion produced from the different sources (Johnson et al.,
or self-awareness. Notably, we have arrived at some- 1993). Even though this system occasionally makes
thing like a self, the p-self, which is not derived from errors, it nonetheless solves the same problem that we
many of the elegant and rich accounts of consciousness have outlined here and therefore distinguishes the self in
(e.g., Metzinger, 2004; Markowitsch and Staniloiu, distinguishing the actual from the imagined. Indeed, the
2011; Panksepp, 1998). Rather, the p-self solves a par- presence of occasional errors in real biological systems
ticular kind of problem created by cognitive systems is evidence for the need to distinguish real from imag-
capable of internal foraging, and specifically embodied ined actions: Their confusion (source memory errors)
prospective foraging. leads to false memories and hallucinations (Brainerd
The p-self as a consequence of embodied prospective and Reyna, 2005; Mitchell and Johnson, 2009; Morri-
foraging is also consistent with evidence that damage to son, 2001).
the hippocampus leads to problems in both future think- The current work also asks when it is beneficial for
ing and self representation (Hassabis et al., 2007; Corkin, an animal, or human, to engage in embodied prospec-
2002). Individuals with hippocampal damage appear to tive foraging, or internal foraging of any kind. Uncer-
show degraded autonoetic consciousness, indicating that tainty is one kind of answer, but we may also ask in
some features of neural processing associated with em- what kinds of environments is embodied prospective
bodied prospective foraging are potentially also features foraging adaptive, and why dispositions to engage in it
required for self-awareness. This is particularly telling differ, for example, across individuals. Moreover, we
because long-term memories do not suffer from hippo- may be interested in further understanding how internal
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 379

representations lead to activation of q-selves—that is, tive bacteria. Annual Review of Microbiology 60: 451–475.
what kinds of memories and experiences are most asso- Clayton NS, Russell J, 2009. Looking for episodic memory in
animals and young children: Prospects for a new minimalism.
ciated with embodied prospective foraging. Furthermore,
Neuropsychologia 47: 2330–2340.
it is interesting to ask how many q-selves are appropri-
Conway MA, 2005. Memory and the self. Journal of Memory and
ate and what stopping rules animals use in shifting from Language 53: 594–628.
internal foraging to actually acting. We hope that the Conway MA, Pleydell-Pearce CW, 2000. The construction of
present work offers a challenge to understand these autobiographical memories in the self-memory system. Psy-
questions as well as to understand what cognitive ca- chological Review 107: 261.
Conway MA, Pleydell-Pearce CW, Whitecross SE, Sharpe H,
pacities may arise out of internal foraging more gener-
2003. Neurophysiological correlates of memory for expe-
ally. rienced and imagined events. Neuropsychologia 41: 334–340.
Cools R, D’Esposito M, 2009. Dopaminergic modulation of flexi-
References ble cognitive control in humans. In: Björklund A, Dunnett S,
Atance CM, O'Neill DK, 2001. Episodic future thinking. Trends Iversen L, Iversen S eds. Dopamine Handbook. Oxford: Ox-
in Cognitive Sciences 5: 533–539. ford University Press, 249–260.
Austerweil JL, Abbott JT, Griffiths TL, 2012. Human memory Corkin S, 2002. What's new with the amnesic patient HM? Nature
search as a random walk in a semantic network. In: Pereira F, Reviews Neuroscience 3: 153–160.
Burges CJC, Bottou L, Weinberger KQ ed. Advances in Neural Costa VD, Tran VL, Turchi J, Averbeck BB, 2014. Dopamine
Information Processing Systems Proceedings of Neural Infor- modulates novelty seeking behavior during decision making.
mation Processing Systems 2012, Lake Tahoe, Nevada. Behavioral Neuroscience 28: 556.
Bainton R, Tsai L, Singh C, Moore M, Neckameyer WS et al., Cruse H, Wehner R, 2011. No need for a cognitive map: Decen-
2000. Dopamine modulates acute responses to cocaine, nico- tralized memory for insect navigation. PLoS Computational
tine and ethanol in Drosophila. Current Biology 10: 187–194. Biology 7: e1002009.
Banks A, Vincent J, Phalp K, 2008. Natural strategies for search. Dani JA, Zhou F-M, 2004. Selective dopamine filter of glutamate
Natural Computing 8: 547–570. striatal afferents. Neuron 42: 522–524.
Barron AB, Søvik E, Cornish JL, 2010. The roles of dopamine Davidson TJ, Kloosterman F, Wilson MA, 2009. Hippocampal
and related compounds in reward-seeking behavior across replay of extended experience. Neuron 63: 497–507.
animal phyla. Frontiers in Behavioral Neuroscience 4: 1–9. Daw ND, Doya K, 2006. The computational neurobiology of
Bell WJ, 1991. Searching Behaviour: The Behavioural Ecology of learning and reward. Current Opinion in Neurobiology 16:
Finding Resources. London: Chapman and Hall Ltd. 199–204.
Block N, 1995. How many concepts of consciousness? Behavioral Decety J, 1996. Do imagined and executed actions share the same
and Brain Sciences 18: 272–287. neural substrate? Cognitive Brain Research 3: 87–93.
Brainerd CJ, Reyna VF, 2005. The Science of False Memory. New Diba K, Buzsáki G, 2007. Forward and reverse hippocampal
York: Oxford University Press. place-cell sequences during ripples. Nature Neuroscience 10:
Bromberg-Martin ES, Hikosaka O, 2009. Midbrain dopamine 1241–1242.
neurons signal preference for advance information about up- Ego-Stengel V, Wilson M A, 2010. Disruption of ripple-associated
coming rewards. Neuron 63: 119–126. hippocampal activity during rest impairs spatial learning in the
Buckner RL, 2010. The role of the hippocampus in prediction and rat. Hippocampus 1: 1–10.
imagination. Annual Review of Psychology 61: 27–48. Engle RW, Kane MJ, Tuholski SW, 1999. Individual differences
Buckner RL, Carroll DC, 2007. Self-projection and the brain. in working memory capacity and what they tell us about con-
Trends in Cognitive Sciences 11: 49–57. trolled attention, general fluid intelligence, and functions of
Burmann B, Dehnhardt G, Mauck B, 2005. Visual information the prefrontal cortex. In: Miyake A, Shah P eds. Models of
processing in the lion-tailed macaque Macaca Silenus: Mental Working Memory: Mechanisms of Active Maintenance and
rotation or rotational invariance? Brain, Behavior and Evolu- Executive Control. New York, NY: Cambridge University
tion 65: 168–76. Press, 102–134.
Carr MF, Jadhav SP, Frank LM, 2011. Hippocampal replay in the Ferreira AS, Raposo EP, Viswanathan GM, da Luz MGE, 2012.
awake state: A potential substrate for memory consolidation The influence of the environment on Lévy random search effi-
and retrieval. Nature Neuroscience 14: 147–153. ciency: Fractality and memory effects. Physica A: Statistical
Chermahini SA, Hommel B, 2010. The (b) link between creativity Mechanics and Its Applications 391: 3234–3246.
and dopamine: Spontaneous eye blink rates predict and disso- Fink J, Smith G, 1980. Mesolimbicocortical dopamine terminal
ciate divergent and convergent thinking. Cognition 115: 458– fields are necessary for normal locomotor and investigatory
465. exploration in rats. Brain Research 199: 359–385.
Cheung A, Collett M, Colle TS, Dewar A, Dyer F et al., 2014. Floresco S, Seamans J, Phillips A, 1996. A selective role for do-
Still no convincing evidence for cognitive map use by honey- pamine in the nucleus accumbens of the rat in random foraging
bees. Proceedings of the National Academy of Sciences 111: but not delayed spatial win-shift-based foraging. Behavioural
E4396–E4397. Brain Research 80: 161–168.
Claverys JP, Prudhomme M, Martin B, 2006. Induction of com- Foster DJ, Wilson MA, 2006. Reverse replay of behavioural se-
petence regulons as a general response to stress in gram-posi- quences in hippocampal place cells during the awake state.
 380 Current Zoology Vol. 61 No. 2

Nature 440: 680–683. life span. Developmental Psychology 49: 2396–2404.

Gallagher S, 2000. Philosophical conceptions of the self: Implica- Holland O, Goodman R, 2003. Robots with internal models: A
tions for cognitive science. Trends in Cognitive Science 4: route to machine consciousness? Journal of Consciousness
14–21. Studies 10: 4–5.
Galhardo RS, Hastings PJ, Rosenberg SM, 2007. Mutation as a Hu D AN, Amsel A, 1995. A simple test of the vicarious trial-
stress response and the regulation of evolvability. Critical Re- and-error hypothesis of hippocampal function. Proceedings of
views in Biochemistry and Molecular Biology 42: 399–435. the National Academy of Sciences 92: 5506–5509.
Grünbaum D, 1998. Using spatially explicit models to characte- Janzen DH, 1970. Herbivores and the number of tree species in
rize foraging performance in heterogeneous landscapes. The tropical forests. The American Naturalist 104: 501–528.
American Naturalist 151: 97–113. Johnson A, Redish AD, 2007. Neural ensembles in CA3 tran-
Gupta AS, van der Meer MAA, Touretzky DS, Redish AD, 2010. siently encode paths forward of the animal at a decision point.
Hippocampal replay is not a simple function of experience. Journal of Neuroscience 27: 12176–12189.
Neuron 65: 695–705. Johnson A, van der Meer MA, Redish AD, 2007. Integrating hip-
Harte J, Zillio T, Conlisk E, Smith AB, 2008. Maximum entropy pocampus and striatum in decision-making. Current Opinion
and the state-variable approach to macroecology. Ecology 89: in Neurobiology 17: 692–697.
2700–2711. Kareiva P, Odell G, 1987. Swarms of predators exhibit “preytaxis”
Hassabis D, Kumaran D, Vann SD, Maguire EA, 2007. Patients if individual predators use area-restricted search. The Ameri-
with hippocampal amnesia cannot imagine new experiences. can Naturalist 130: 233–270.
Proceedings of the National Academy of Sciences 104: 1726– Klein SB, Loftus J, Kihlstrom JF, 2002. Memory and temporal
1731. experience: The effects of episodic memory loss on an amne-
Hassabis D, Kumaran D, Maguire EA, 2007. Using imagination sic patient's ability to remember the past and imagine the fu-
to understand the neural basis of episodic memory. The Jour- ture. Social Cognition 20: 353–379.
nal of Neuroscience 27: 14365–14374. Longstaffe KA, Hood BM, Gilchrist ID, 2014. The influence of
Hau R, Pleskac TJ, Kiefer J, Hertwig R, 2008. The description- cognitive load on spatial search performance. Attention, Per-
experience gap in risky choice: The role of sample size and ception, and Psychophysics 76: 49–63.
experienced probabilities. Journal of Behavioral Decision Macnab RM, Koshland DE, 1972. The gradient-sensing mecha-
Making 21: 493–518. nism in bacterial chemotaxis. Proceedings of the National
Hesslow G, 2002. Conscious thought as simulation of behaviour Academy of Sciences of the United States of America 69:
and perception. Trends in Cognitive Sciences 6: 242–247. 2509–2512.
Hesslow G, 2012. The current status of the simulation theory of Markowitsch HJ, Staniloiu A, 2011. Memory, autonoetic con-
cognition. Brain Research 1428: 71–79. sciousness, and the self. Consciousness and Cognition 20: 16–
Hills TT, 2006. Animal foraging and the evolution of goal- 39.
directed cognition. Cognitive Science 30: 3–41. Mauck B, Dehnhardt G, 1997. Mental rotation in a California sea
Hills T, Dukas R, 2012. The evolution of cognitive search. In: lion Zalophus Californianus. Journal of Experimental Biology
Todd PM, Hills T, Robbins T eds. Cognitive Search: Evolution, 200: 1309–1316.
Algorithms, and the Brain. Strüngmann Forum Reports, vol. 9, McNamara CG, Tejero-Cantero Á, Trouche S, Campo-Urriza N,
J. Lupp, series ed. Cambridge, MA: MIT Press. Dupret D, 2014. Dopaminergic neurons promote hippocampal
Hills TT, Pachur T, 2012. Dynamic search and working memory reactivation and spatial memory persistence. Nature Neuros-
in social recall. Journal of Experimental Psychology: Learning, cience 17: 1658–1660.
Memory, and Cognition 38: 218–228. Meister IG, Krings T, Foltys H, Boroojerdi B, Müller M et al.,
Hills TT, Jones MN, Todd PM, 2012. Optimal foraging in seman- 2004. Playing piano in the mind: An fMRI study on music
tic memory. Psychological Review 119: 431–440. imagery and performance in pianists. Cognitive Brain Re-
Hills TT, Kalff C, Wiener JM, 2013. Adaptive Lévy processes and search 19: 219–228.
area-restricted search in human foraging. PLoS ONE 8: e60488. Metzinger T, 2004. Being no one: The self-model theory of sub-
Hills TT, Todd PM, Goldstone RL, 2008. Search in external and jectivity. Cambridge, MA: MIT Press.
internal spaces: Evidence for generalized cognitive search Miller EK, Cohen JD, 2001. An integrative theory of prefrontal
processes. Psychological Science 19: 802–808. cortex function. Annual Review of Neuroscience 24: 167–202.
Hills TT, Todd PM, Goldstone RL, 2010. The central executive as Mitchell KJ, Johnson MK, 2009. Source monitoring 15 years later:
a search process: Priming exploration and exploitation across What have we learned from fMRI about the neural mechanisms
domains. Journal of Experimental Psychology: General 139: of source memory? Psychological bulletin 135: 638.
590–609. Morrison AP, 2001. The interpretation of intrusions in psychosis:
Hills TT, Todd PM, Lazer D, Redish AM, Couzin I et al., 2015. An integrative cognitive approach to hallucinations and delu-
Exploration versus exploitation in space, mind, and society. sions. Behavioural and Cognitive Psychotherapy 29: 257–276.
Trends in Cognitive Sciences 19: 46–54. Moser EI, Kropff E, Moser MB, 2008. Place cells, grid cells, and
Hills TT, Brockie PJ, Maricq AV, 2004. Dopamine and glutamate the brain's spatial representation system. Neuroscience 31: 69.
control area-restricted search behavior in Caenorhabditis ele- Muenzinger KF, Fletcher FM, 1936. Motivation in learning: VI.
gans. The Journal of Neuroscience 24: 1217–1225. Escape from electric shock compared with hunger-food ten-
Hills TT, Mata R, Wilke A, Samanez-Larkin G, 2013. Mecha- sion in the visual discrimination task. Journal of Comparative
nisms of age-related decline in memory search across the adult Psychology 22: 79–91.
 HILLS TT, BUTTERFILL S: From foraging to autonoetic consciousness 381

Nicola SM, Surmeier J, Malenka RC, 2000. Dopaminergic modu- What is mental time travel, and is it unique to humans? Be-
lation of neuronal excitability in the striatum and nucleus ac- havioral and Brain Sciences 30: 299–313.
cumbens. Annual Review of Neuroscience 23: 185–215. Tagini A, Raffone A, 2010. The ‘I’ and the ‘Me’ in self-referential
Panksepp J, 1998. The periconscious substrates of consciousness: awareness:  neurocognitive hypothesis. Cognitive Processing
Affective states and the evolutionary origins of the self. Jour- 11: 9–20.
nal of Consciousness Studies 5: 566–82. Taylor SE, Pham LB, Rivkin ID, Armor DA, 1998. Harnessing
Pennartz CMA, Ito R, Verschure PFMJ, Battaglia FP, Robbins TW, the imagination: Mental simulation, self-regulation, and cop-
2011. The hippocampal-striatal axis in learning, prediction and ing. American Psychologist 53: 429.
goal-directed behavior. Trends in Neurosciences 34: 548–559. Thompson GW, Kello CT, 2014. Walking across Wikipedia: A
Perner J, Kloo D, Stöttinger E, 2007. Introspection and remem- scale-free network model of semantic memory retrieval. Fron-
bering. Synthese 159: 253–270. tiers in Psychology 5: 86.
Peters J, Büchel C, 2010. Episodic future thinking reduces reward Tolman EC, 1948. Cognitive maps in rats and men. Psychological
delay discounting through an enhancement of prefrontal-me- Review 55: 189–208.
diotemporal interactions. Neuron 66: 138–148. Tolman EC, Gleitman H, 1949. Studies in learning and motivation:
Pezzulo G, van der Meer MA, Lansink CS, Pennartz CM, 2014. I. Equal reinforcements in both end-boxes, followed by shock
Internally generated sequences in learning and executing goal- in one end-box. Journal of Experimental Psychology 39: 810–
directed behavior. Trends in Cognitive Sciences 18: 647–657. 819.
Pfeiffer BE, Foster DJ, 2013. Hippocampal place-cell sequences Tsoar A, Nathan R, Bartan Y, Vyssotski A, Dell'Omo G et al.,
depict future paths to remembered goals. Nature 497: 74–79. 2011. Large-scale navigational map in a mammal. Proceedings
Plank M, James A, 2008. Optimal foraging: Lévy pattern or proc- of the National Academy of Sciences 108: E718–E724.
ess? Journal of the Royal Society Interface 5: 1077–1086. Tulving E, 1972. Episodic and Semantic Memory. Tulving E,
Raaijmakers JGW, Shiffrin RM, 1981. Search of associative Donaldon W ed. Organization of Memory. New York, London:
memory. Psychological Review 88: 93–134. Academic Press, 381–402.
Rosen VM, Engle RW, 1997. The role of working memory capacity Tulving E, 1985. Memory and consciousness. Canadian Psycholo-
in retrieval. Journal of Experimental Psychology: General 126: gy/Psychologie Canadienne 26: 1–12.
211–227. Tversky A, Shafir E, 1992. Choice under conflict: The dynamics
Schacter DL, Addis DR, Buckner RL, 2007. Remembering the of deferred decision. Psychological Science 3: 358–361.
past to imagine the future: The prospective brain. Nature Re- Unsworth N, Engle RW, 2007. On the division of short-term and
views Neuroscience 8: 657–661. working memory: An examination of simple and complex span
Scoville WB, 1968. Amnesia after bilateral mesial temporal-lobe and their relation to higher order abilities. Psychological Bul-
excision: Introduction to the case HM. Neuropsychologia 6: letin 133: 1038.
211–213. Wehner R, Srinivasan MV, 1981. Searching behaviour of desert
Seamans J, Floresco S, 1998. D1 receptor modulation of hippo- ants, genus Cataglyphis (Formicidae, Hymenoptera. Journal of
campal-prefrontal cortical circuits integrating spatial memory Comparative Physiology 142: 315–338.
with executive functions in the rat. The Journal of Neurosci- Wilson MA, McNaughton BL, 1994. Reactivation of hippocampal
ence 18: 1613–1621. ensemble memories during sleep. Science 265: 676–679.
Shepard RN, Metzler J, 1971. Mental rotation of three-dimen- Winstanley CA, Robbins TW, Balleine BW, Brown JW, Büchel C
sional objects. Science 171: 701–703. et al., 2012. Search, goals, and the brain. In: Todd PM, Hills
Shohamy D, Adcock RA, 2010. Dopamine and adaptive memory. TT, Robbins TW eds. Cognitive Search: Evolution, Algorithms,
Trends in Cognitive Sciences 14: 464–472. and the Brain. Cambridge, MA: The MIT Press.
Skaggs WE, McNaughton BL, 1996. Replay of neuronal firing Vandekerckhove M, Panksepp J, 2009. The flow of anoetic to
sequences in rat hippocampus during sleep following spatial noetic and autonoetic consciousness: A vision of unknowing
experience. Science 271: 1870–1873. (anoetic) and knowing (noetic) consciousness in the remem-
Smith KA, Huber DE, Vul E, 2013. Multiply-constrained seman- brance of things past and imagined futures. Consciousness and
tic search in the Remote Associates Test. Cognition 128: 64–75. Cognition 18: 1018–1028.
Stephens DW, Krebs JR, 1987. Foraging Theory. Princeton: Vauclair J, Fagot J, Hopkins WD, 1993. Rotation of mental ima-
Princeton University Press. ges in baboons when the visual input is directed to the left
Suddendorf T, Corballis MC, 2007. The evolution of foresight: cerebral hemisphere. Psychological Science 4: 99–103.