Psicothema, 2000. Vol. Vol. 12 (Suplem.2). 500-505

Juan Francisco Roy Delgado, G. J. Dalenoort^* and Alfonso Pitarque Gracia

For the study of psychological processes in cognitive science modelling two general approaches rule nowadays research: Artificial Intelligence (top-down) functional symbolic models, and Connectionist (bottom-up) neural networks modelling. Our goal in this paper is to show that analyzing the theoretical level of description and explanation of this models an important theoretical gap between both is found. Connectionist modelling through neural networks face at present several theoretical problems that have to be accounted in order to build realistic and feasible models of the brain. The lack of biological constraints, network stability or serial behaviour, for example, are relevant issues to bear in mind. Our proposed model, conceptual networks, can be located halfway between the traditional semantic networks and artificial neural networks modelling, and is presented as an attempt of building a necessary bridge between this levels of description, focussing on the correspondence between the functional level and the level that can be modelled by artificial neural networks. The elements and functioning of conceptual networks are based in biological and psychological constraints necessary to build realistic models of actual cognitive processes in brain functioning.

Relevancia teórica y biológica de algunos postulados conexionistas. El desarrollo de redes conceptuales. En el estudio de los procesos psicológicos a través de modelamiento en ciencia cognitiva, dos aproximaciones generales se imponen actualmente en investigación: los modelos funcionales o simbólicos (descendientes), provenientes de la Inteligencia Artificial, y los modelos conexionistas (ascendentes) a través del modelamiento de redes neuronales artificiales. Nuestro objetivo en este trabajo es mostrar que a través del análisis del nivel teórico de descripción y explicación de estos modelos, encontramos una distancia importante de separación. El modelamiento conexionista a través de redes neuronales se enfrenta en la actualidad a numerosos problemas teóricos que deben ser superados con el fin de construir modelos admisibles y realistas de la estructura del cerebro. La carencia de restricciones biológicas, la estabilidad de la red o la conducta serial, por ejemplo, son aspectos importantes que deben ser tenidos en cuenta. Nuestro modelo propuesto, las redes conceptuales, puede ser enclavado entre las redes semánticas tradicionales y el modelamiento a través de redes neuronales artificiales, y es presentado como un intento de construir un puente entre los niveles de descripción del cerebro citados anteriormente, deteniéndonos en la correspondencia entre el nivel funcional y el nivel que puede ser modelado por las redes neuronales artificiales. Los elementos y el funcionamiento de las redes conceptuales están basados en restricciones biológicas y psicológicas necesarias para construir modelos realísticos de los procesos cognitivos auténticos del funcionamiento del cerebro.

The burgeoning interest of several disciplines in artificial neural networks is manifest. This recent and eager process has developed several network architectures and learning algorithms which have proved to be very useful and successful in scientific research (including a long list of disciplines such as psychology, biology, computer science and engineering, medicine or linguistics), and also in industrial and management applications (such as pattern recognition and prediction, in banks and industrial control processes). These structures and algorithms are neurally inspired, but this does not mean that there is a direct correspondence with the real neural processes of the brain.

One of the classical philosophical issues in Cognitive Science is the mind-body problem. Contemporary thinkers say that we don’t know at what precise point the cascades of neuronal firing have an effect on consciousness or belief, nor do we know exactly how this effect is accomplished. This philosophical problem have created a big historical gap, not resolved yet, in cognitive modelling between functional symbolical models for cognitive functioning and the available connectionist models for brain architecture and processes.

The history of neural networks in cognitive science shows huge gaps, and the present revival is still too recent for a coherent doctrine to emerge. The variety of connectionist techniques, models and applications is bewildering, and there is much discussion and a lot of confusion about the proper methods, goals and claims of neural networks modelling (Memmi, 1991).

The basic assumption of Connectionist models is the representation of underlying material/physical/biological processes of cognitive functional properties. In their most recent introductory manual for connectionist modelling McLeod, Plunckett and Rolls (1998) state four basic principles of connectionist modelling:

• The basic computational operation in the brain involves one neuron passing information related to the sum of the signals reaching it to other neuron.

• Cognitive processes involve the basic computation being performed in parallel by large numbers of neurons.

• Learning changes the strength of the connections between neurons and thus the influence that one has on another.

• Information, whether about an incoming signal or representing the network’s memory of past events, is distributed across many neurons and many connections (McLeod, Plunckett and Rolls, 1998).

But this principles, thought they show a good neurally inspired epistemological stratus, they still are too much simplistic, and they don’t solutionate the mind-body problem, there is still a gap in between. In order to find a solution to this problem we must look the way that at present cognitive modelling assume the relationship between different levels of description in the brain (see table 1).

Connectionist models emerge from a bottom-up strategy. They build models based in the most general and simple neurological constraints of the brain. Connectionist models generally fulfil the following assumptions: Their basic processing unit is the neuron, neurons integrate basic information about the level of their input and pass this information to other neurons. The influence of one neuron on another depends on the strength of the connection between them, a neuron in the brain usually influences the firing of several thousand other neurons, this influence takes place at a connection, called a synapse and makes the other neuron more or less likely to fire. The strength of connection is called the weight of the connection, and learning is achieved by changing the weight between two neurons (see McLeod, Plunckett and Rolls,1998).

Therefore we can conclude that the properties of a system are related (reduced) to the properties of the parts, thus we can say this strategy is reductionistic.

From the standpoint of classical AI symbolic modelling, a top-down strategy is followed, reducing the model description level to the relationship between the behavioural and the functional level. Representations in classical AI models are discrete and arbitrary, they are abstract propositions and not analogue images (Memmi, 1991). AI assumes unquestioningly the existence of mental representations, and models cognition as the processing of symbolic representations. Another assumption of AI is functionalism, the conviction that the level of symbolic representations is the appropriate level of description in order to develop cognitive processing models. Usual analogies in computer science engineering may state that only the software is important, not the hardware. So AI cognitive modelling mainly follows a top-down strategy, assuming that there is no a relevant correspondence from higher-order psychological levels of description to the lower-order biological levels, at least for the goals and interests of an AI (symbolist) cognitive researcher. Therefore the parts are understood in terms of their function for the system as a whole (Dalenoort & de Vries, 1997). The biological substratum is totally ignored and biological constraints are not relevant for cognitive modelling. We consider this strategy of modelling is functionalistic.

Connectionist and symbolist cognitive modelling present a lack of relevant (neuro) biological and psychological constraints. Researchers may select a mathematical learning algorithm or a computational semantic rule before assuming theoretical constraints in their model. From our point of view, while working in cognitive modelling some basic biological and psychological constraints must be assumed before starting to build a realistic and feasible model of cognitive processes. Our model, Conceptual Networks, emerges from this epistemological impositions.

Many connectionist models claim to have given a precise match to data obtained in experimental psychology tasks with human subjects, for example in reading-aloud tasks (Seidenberg & McClelland, 1989), in accounting of language acquisition as past tense learning (Plunkett & Marchman 1996) or early lexical development (Plunkett et al., 1992), and also in neuropsychology research, as for example simulation of deep dyslexia (Hinton & Shallice, 1991) or information processing deficit in schizophrenia (Cohen & Servan-Schreiber, 1992).

There are also claims to have developed simulations of processes that are closer to neurophysiology, like episodic-memory formation in the hippocampus and visual-object recognition in one of the visual pathways of the primate brain, for example. (See McLeod, Plunckett and Rolls 1998, for a wide explanation of these models).

Although the correlation with real data is high, connectionist models can capture the intricacies of brain processes only to a very limited extent, since in essence their architectures and learning algorithms are mainly based in classification. They produce a mapping from a space of input features onto a set of classes, using backpropagation training algorithms and related methods. Or they carry out memory pattern association trough the operation of the Hebbian rule. But it is hard to see how this could work in a natural system, for example the popular training algorithm of backpropagation, and variations thereof, as used in connectionism systems, is an important mechanism for artificial systems, but it is not a plausible mechanism in biological systems. It is based on the assumption that the path from an output neuron to the neurons that made it fire, can be «found» by the system, or can influence the growth of local synapses. Backpropagation allows for specifically strengthening or weakening the contribution from that neuron, so that the chosen output neuron will react more strongly to the stimulus being trained, and the activity of other output neurons for that specific stimulus can be weakened. It is clear that the artificial process of backpropagation is strongly supervised: a trainer is needed to decide if a response of the system is appropriate for the stimulus that was presented, so it does not satisfy the conditions of self-organization, since it seems impossible that the path of excitations leading to a response can be traced back (Dalenoort & de Vries, 1997). Therefore any supervised learning algorithm as backpropagation cannot be quoted definitively as neurally plausible.

Anyway, we may find some models in literature that may be considered neurally plausible. For example some neurophysiologycal models quoted before and described in McLeod, Plunckett & Rolls (1998). These models operate by self-organising competitive learning which is implemented by using lateral inhibition which is a well-known property of cortical architecture.

We consider the concept of self-organization as an important theoretical and biological constraint (Dalenoort, 1989 and Dalenoort & de Vries, 1997). Self-organization as the emergence of global order from local interactions between the elements of a system has to be accounted for artificial neural network architectures, because there is no internal or external programmer in order to control neural processes, the only programmer that can be considered is genetical disposition through natural evolution.

There are other problems that have emerged through the development of neural networks modelling (see Taatgen, 1999). One of this problems is network stability. Neural networks are famous for their capacity to learn, but maintaining this knowledge, as our brain in a marvellous way does, is harder though. In a standard three layer network, training on a certain set of data and adding new information can produce the interference of new information with old, or even partial or total loss of information or knowledge. Since we cannot count on the outside world to orchestrate presentation of new and old information in the way the network, or our epistemological constraints, would like it, McCleland hypothesises this is a function of a different control subsystem, the hippocampus. Another solution is to design networks that are not as susceptible to forgetting as the standard networks. Grossberg´s ART networks are an example of this idea. An ART network first matches a new input with stored patterns. If the new input resembles one of the stored patterns close enough, learning allows the new input to interact with the stored pattern, possibly changing it due to learning. If a new input does not resemble any stored pattern, a new node is created to accumulate knowledge on a new category. In this way, new classes of input patterns cannot interfere with established concepts (Carpenter & Grossberg 1988, Carpenter, Grossberg & Reynolds,1991).

Another problem is serial behaviour. Many cognitive task, most notably problem solving, require more than one step to perform. In order to do this, a possible solution is setting up some control structure in order to store intermediate results. Recurrent networks (see for example, Elman 1993) have this capability in some sense. A recurrent network is a three layer network with an additional «state» that feeds back to the hidden layer of the network. If several inputs are presented in a sequence, for example in processing a sentence, this state can be used to store temporary results.

Artificial neural networks face at present more critics, for example most of classical connectionist networks are trained for a very specific task, and sometimes also for a very specific set of data. If afterwards the system is trained with another set of data, there occurs what is called catastrophic interference. Also the black box effect (interpretation of the functioning of the system), and several implementation technical issues (see for a deep analysis in Spanish: Martin & Sanz, 1997), but neural networks modelling is a young field indeed, and there is no model yet that adds up all this issues in a theoretically feasible and successfully implemented mainframe that will lead to a new scientific paradigm.

A brief theoretical introduction to Conceptual Networks

The proposed model, Conceptual Networks, can be located halfway between the traditional semantic networks and neural networks and focuses on the correspondence between the functional level and the level that can be traditionally modelled by artificial neural networks. The nodes and the edges between nodes of the conceptual network have no labels that can be used by the network. One may even not state that a single node represents a concept, all semantics must be considered to emerge from the activity through self-organization. For the description of brain processes, and the conditions under which cognitive phenomena can be respectively be produced and developed we impose psychological constraints on biological modules, assuming therefore the following levels of description: The neuron (biochemical level), cell-assemblies (built of neurons), excitation loops (branching chains of cell assemblies), subnetworks (modules of excitation loops to represent conceptions and procedural knowledge), and conceptual networks, consisting of subnetworks, and representing the modules underlying cognitive tasks.

Cell-assemblies. For the study of cognitive tasks, the elementary structural (biological) module in a conceptual network is not the classical connectionist neuron, it is a distributed assembly of neurons, a cell-assembly. It comes into existence on the basis of the Tanzi-Hebb learning rule (d’Angiully & Dalenoort, 1996). The connections between these neurons have been formed in the basis of their simultaneous activity. If two neurons are simultaneously active then a connection between them will come into existence. If in the future only one of the two is active then the other will be activated through the established connection. The development of connections on the basis of this mechanism leads to the formation of an assembly of neurons that have a critical threshold. If a sufficient number of neurons is excited from outside, the overall activity in the assembly rises autonomously to a maximum. When the activation level of a cell assembly is above the critical threshold, the corresponding memory trace is assumed to enter consciousness, and activation of eventually connected motor units is effectuated. Cell-assemblies form a network that at the functional level can be said to contain all our memory traces: a conceptual network. In order to stabilize the activity in the network as a whole it is assumed that within a cell assembly there exist inhibitory mechanisms which redress its autonomous excitation growth and which give the assembly its modular character. An autonomous inhibition complements the autonomous growth.

Cell assemblies are in their simplest form a group of neurons that among them have more synaptic interactions than with other neurons outside the assembly. In a learning situation new synapses come into existence, the beginning of the emergence of a cell assembly. High interconnectivity between the neurons of a cell-assembly will lead to a fast growth of the activity of the assembly as a whole. At some time this activity has to stop, when other assemblies (have to) become active. This introduces the need for inhibitory neurons, that will bring down the overall activity of an assembly. The idea of cell-assemblies serves to explain an important empirical fact: The relative robustness of our memories (neural death). This fact also leads to the idea that memory traces and corresponding cell-assemblies are distributed over relatively large areas.

We relate cell-assemblies to the structural level, and the concept they represent to the functional level. Since the cell-assemblies correspond to memory traces or concepts at the psychological level, the network is referred to as a conceptual network (Dalenoort, 1985).

Excitation loops (see figure 1 and 2). Structures of cell-assemblies form loops, that thus exist as biological modules at a higher level of aggregation. The neural activity in such loops display oscillatory behaviour, and it can serve as the material process underlying the functional (information-processing) level. The occurrence of an excitation loop in a conceptual network corresponds to the finishing of an operation within a psychological task. Both binary and ternary loops are under investigation, loops consisting if larger number of assemblies are assumed to be composed out of these smaller loops. Besides the strengths of the connection between the assemblies, a crucial parameter is the degree of backward inhibition: the extent to which an activating assembly is silenced once an activated assembly has reached its critical threshold. Backward inhibition may plan an important role in keeping the activation in loops within bounds. It is a regulating mechanism within the loop module. Excitation loops must be extremely robust given the wide range of conditions (such as cell death) under which information-processing takes place in natural systems. Therefore, the sequence of operations of which a task is composed, is represented s the propagation of excitation loops in a conceptual network. Another important issue in conceptual networks is the physical concept of resonance. In a causal description of a cognitive task the finishing of each operation has to correspond to a dynamically stable state. Such a state of resonance is the principle on which all information processing in the brain is to be explained. Within a conceptual network resonance is represented as a excitation loop. For example, two cell-assemblies keep each other active and start to oscillate. For the oscillation to occur a process of backward inhibition is assumed to be operating, once the excitation level of a cell assembly has risen above its critical threshold, it will start to inhibit the assemblies that it received activation from. In our model backward inhibition (as noted before) will play a crucial role in the selective propagation of excitation loops (henceforth abbreviated to b.i.).

The sequence of operations of which a task is composed, is represented as the propagation of excitation loops in a conceptual network. An essential psychological characteristic of a task is that people develop a «set» for it: they can prepare themselves for the performance of (part of the) task and when a certain (internal or external) condition is fulfilled the (part of the) task is actually executed. A subnetwork representing a «set» is displayed in figure 1. It also illustrates the role of subpopulations of an assembly. The larger circles denoted cell-assemblies whereas the smaller ones refer to the subpopulations that are the carriers of a connection from one assembly to another. From each subpopulation there also exist a connection backward, and therefore a cell-assembly and one of its subpopulations form an excitation loop. In figure 2 the propagation of excitation loops starts as an oscillation of loop (A, A’), which is activated from outside the subnetwork. It then activates the assemblies C and D, but only to a degree that is below the critical threshold, depending on the parameters of the loop (cf. the excitation curves in figure 2 for C and D, for t = 20 until t = 80). In the model excitation processes as this level correspond to a «set». The same or comparable processes may be assumed to underlie the phenomenon of priming.

The coincidence of the activation from two separate sources (loops) does not amount to the addition of their activation levels. In order to obtain a robust mechanism a special decay process was introduced which suppresses excitation that enters a cell-assembly is there is no simultaneous input from a different source. At the neural level this effect of the coincidence of excitation corresponds to the synchronous firing of neurons in different cell- assemblies, cf. the synfire chains studied by Abeles (1991, ch. 7). In some situations strong «priming» is necessary, for others it is necessary that there is none. In general an excitation loop must not propagate without there being coincidence of activation. The subpopulations of neurons, also referred to as «subnodes», are necessary to maintain an excitation loop to represent «set» for incoming contingent excitation. If such subnodes were not present the network would have to be tuned very accurately. It would then lack the robustness characteristic of the neuronal processes in the brain. Since subnodes keep the oscillation in an assembly going they do not have a critical threshold of their own. Only when the oscillation of the loop (B, B’) starts, there is the necessary coincidence of excitation in the assembly C, which triggers the oscillation in loop (C, C’).

The cause of the b.i. from assembly C loop (A, A’) extinguish to a degree determined by the parameters of the loop. This is a characteristic of the propagation of an excitation loop. Within a conceptual network priming is the accumulation of activation in a excitation loop that remains below the critical threshold. Once the activation of a loop exceeds the critical threshold of the excitation level is decreased by b.i. to a given level from which accumulation will start anew. Given such interactions of loops, various patterns of priming are possible. Ratcliff & McKoon (1997) have demonstrated a similar variation of effects of priming.

Subnetworks. Combinations of loops form subnetworks that have significance as components in a tasks analysis. Functionally they are comparable to elementary production rules, to a part of a semantic network, or to the procedures that search such a network in «classical» AI programs. It is characteristic of a conceptual network that both declarative (data elements) and procedures are represented by networks. The analysis of the task of serial recall of a list of items will exemplify some of these functional models. Although this task is a classical psychological subject, its analysis clearly demonstrates the necessity of temporary connections between the cell assemblies in different subnetwork modules. The task requires that the presented items, which are part of a data module, are temporarily coupled to the memory traces for rank order, which are part of a sequence module.

The various subnetworks of the serial recall task are the following, in which we distinguish between declarative and procedural networks:

A declarative item network, including a stylized form of input and output. The process of pattern recognition, for instance how a letter is composed out of elementary features as edges and corners, is not represented explicitly in the model. Procedures in the simulation program emulate the role of pattern recognition within cognitive tasks. Characteristic of the serial recall task is the representation in the item network of properties that determine whether an item is a member of the list to be reproduced.

A procedural network for testing whether a presented item specifies the criteria for the list (this is a task-specific network, e.g. the task instruction may require the reproduction of only the vowels in a list of letters, or only those spoken by a particular voice).

Each functional module is constructed in a way that conforms to principles of self-organization. Therefore within all modules the same structures can be distinguished.

The organization of a procedural subnetwork (see figure 3) is based on the distinction between a single global loop and a number of local loops. Activation of the global loop corresponds to the execution of a procedure as a hole, whereas activation of a local loop corresponds to the execution of an operation within a task. When the global loop extinguishes, the execution of the procedure is stopped. Activation in a local loop may survive the extinction of the global loop and thereby - for some time - provide a point of re-entry when the procedure is restarted.

In each procedural subnetwork the global loop activates, as well as suppresses, local loops, cf. Figure 3. Excitation loops are always in cooperation and competition. However, if the assemblies in a local loop are active above their critical threshold, they will put off this inhibition. The inhibition exerted by the global loop is a fundamental regulating mechanism within a subnetwork module. In the discussion of the different levels at which modules can be distinguished, it was noticed that each level has its proper regulating mechanisms: autonomous inhibition, backward inhibition, and suppression of local loops by a global one. This last mechanism for the control of activation can be generalized for the network as a whole. An arousal control system is assumed to exist that disposes of excitation below the critical threshold. Loops that are active above the critical threshold, however, are able to suppress this inhibition. This regulating mechanism, like the other ones, is «blind» to content and operates solely according to the causal laws of neural excitations. Functional concepts like attention are a product of the interaction of these regulatory mechanisms.

The study of constructed conceptual networks, such as the one for the task of serial recall, is a first step in the construction of such networks, and of the underlying neural networks, the representation and functioning of the whole conceptual network is far too much complex, and will not be displayed here.

Conceptual networks are an effort in order to introduce the necessity of flexibility in network models, from classical static task based connectionist structures to true dynamic adaptive systems. Once the architecture of the system is understood, we can proceed by making these networks emerge according to principles of self-organization, in agreement with the development of the discussed hierarchy of levels and modules.

At present we have developed and tested several models of classical and relevant cognitive processes, like reproduction of series of items and feature visual integration, there is no place in this theoretical introduction for a detailed description of these simulations and experiments. Interested readers must refer to Vries (1995), Vries & Dalenoort (1995), Dalenoort & de Vries (1997), and to forthcoming papers from the authors.

Abeles, M. (1991). Corticonics - Neural circuits of the cerebral cortex. Cambridge University Press. Cambridge.

d’Angiully & Dalenoort (1996) Tanzi’s contribution to modern neuroscience, Brain and Cognition 32, 252-254.

Carpenter, G. & Grossberg, S. (1988). The ART of adaptive pattern recognition by a self organizing neural network. IEEE Computer, 77-88.

Carpenter, G., Grossberg, S. & Reynolds, J.H. (1991). ARTMAP: Supervised real-time learning and classification of non-stationary data by self organizing NN. Neural Networks, vol 4, 565-588.

Cohen, J. & Servan-Schreiber, D. (1992). Context, cortex and dopamine: A connectionist approach to behavior and biology in Schizophrenia. Psychological Review, 99, 45-77.

Dalenoort, G. J. (1985). The representation of tasks in active cognitive networks. Cognitive Systems, 1, 253-27.

Dalenoort, G. H. & de Vries, P.H. (1997). Understanding in terms of correspondences between levels of description, in: C. Tadei-Ferretti (Ed.), Downward processes in the representation of perceptual mechanisms. Proc. of a summerschool, Ischia (Napoli) October 21-26, 1996.

Dalenoort, G. H. & de Vries, P.H. (1997). Self-organization in the brain, in: C. Tadei-Ferretti (Ed.), Downward processes in the representation of perceptual mechanisms. Proc. of a summerschool, Ischia (Napoli) October 21-26, 1996.

Elman, J.L. (1993). Learning and developmenting neural networks: The importance of starting small. Cognition, 48, 71-99.

Hinton, G. & Shallice, T. (1991). Lesioning an attractor network: Investigations of acquired dyslexia. Psychological Review, 98, 74-95.

Martin, B. & Sanz, A. (1997). Redes neuronales y sistemas borrosos. RA-MA: Madrid.

Memmi, D. (1991). A systematic comparison between connectionism and Artificial Intelligence. Cognitive Systems 3-1, January, pp 43-67.

McCleod, P., Plunkett & K., Rolls, E.(1998). Introduction to connectionist modelling of cognitive processes. Oxford University Press: New York.

Plunkett, K., Sinha, C., Muller, M.F. & Strandsby, O. (1992). Symbol grounding or the emergence of symbols? Vocabulary growth in children and a connectionist net. Connection Science, 4, 293-312.

Plunkett, K. & Marchman, V. (1996). Learning from a connectionist model of the acquisition of the English past tense. Cognition, 61. 299-308.

Ratcliff, R. & McKoon, G. (1997). A computer model for implicit priming in perceptual word identification. Psychological review 104 (2), 319-343.

Seidenberg, M. & McClelland, J. (1989). A distributed model of word recognition and naming. Psychological review, 96. 523-568.

Taatgen, N. (1999). Learning without limits. From problem solving towards a unified theory of learning. PhD Thesis.

Vries, P.H. de & Dalenoort, G.J. (1995). The binding problem in distributed systems, in Kappen, B. & Gielen, S (Eds.) Neural Networks: Artificial intelligence and Industrial Applications. Springer, London, 1995, 77-80.

Vries, P. H. De (1995). Downwards emergence in conceptual networks, Cognitive Systems, 4(2), 233-246.