Department of Neural Information Processing Technologies (DNIPT) is a part of International Research and Training Center for Information Technologies and Systems, National Academy of Sciences of Ukraine and Ministry of Education and Science of Ukraine (IRTC ITS). We inherit and continue developments of Department of Biocybernetics founded in the Institute of Cybernetics by V.M. Glushkov and N.M.Amosov in 1961. The department was and still is one of the most known AI groups of the former USSR.
- The “Bible” of the Biocybernetics Department
- Peculiarities of Amosov's School
- The network of information models (M-network)
- Period of Robototechnics
- Neurocomputers
The “Bible” of the Biocybernetics Department
In 1964 Nikolai Mikhailovitch Amosov formulated a hypothesis on the information processing mechanisms of the human brain. Within this hypothesis he expressed his system-level observations on the brain’s structure and the mechanisms that are made operational by a human’s mental functions. Of principal importance was the fact that it was not the separate structures, mechanisms or functions (such as memory, perception, learning and so on) that became the simulation object, but the brain of the human as a social being – the brain of homo sapiens. Such was the main idea of the monograph “Modeling of Thinking and of the Mind”, published in 1965, which for a couple of decades became the bible for several generations of Department’s researchers (and not only for them).
The reaction of the Soviet scientific circles to the appearance of this monograph can in general be evaluated as a weakly aggressive one. On the one hand, by that time physiologists and the “great philosophers” of physiology were clearly fatigued by the wide discussion concerning the firmness of core postulates of I.P. Pavlov’s theories of higher nervous activity, that was hosted by the “Voprosy filosofii” (“Philosophical Issues”) journal and initiated by a well-known physiologist A.N. Bernstein. In this discussion the traditional physiologists suffered a considerable defeat, albeit not a complete one. Much became “allowed”, in particular, the statement on irreducibility of higher brain functions to lower ones was shaken. On the other hand, psychologists and the “great philosophers” of psychology were by that time thoroughly contaminated by widely accepted cybernetics concepts. Studies in which mental processes were explicitly treated as information processing started to appear, and ideological doctrines that dominated psychology for many years had already been weakened, but remained influential. (It is interesting to note that in those years, psychological publications were still found in bookstores under the heading “The Theory of Marxism-Leninism”.) Under these conditions the publishing of N.M. Amosov’s monograph was not met with official rejection (which would absolutely have not been possible five or ten years ago) and the Department’s work continued without interference. The fact that N.M. Amosov was a deputy of Supreme Soviet of the Soviet Union has apparently played its role: at the time, this was an important argument for “ideological acceptability” of his theories.
The ideas, which N.M. Amosov put forward in his book “Modeling of Thinking and of the Mind” were further developed in his subsequent works (“Modeling of Complex Systems”, “Artificial Intelligence”, “Algorithms of the Mind”, “Human Nature”).
Peculiarities of Amosov's School
On a theoretical level, two main features characterize the research of Amosov School.
The first feature is that not an individual neuron, but a set of neurons organized in a particular way – neuron assembly – is considered to be the core functional element of a neural network, its “principal character”. Given this, the neural network appears now as a structure consisting of a multitude of interacting assemblies, each of which corresponds (and this is a very important point) to some individual image or concept out of a set of images and concepts that participates in forming integrative mental functions realized by the brain. That is, this set participates in the thought process. Thus the neural network turns out to be a network with semantics (a special kind of a semantic network). The origins of the present approach can be traced to the early works of a well-known physiologist D. Hebb, whose main study was published as early as in 1949.An important characteristic of this kind of network is that all of its elements at any point in time are active to some degree. The magnitude of this activity varies in time, reflecting the interaction of concepts represented by the network’s nodes.
The second feature of Amosov school research concerns the introduction of the notion of a specific system for reinforcement and inhibition (SRI) to scientific use. This system is an integral neural network attribute, and in network functioning it plays a role comparable to that of functions of attention in the thought processes. The idea of SRI is entirely original. Using this system allows to introduce a direction component into neural network information processing, and, what is very important, to use value characteristics of information in organizing this processing.
The presence of variable activity of all network's nodes and of SRI control action enables to say that there are two interacting levels of information processing – the subconscious and conscious levels. On the subconscious level occurs a constant interaction and interference of informational concepts represented in the network. In the course of this interaction the activity level of individual network nodes increases to such extent that they are “perceived” – they become the object of “attention” (SRI), which leads to a radical redistribution of activity of all other nodes. SRI acts according to its own rules, which do not allow any individual concept to stay for long in the attention span. This ensures the “shift” of attention to other concepts, and thus imitates the train of “conscious” thought.
The aforementioned features of the starting hypothesis enabled to obtain, within a short period of time, a whole series of nontrivial results in the field of theoretical research of several complex thought functions and of practical verification of theoretical conclusions through computer modeling. Publication of two monographs marks the time limits of this period. One of these (“Modeling of Thinking and of the Mind”) sets the starting positions of the scientific line of research, and the other (“Automata and the Mindful Behaviour”) sums up the result of the specific research studies cycle.
The network of information models (M-network)
N.M. Amosov’s theoretical statements on the brain’s information processing mechanisms and on the principles of complex mental function generation were utilized in creating a special class of neuron-like networks, which was called an M-network. A. Kasatkin and L. Kasatkina proposed the M-network as a tool for modeling the information processing mechanisms in 1966. The M-network elements correspond to integral neural assemblies and are described as nonlinear analog data converters. Every element is associated with a specific concept. The links between elements reflect the interrelation and interaction of real or virtual objects, events, actions, states and so on, which are related to such concepts. Thus the M-network extends the neural network capacities due to introduction of a series of characteristics of semantic networks.
The main feature of an M-network is that excitation transfer procedures are determined on the multitude of its nodes and links. Excitation is a numerical value, which characterizes instantaneous actuality or value of informational concepts presented by the network’s nodes. At any point in time, the state of M-network can be described with excitations distribution of its nodes. A special procedure conducts excitation transformation and transfer from some nodes to others, using the directional links already existing between the nodes. Every connection has a weight and can be reinforcing or inhibiting. A certain part of the M-network nodes is assigned a status of input (receptor) and output (effector) nodes. The actual processing of information coming through the receptor entries occurs in the M-network's central part. This part reflects the knowledge and reasoning of a human specialist within the data domain being modeled. A vital part in the operation of an M-network is played by the system for reinforcement and inhibition, the SRI. At any point in time, SRI selects the most active node, that is, the most topical information, and reinforces the influence of this information on the subsequent processes within the network. SRI selection of one of the output nodes is interpreted as making a decision, which corresponds to the semantics of this particular node.
Computer models of intelligent behaviour, REM (1965-67) and MOD (1968-71), which were developed during this period, allowed to demonstrate that it was in principle possible to create neural networks imitated by mechanisms generating complex mental functions. Specifically, the mechanisms of emotion generation and their effect on behavioural act formation were modeled. REM and MOD were created as prototypes of integral robots capable of independent assessment of their own state (their “body” state) and of environmental conditions, of planning their behaviour and making necessary decisions for the plan implementation or correction. The REM and MOD models were the first major attempts to reflect the psychological aspects of higher species and human behaviour by means of a neural network.
REM’s M-network structure consisted of interrelated substructures, which implemented the functions of perception, conceptual generalizations, emotional evaluations and decision-making. This was the first attempt at a model instantiation of N.M. Amosov’s hypothesis. It was not clear whether it would be altogether possible to organize the M-network in the required way, how complex the task of “adjusting” it would be in order to obtain at least somewhat intelligent behaviour and whether SRI actions would have the necessary effect. The answers to these questions could have been obtained only by creating a functionally non-specialized model. This was the exact reason for choosing a “robot” subject – movement in a conditional checker-board pattern environment containing objects that are “dangerous” and “useful” to REM, that is, everyday behaviour. Model’s conformance to N.M. Amosov’s hypothesis was evaluated based on two criteria: expediency of external behavioural manifestations (of the actual movement) and expediency of “external” reactions, that is, the stimuli for choosing one or another action. Through modification of M-network's structure (or simply of individual connection weights), different “personality” types for REM were being created - aggressive, calm, and timid. The M-network’s structure was not changed during the experiment. Quite a few experiments were conducted, which was not easy at the time (1966-67) - M-220 was the only “large” computing machine in the Institute of Cybernetics, and it was fairly difficult to “get” the computing time. However, after REM it became clear that it is indeed possible, based on N.M. Amosov’s hypothesis, to create neural network structures which generate intelligent behaviour, and, what is more interesting, an intelligent motivation for such behaviour.
The subsequent model (MOD) was implemented already on a more powerful and high-speed computer BESM-6, but since the model itself was more complex, even relatively simple experiments on this machine required 1.5 to 2 hours of computing time. The main distinction from REM was in MOD’s ability to plan its movement and to learn in the process of interacting with the environment.
Period of Robototechnics
The initial orientation of Amosov school research towards complex modeling of mental functions has determined to a considerable extent the “robot-technical” trend of subsequent research conducted in the Department.
It should be noted that in 1980s, the robot-technical subject area became quite popular in the USSR. Moreover, even direct instructions from the party and the government were given, concerning the particular topicality of research in the given direction. Thus a certain niche was created, in which many developers of issues within AI could find a spot. As time went by, however, it turned out that the country’s industry could not ensure effective use of high-intelligence devices offered by the scientists, thus the robot-technical boom slowly subsided. Nevertheless, for several years this line of research was yielding some quite interesting scientific results.
The signs of approaching crisis in neural cybernetics were stimulated by work aimed at obtaining results that could have a practical, applied value. For N.M. Amosov’s Department, this research turned out to be naturally connected to creating prototypes of autonomous mobile robots and to neural network control systems development. It should be said that in the initial stage of this “prototype epic” Nikolay Mikhailovitch Amosov strongly objected to such a digression to hardware. But he was successfully reassured, and as a result, a whole family of robots of this kind was developed and studied. A research member of the Amosov Department, E.M. Kussul, initiated and supervised this line of research (it was him who headed the Biocybernetics department after N.M. Amosov has vacated that position). In 1972-1975, the first autonomous transport robot in the USSR was created, TAIR (see Figure 3). Its development was completed with a successful presentation of the derived results during the Fourth International Conference on Artificial Intelligence in Tbilisi in 1975 (a short-length film about TAIR was shot and demonstrated at the conference). The robot demonstrated purposive movement in natural environment, obstacle avoidance and similar actions. TAIR was a three-wheel power barrow equipped with a system of sensors (rangefinder and tactile sensors). It was controled by a hardware-implemented neural network (the network nodes – special transistor electronic circuits; links between nodes – resistors).
TAIR’s testing was conducted in the park near the Amosov Clinic for heart surgery. (At the time, the department of biocybernetics was located on the clinic's premises and N.M. Amosov headed simultaneously both the clinic and the Department). While moving, the robot was supposed to avoid obstacles, such as people, trees, park benches and so on. Coordinates of a point on the environment gave the target of robot’s motion. TAIR’s pilot research demonstrated that it was in principle possible to create an entirely autonomous robot operated by a hardware-implemented neural network. At the same time, it showed the overall complexity of organization of the robot's interaction with the natural environment, as well as the necessity of using trainable neural networks. The analysis of capabilities for creating teachable robots with neural network control systems was carried out using the MALYSH prototype created in 1979.
Later on several more robots were built, on which various motion control and natural environment interaction circuits were tested. In 1980-1981, a STAR transport robot prototype was built, which was used for working through control algorithms for a “large” transport robot created on basis of a commercial truck loader.
In 1984-1986, analysis and refining of algorithms for control of robot movement in a natural environment were continued using the MAVR prototype. This research work was ordered by the USSR Ministry of Defense and was directed at building an autonomous robot capable of moving under conditions of a complex uneven environment. Unique structural solutions provided MAVR with high maneuverability and reliable protection of control circuits, located inside barrel-shaped wheels. Environmental data came through rangefinders, optical and tactile sensors onto the hardware-implemented (vehicle-borne computer) neural network. Decision about the movement direction or other actions within the decision-making block was made upon processing the input information. The decisions taken were activated by appropriate actuating mechanisms.
The MAVR research marked the end of the “robot-technical” period of the N.M. Amosov School. The obtained results are presented in a composite monograph “Neurocomputers and Intelligent Robots”. It should be noted that this monograph summed up the main results of departmental research work during 1980s, and its contents were well beyond the robot-technical subject-matter itself. In particular, this monograph presented the findings of intensive research work directed at creating methods and practical techniques for using neural networks in building expert systems based on formalization of estimative expert knowledge, which is usually quite vague and difficult to formalize. Creation of a development program system VESTA was the practical outcome of this research work. VESTA supported expert's efforts on autoformalization of its own knowledge in the form of a neural network structure, capable of automatically transforming itself into a decision-making support system.
Neurocomputers
During the same period an innovative line of research was actively developing with the efforts of Doctor of Technical Science E.M. Kussul. It concerned using stochastic methods to model assembly neuron-like networks. E.M. Kussul has put forward and analyzed a new neural network paradigm, which enabled to create neuron-like structures of great versatility. These structures are known as associative-projective neuron-like networks.
In mid-1908s a new term appeared in the field of human brain modeling – “neurocomputer”. It essentially marked a new wave of research and development in the field of neural network methods of information processing, and almost completely replaced the term “neurocybernetics”. The hopes put on early AI systems creation projects were naturally transferred onto neurocomputers, which were perceived of, in a broad sense, as prototypes of an “artificial brain” – an intelligent system supposed to be built and to function analogously to the human brain. The prefix “neuro” stressed the distinction of such a system from a traditional computer, as well as the functional proximity to the brain.
Department member D.A. Rachkovskij is debugging the neurocomputer-PC interaction system (1989)
The actual state of affairs has fairly quickly forced to narrow down the meaning of the term “neurocomputer” to identifying it with artificial neural networks. Most of present research utilizes this term (or the term “neurocomputer”) to indicate the whole range of research work within the approach to AI system development that is based on modeling of elements, structures, interactions and functions of different nervous system levels. In the modern sense, neurocomputer is a specialized software- or hardware-implemented computing device, which imitates the neural network operation.
The first hardware-based neurocomputer in USSR was developed in 1988-1989 on the basis of stochastic assembly neural network ideology. Research was carried out under the supervision of the Doctor of Technical Science E.M. Kussul, to whom Nikolai Mikhailovitch had already handed the Department over. The first neurocomputer prototype (1989), built on domestic element base, was a personal computer add-on. In subsequent prototypes, a more advanced element base was used. In 1992, jointly with a Japanese company WACOM, the latest neurocomputer version was developed and tested on image recognition tasks.
Subsequent departmental research was related to the development of neural network information technologies. Effective neural network classification systems were developed, used in texture recognition, voice-based identity verification, handwriting and connected word recognition and so on. Notwithstanding the applied nature of this research work, the Department preserved the skills instilled by N.M. Amosov: the global approach to problems within Artificial Intelligence, the ability to view the task as a whole and to accumulate experience for “breakthroughs” to come.