How to Grow Associative Maps?
October 25, 2011 § Leave a comment
It is probably only partially correct to claim that the borders of the world are constituted by the borders of language. Somehow it seems more appropriate to think that the borders of the world are given by the borders of the gardens, in which the associative maps are growing. Well, I admit, we then would have to discuss what’s been first, those gardens or language. Leaving this aside for a moment, then the most essential problem can be put forward in a simple manner:
How to run the garden of associativity?
Before we start I want to briefly recapitulate the chapter about growth. There we investigated what we called “abstract growth.” We related it to a general notion of differentiation, described and operationalized by the concept of “signal strength length.” Based on that, we explored the possibility of software systems that grow by itself, i.e. without external programmers. Here, we will now proceed the next step by explicating the candidate structure for such a structure. Besides the abstract aspect of growth and differentiation, we also have to keep in mind the medial, or if you like, the communicological aspects of the regulation of the actual growth, which needs to be instantiated into a technical representation without loosing too much of the architectural structure.
Unfortunately we can not follow the poetic implications of our introduction. Instead, we will try to identify all possible ways of how our candidate structure can grow. The issues raised by the concept of associativity will be described in a dedicated chapter.
The structure that we have to choose as the basic “segmental” unit needs to be an associative structure by itself. For that, any machine learning algorithm would do it. Yet, it is not reasonable, to take a closed algorithmic procedure for that job, since this would constrain the future functional role—which is necessarily unknown at implementation time—of the unit too much. Else, it should provide robustness and flexibility. For many reasons, the Self-Organizing Map (SOM) is the best choice for the basic unit. A particularly salient property of the SOM is the fact, that it is a network which can change its (abstract) symmetry parameters, something which Artificial Neural Networks can’t as easily achieve. This means that different types of symmetry breaks can be triggered in a SOM, and in turn, that the topology of the connectivity may change even locally. In this way, a single MAP may separate into two (or several). The attractive property of such separation processes in SOM is that any degree of connectivity between the parts can establish in a self-organized manner. In other words, in the SOM the units may develop any kind of division of “labor” (storage, transmission, control), and any unit can split off and develop into a fully-fledged further MAP.
A probabilistic manifold of networks can grow and differentiate in several different ways; any of the growth patterns we described elsewhere (see chapter about growth) may apply:
- – growth by accretion on the level of atoms, completely independent of context;
- – ordered growth due to needs, i.e. controlled by inner mechanisms, mainly at the “tips” of an arrangement, or similar to that, metameric growth as in the case of worms
- – differentiation by melting, folding and pullulation inside a particular map
Maps that have been split off from existing ones may loose all direct links to its “mother map” after a certain period of time. It would then receive messages through the common and anonymous messaging mechanism. In this way a population of SOMaps will be created. Yet, the entities of this population may develop second-order ties, or in order to use a graph-theoretic term, cliques, just dependent on the stream of data flowing in and being processed by the population.
The additional SOM, whether created through separation or “de novo” by duplication need not work all on the same level of integration. If the data from external sources are enriched by variables about the SOM itself by default, for any SOM in the population, a high-level division of labor will emerge spontaneously, if the whole system is put under time or energy constraints.
It is pretty clear, that this garden of associativity has to run in a fully autonomous manner. There is no place for a gardener in this architecture. Hence, growth must be regulated. This can be effectively achieved by two mechanisms: reinforcement based on usage, and even simple evolutionary selection processes on the basis of scarcity of time, absolute space, or the analogs to energy or supply of matter.
Despite such a system it might appear distantly similar to ant hive or swarm simulations, where the task of the individual entity is that of a single, yet complete SOM, we would like to deny such a relationship.
Of course, the idea of growing SOM have been around for some time. Examples are  or . Yet, these papers or systems did conduct neither an analysis of growth processes beforehand, nor of the broader epistemological context, probably because they have been created by software engineers; hence these approaches remain rather limited, albeit they point to the right direction.