FluidSOM (Software)

January 25, 2012 § 7 Comments

The FluidSOM is a modular component of a SOM population

that is suitable to follow the “Growth & Differentiate” paradigm.

Self-Organizing Maps (SOM) are usually established on fixed grids, using a 4n or 6n topology. Implementations as swarms or gas are quite rare and also are burdened with particular problems. After all, we don’t have “swarms” or “gases” in our heads (at least most of us for most of the time…). This remains true even if we would consider only the informational part of the brain.

The fixed grid prohibits a “natural” growth or differentiation of the SOM-layer. Actually, this impossibility to differentiate also renders structural learning impossible. If we consider “learning” as something that is different from mere adjustment of already available internal parameters, then we could say that the inability to differentiate morphologically also means that that there is no true learning at all.

These limitations, among others, are overcome by our FluidSOM. Instead of fixed grid, we use a quasi-crystalline fluid of particles. This makes it very easy to add or to remove, to merge or to split “nodes”. The quasi-grid will always take a state of minimized tensions (at least after shaking it a bit … )

Instead of fixed grid, we use a quasi-crystalline fluid of particles. This makes it very easy to add or to remove, to merge or to split “nodes”. The quasi-grid will always take a state of minimized tensions (at least after shaking it a bit … )

As said, the particles of the collection may move around “freely”, there is no grid to which they are bound apriori. Yet, the population will arrange in an almost hexagonal arrangement… if certain conditions hold:

  • – The number of particles fits the dimensions of the available surface area.
  • – The particles are fully symmetric across the population regarding their properties.
  • – The parameters for mobility and repellent forces are suitably chosen

Deviations from a perfect hexagonal arrangement are thus quite frequent. Sometimes hexagons enclose an empty position, or pentagons establish instead of hexagons, frequently so near the border or immediately after a change of collection (adding/removing a particle). This, however, is not a drawback at all, especially not in in case of SOM layers that are relatively large (starting with N>~500). In really large layers comprising >100’000 nodes, the effect is neglectable. The advantage of such symmetry breaks on the geometrical level, i.e. on the quasi-material level, is that it provides a starting point for natural pathway of differentiation.

There is yet another advantage: The fluid layer contains particles that not necessarily are identical to the nodes of the SOM, and also the relations between nodes are not bound to the hosting grid.

The RepulsionField class allows for a confined space or for a borderless topology (a torus), the second of which is often more suitable to run a SOM.

Given all the advantages, there is the question why are fixed grids so dramatically preferred against fluid layouts? The answer is simple: it is not simple at all to implement them in a way that allows for a fast and constant query time for neighborhoods. If it takes 100ms to determine the neighborhood for a particular location in a large SOM layer, it would not be possible to run such a construct as a SOM at all: the waiting time would be prohibitive. Our Repulsion Field addresses this problem with buffering, such it is almost as fast as the neighborhood query in fixed grids.

So far, only the RepulsionField class is available, but the completed FluidSOM should follow soon.

The Repulsion Field of the FluidSOM is available through the Google project hosting in noolabfluidsom.

The following four screenshot images show four different selection regimes for the dynamic hexagonal grid:

  • – single node selection, here as an arbitrary group
  • – minimal spanning tree on this disjoint set of nodes
  • – convex hull on the same set
  • – conventional patch selection as it occurs in the learning phase of a SOM

As I already said, those particles may move around such that the total energy of the field gets minimized. Splitting a node as a metaphor for natural growth leads to a different layout, yet in a very smooth manner.

Fig 1a-d: The Repulsion Field used in FluidSOM.
Four different modes of selection are demonstrated.

To summarize, the change to the fluidic architecture comprises

  • – possibility for a separation of physical particles and logical node components
  • – possibility for dynamic seamless growth or differentiation of the SOM lattice, including the mobility of the “particles” that act as node containers;

Besides that FluidSOM offers a second major advance as compared to the common SOM concept. It concerns the concept of the nodes. In FluidSOM, nodes are active entities, stuffed with a partial autonomy. Nodes are not just passive data structures, they won’t “get updated2 by a central mechanism. In a salient contrast they maintain certain states comprised by activity and connectivity as well as their particular selection of a similarity function. Only in the beginning all nodes are equal with respect to those structural parameters. As a consequence of these properties, nodes in FluidSOM are able to outgrow (pullulate) new additional instances of FluidSOM as kind of offspring.

These two advances removes many limitations of the common concept of SOM (for more details see here).

There is last small improvement to introduce. In the snapshots shown above you may detect some “defects,” often as either holes within a perfect hexagon, or sometimes also as a pentagon. But generally it looks quite regular. Yet, this regularity is again more similar to crystals than to living tissue. We should not take the irregularity of living tissue as a deficiency. In nature there are indeed highly regular morphological structures, e.g. in the retina of the eyes in vertebrates, or the faceted eyes of insects. In some parts (motor brains) of some brains (especially birds) we can find quite regular structures. There is no reason to assume that evolutionary processes could not lead to regular cellular structures. Yet, we never will find “crystals” in any kind of brain, not even in insects.

Taking this as an advice, we should introduce a random factor into the basic settings of the particles, such that the emerging pattern will not be regular anymore. The repulsion principle still will lead to a locally stable configuration, though. Yet, strong re-arrangement flows are not excluded either. The following figure show the resulting layout for a random variation (within certain limits) of the repellent force.

Figure 2: The Repulsion Field of FluidSOM, in which the particle are individually parameterized with regard to the repellent force. This leads to significant deviations  from the hexagonal symmetry.

This broken symmetry is based on a local individuality with regard to repellent force attached to it. Albeit this individuality is only local and of a rather weak character, together with the fact of the symmetry break it helps to induce it is nevertheless important as a seed for differentiation. It is easy to imagine that the repellent forces are some (random) function of the content-related role of the nodes that are transported by the particles. For instance, large particles, could decrease or increase this repellent force, leading to a particular morphological correlates to the semantic activity of the nodes in a FluidSOM.

A further important property for the determining the neighborhood of a particle is directionality. The RepulsionField supports this selection mode, too. It is, however, completely controlled on the level of the nodes. Hence we will discuss it there.

Here you may directly download a zip archive containing a runnable file demonstrating the repulsion field (sorry for the size (6 Mb), it is not optimized for the web). Please note that you have to install java first (on Windows). Else, I recommend to read the file “readme.txt” which explains the available commands.

Tagged: , , , , , , ,

§ 7 Responses to FluidSOM (Software)

  • VM says:

    Hi
    I wrote a SOM in Matlab, but with some modifications comparing to existing SOMs. It doesn’t hold the conditions of fluid SOM as you described here, but normally, in SOM nodes are in the grid and their positions are fixed, while their weight vector is evolving.

    But, in this case, I tried to apply the same idea for the updating of the SOM nodes’ positions. So, the nodes are not fixed in SOM space anymore and they are initialized randomly or can be by considering PCs of original data and they move toward the center of the clusters like the magnetic fields, while their weight vectors are updating as classic SOM as well.

    The attached pictures, are two snapshots of a SOM trained by a “randomized 3 dimensional data set, but with 3 predetermined classes”. It is amazing that the nodes are arranged around the clusters and I think they “grasp” the volatilities as well. So, we no longer needs the original data and each node, itself is not important.

    first pic: random initial SOM
    https://docs.google.com/file/d/0B9Z7rVJcvzQAREp4bmU3VXd4alE/edit
    second pc: trained SOM
    https://docs.google.com/file/d/0B9Z7rVJcvzQAc3NFTVBuOV9FWUU/edit

    Best
    VM

    • monnoo says:

      A nice idea, indeed. Seems to be a mixture between an agent structure and SOM gas. Yet, there is a strong conceptual difference (I also think an important one): the physical node and the property of holding a particular intension (=particular values in the nodes vector) are still linked. In SomFluid they are independent. This does not mean that the agent approach would work less, but it is different.
      Think about the difference of neurons and ants. ants are comprised of a body and of a population of neurons. Both are tied together, a recognized pattern is wandering around AS an ant. In neurons, patches of electrical activity are wandering around, but neurons themselves are almost fixed, moving around and changing morphologically only slowly.
      Yet, both approaches allow for smooth growth!
      (finally do not forget about validation tasks!)

      • VM says:

        well quite different approaches then. So, I think in this way we need different mechanisms for training steps… because, in normal SOM each node is a whole itself, reflecting a possible (probable) state of the observed environment.

        Thanks a lot.

      • monnoo says:

        No. The mechanisms on the level of the handling of the data remain the same. It is just that the informational layer (patterns, vectors, data) is “independent” from the physical layer (agents, bodies, neurons). The SOM update mechanism is just ignorant about the form of the locational arrangement: it takes as it is. The physical growth is triggered by some physical or informational events, but beyond that it is also ignorant about the informational layer. There is a dualism, but the work (ok, one of its effects) of the SOM is precisely to overcome the Cartesian dualism by establishing the said linkage. To be clear: there must be such a dualism!!

      • VM says:

        So far, I didn’t think about SOM in this way :) That’s great!! and besides that I didn’t know this intension in SOM for overcoming dualism.
        But as I understood about Cartesian dualism, there should be two independent, but “interactive” layers, such as mind and matter or informational and physical layer, and in this way in SOM and specially in my case they are “dependent”, since in my case the update mechanism in physical layer (coordinates of nodes and their locations) is the mirror of the update mechanism in the informational layer (weight vectors) and in classic SOM one layer is fixed by design and the other layer is dependent on the other one (by neighborhood function) .
        But I guess if we accept this dualism, the informational layer sh(c)ould be prior to the physical one and so we should be able to materialize the informational layer in different physical ones. Some thing like a gradual change in physic, while mind remains fixed (regardless of its internal change) could be possible.
        I am not sure, if I get the possible realization of this kind of dualism in SOM, or maybe I am wrong about the dualism itself.

      • monnoo says:

        yes, almost! …as far as I understand. I just would not say “materialize the informational layer in different physical one” … it is more that they actualize each other in their own domain.
        Do you already have some tech paper about your approach such that I can understand better? (I am not a mat lab guy…) I will keep it non-disclosed, of course (so… by email if you want at all)

  • VM says:

    Hi
    Sure, no problem.
    I can write it in a pseudo-code format and will send it for you.
    just takes a few times, less than 1 or 2 days, since I think I need to clean the code, its arguments and its parameters.
    Best
    I think I have your gmail from your code.google website. or previous one?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

What’s this?

You are currently reading FluidSOM (Software) at The "Putnam Program".

meta

%d bloggers like this: