Slide 12 of 19
Notes:
It is also possible to study probabilistic clustering by invoking the maximum entropy principle (as in Rose et al. 1995). In few words, rather than looking for a single configuration S which maximizes the likely-hood, one may study classes of probability distributions P{S} on the space of possible cluster structures. The P{S} can be identified by the maximum entropy requirement, i.e. by focusing on those classes of distributions with fixed average likely-hood which have maximal entropy. It is well known and easy to see that this leads to Maxwell-Boltzmann distribution Pb{S} where b arises as a Lagrange multiplier and is the inverse temperature in statistical mechanics. As b increases the distribution Pb{S} becomes more and more sharply peaked around maximum likely-hood solutions. When b goes to infinity maximum likely-hood solutions are recovered. We studied the behavior of probabilistic clustering as a function of b using standard Metropolis algorithm to sample the distribution Pb{S}. We monitored the average energy (which is -log[likelyhood]/N) and a persistence cb which measures the probability that, if two objects are found in the same cluster at some stage of the simulation, they remain in the same cluster even at later stages (see [1] for details). We studied 4 different data sets:
- A synthetic data set generated from our model with gs=0 (no correlations). Our clustering procedure does not identify any structure. This is an important test.
- Financial data sets with two different lengths D (see later and [1])
- A synthetic data set generated from our model with known correlations. The method is able in this case to recover the correct correlation structure.
