Clusters of assets:
A subset of 2000 or 4000 most actively traded assets has been investigated (more precisely, for each asset we computed the distribution of the total volume of transactions over the different days. Assets were ranked according to the entropy of this distribution and the first 1000, 2000 or 4000 were considered). Click on the links to display the list of assets with their names and SIC codes. The cluster structure of the subset of 2000 assets is available here. Some cluster of assets roughly correspond to economic sectors, whereas others (such as the one labeled 3 on the left) group companies with different economic activities. The size distribution satisfies Zipf's law!
States correspond to groups of clusters. For example all clusters with very small likelihood (in the middle) can be grouped into a single "random" state (click here for the precise correspondance of states and clusters or here for a figure). The frequency of states also shows scale free properties, as shown in the inset of this figure.
Market's performance in different states
The market performs differently in different states: The plot on the left shows the average return <r|w> of assets in different states w. Each asset corresponds to one point in each quadrant and it is coloured according to the correlated sector it belongs to (see above).
Market's predictability
In order to study market's predictability, we consider the symbolic sequence of market states (first column of this file) and compute the transition matrix from state to state in k days. The second eigenvalue l of this matrix defines the time t=-1/log l with which the stationary state is reached. For k=1 this time is well outside a noise background, meaning that significan correlations exist.
The process is not Markovian, because otherwise t would fall off as 1/k and no significant correlation beyond few days would exist (see figure). Instead we find a much slower decay of t with k and significant correlations up to 50 days (see inset of figure). However when this information is used to predict market's returns we find very low predictability (see figure). For more details see the discussion in the paper.
Dynamics after market's crashes
During the period we have studied, two major extreme events occurres: the 27 October 1997 and the 31 August 1998 crashes. The state process w(t) is different before the crash, but is quite similar after it. The strings of states, starting from the day of the crash, read 2136613611... and 2126614633... in the two cases. This is a significant similarity which suggests the existence of a particular dynamical pattern with which markets respond to extreme events.
Plot of the log likelihood of cluster structures with a given number of clusters found in the hierarchical clustering algorithm. For more details go to the data clustering web page.