A new unsupervised vectorial segmentation method developed for color images is presented. Both spatial and color information have been used during the classification process of the pixels. To overcome the problem of memory space associated with multidimensional histograms analysis and avoid a color requantization, this method is CGIV 2010 and MCS’10 Final Program and Proceedings xv based on a multidimensional compact histogram and an original compact spatial neighborhood probability matrix. The multidimensional compact histogram allows a drastic reduction of memory space necessary for coding color histograms without any data loss. Leaning upon the compact histogram, a spatial neighborhood probability matrix has been computed. It contains all non-negative probabilities of spatial connectivity between pixel colors and allows a spatial distance inside a variable size neighborhood to be defined. In an unsupervised histogram analysis classification process, two phases are classically distinguished: (i) a learning process during which histogram modes are identified and (ii) a second step called the decision step in which a full partition of the colorimetric space is carried out according the previously defined classes. In a standard colorimetric approach, in the second step, a colorimetric distance like Euclidean or Mahalanobis is used. We introduced here a spatio-colorimetric distance taking into account the information of pixel neighborhood colors. This distance is defined as a weighed mixture between a colorimetric distance and the spatial distance calculated from the spatial neighborhood probability matrix. The vectorial classification method is based on previously presented principles, achieving a hierarchical analysis of the color histogram using a 3D-connected components labelling.