A Randomized Approximate Nearest Neighbors Algorithm

The classical nearest neighbors problem is formulated as follows: given a collection of N points in the Euclidean space R^d, for each point, find its k nearest neighbors (i.e. closest points). Obviously, for each point X, one can compute the distances from X to every other point, and then find k shortest distances in the resulting array. However, the computational cost of this naive approach is at least (d*N^2)/2 operations, which is prohibitively expensive in many applications. For example, naively solving the nearest neighbors problem with d=100, N=1,000,000 and k=30 on a modern laptop can take about as long as a day of CPU time. Fortunately, in such areas as data mining, image processing, machine learning etc., it often suffices to find approximate nearest neighbors instead of the true ones. In this work, a randomized approximate algorithm for the solution of the nearest neighbors problem is described. It has a considerably lower computational cost than the naive algorithm, and is fairly fast in practical applications. We provide a probabilistic analysis of this algorithm, and demonstrate its performance via several numerical experiments.