Self-organizing incremental neural network (SOINN) is a two layered, competitive learning based neural network which is able to represent the topology structure of input data and cluster online non-stationary data without prior knowledge, and also robust to noise. The incremental nature of SOINN enables it to learn novel patterns from data stream without affecting previously learned patterns. In this respect, it is appropriate to expect that SOINN could serve as a general approach to unsupervised learning problems. With some modifications, SOINN could handle other kinds of learning tasks such as supervised learning, associative memory, pattern based reasoning and manifold learning as well. SOINN has been used in many kinds of applications including robotics, computer vision, expert systems, and anomaly detection. This paper presents a survey of its basic ideas, improvements and applications.