In recent years, the localization and tracking of moving targets have been widely used in scenes including indoor navigation, smart homes, security monitoring, and smart medical services. Radio frequency (RF)-based contactless localization and tracking have attracted extensive attention from researchers. Among them, the commercial IR-UWB-based technology can achieve target localization and tracking at low costs and power consumption and has strong development potential. However, most of the existing studies have the following problems: 1) Limited tracking scenes. Modeling and processing methods are only for outdoor or relatively empty indoor scenes under ideal conditions. 2) Limited movement states of targets and unduly ideal modeling. 3) Low tracking accuracy caused by fake moving targets. To solve these problems, this study proposes a moving target tracking method using IR-UWB on the basis of understanding the composition of the received signal spectrum in multipath scenes. First, the dynamic components of the originally received signal spectrum are extracted. Then, the Gaussian blur-based multipath elimination and distance extraction algorithm is employed to eliminate multipath interference, which only retains primary reflection information directly related to the moving target and therefore accurately obtains the distance variation curve of the target. Subsequently, a multi-view fusion algorithm is proposed to fuse the distance information of the devices from different views to achieve accurate localization and tracking of a single freely moving target. In addition, a real-time moving target tracking system based on the low-cost commercial IR-UWB radar is established. The experimental results in the real indoor home scene show that the error between the center position of the human body estimated by the system and the real motion trajectory is always within 20 cm. Moreover, the system remains robust even if influencing factors such as the experimental environment, experimenter, activity speed, and equipment height are altered.