Distribution network (DN) fault diagnosis and risk warning are of great significance to the safe and stable operation of power system (PS). At present, the imbalance classification and outlier data greatly affect the accuracy of fault analysis and the comprehensiveness of fault risk levels (FRLs) prediction in urban distribution network (UDN). In this paper, an abnormal data identification (ADI) method of FRLs prediction based on improved natural breaks and ensemble learning is proposed. The method of improved natural breaks is first introduced for threshold movement to achieve dynamic division of FRLs and solve the imbalance problem of fault categories. Then, the prediction accuracy of FRLs and Kappa statistic are used as the new evaluation indicators for ADI. Combined with the accuracy and timeliness of identification, the detection results of each ADI under different data sets with different abnormal proportion are analyzed, and then several specific identification algorithms are selected to form an optimal identification model based on ensemble learning. Finally, the rationality and effectiveness of the proposed method are proved by the actual data and critical node fault simulation data of UDN in a certain area.