The convolutional neural network (CNN) has won unprecedented triumph in super-resolution (SR) reconstruction via making full use of the advantages of fast parallel computing, end-to-end high training accuracy. However, the model CNN based can only extract fixed-size local feature information, and has limited ability to obtain global information. Moreover, scale fusion mechanism was rarely employed in model training in the past, and diverse degradation models with different scales need to be trained and tested separately. We put forward a new self-attention based multi-scale learning network (SAMSSR), which can address the above problems. On the one hand, extract the global similarity in degraded images in virtue of self-attention mechanism, and the similarity is acted as the feature weights to redistribute the feature channel information, so as to enhance the important information and suppress the non- important information, finally improve the reconstruction quality and simplify the training process. On the other hand, taking advantage of intrinsic similarity and correlation of SR results with different scales reconstructed by model and deeply merging them, we ultimately achieved multi-scale SR outputs, consequently realized a single end-to-end multi-scale SR reconstruction. Extensive experiments demonstrated that SAMSSR obtained competitive promotion in accuracy and visual perception compared with state- of-the-art methods.