To solve the problem that it is difficult to obtain the discriminant features of sheep face and the model is too large to be applied effectively in sheep face recognition. A lightweight sheep face recognition model based on double attention mechanism is proposed. The model is based on ShuffleNetV2 and add SKNet convolution attention module to adaptively adjust the convolution kernel size in order to capture sheep face features at different scales. Add an improved channel attention module in the channel dimension to suppress the interference of redundant information and enhance the expression of salient features of sheep face, so that the model could extract highly discriminant sheep face features. The model use Mish activation function to replace ReLU activation function to reduce the loss of feature information. Select adaptive scaling cosine metric function (Adacos) to train the model to speed up the convergence and further improve the recognition rate. On the constructed sheep face recognition datasets, the experimental results show that the recognition rate of the proposed sheep face recognition model is 91.52%, the model size is 4.45MB, and the computational quantity is 147MFLOPs. Experimental results show that the improved sheep face recognition model has better feature extraction ability and balance the recognition accuracy and complexity of the model, which can provide a reference for the practical application of sheep face recognition.