In order to improve the accuracy of tomato-picking robots’ recognition in complex environments, a tomato recognition method based on an improved SSD model was proposed. Based on the traditional SSD (Single Shot Multi-Box Detector) network model, an RFB (Receptive Field Block) module was added and the loss function was improved by adding repulsion loss to construct a network model for tomato fruit recognition in complex environments. The improved SSD model was tested with a self-made dataset, and a comparison test was conducted for tomatoes under different shade on sunny and cloudy days. The test results showed that the differences between the two algorithms were compared and analyzed with F1 as the evaluation value. Test results showed that the F1 values for single-fruit shade and multi-fruit shade were 87.3% and 85.6% on sunny days while the figure for leaf shade and root-stock shade were 83.6% and 89% respectively, increasing by 7%, 4% and 6%, 6% compared with the traditional SSD model. On cloudy days, the F1 values for the same four types of shades reached 79%, 79.6%, 78.3% and 80.3% respectively, rising by 2%,1.3% and 4.3% 5.7%. These results proved that the network could realize the accurate detection of tomato fruits in complex environments and provide a new method for testing agricultural robots for picking tomatoes.