Fruit yield estimation is an important task in modern orchard management. Producers can allocate labors, optimize resources, and increase harvest efficiency by accurately estimating fruit yields. In this paper, a video counting algorithm for kiwifruit was proposed based on YOLOv3-tiny and Kalman filter. Results showed that P, R and AP values of the YOLOv3-tiny reached 0.8258, 0.8353 and 0.8666, respectively, while average detection speed reached 31 ms per image. The tested average counting error between the video counting algorithm and manual counting is 7.2 fruits per video. The experimental results demonstrated that the video counting algorithm can be recommended for fruit counting in kiwifruit orchard.