In the agricultural field environment, in order to meet the strict requirements of modern intelligent agriculture on the accuracy and speed of pest detection algorithms, we proposed a lightweight farmland pest detection algorithm based on an improved YOLO v5. First, we constructed a new backbone by combining PPLCNet and BotNet to reduce network parameters and enhance the ability of image feature extraction. Second, the soft non-maximum suppression (Soft-NMS) algorithm is used instead of the NMS algorithm to improve the detection of target feature areas and minimize duplicate detection of the same areas. Then, the scylla intersection over union is applied to replace the original loss function complete intersection over union in the model to obtain higher quality anchors. Besides, mosaic data enhancement, scaling, translation, rotation, and color transformation methods are used to enhance the data volume, and letterbox is employed to unify data set sizes. Finally, we performed performance analysis experiments on a dataset containing 15 species of agricultural pests. From a large number of experimental results, we can conclude that our model improves the precision by 4.3% compared with YOLOv5n, while reducing the reasoning speed and model size by 10.1% and 50%. Furthermore, compared with faster R-CNN, cascade R-CNN, CenterNet, RetinaNet, YOLOv3, YOLOv4, YOLOv5, and YOLOx, our model shows the highest detection accuracy and the fastest reasoning speed. The mAP@0.5 is 95.3%, and the reasoning time of each image is only 1.7 ms.