The aflatoxin produced by peanut mildew is highly carcinogenic, and it seriously affects food safety. In order to accurately and quickly identify moldy peanuts, this project proposes a detection method for moldy peanuts based on machine vision. Firstly, the peanut image was double-sided filtering and noise reduction, and then the image was converted to HSV space. The moldy peanut was recognized and detected by superimposing the mold color range extracted in H and S space and the open processing results of V space. The experimental results showed that the recognition accuracy of this method for moldy peanuts reached 95.3%, and the processing time for a single frame of peanut image was 0.6 seconds. Compared with other algorithms, this method had the advantages of fast speed and high accuracy, which can meet the real-time detection of moldy peanuts. At the same time, the grading processing of peanut mold is also more practical.