To achieve precise detection of coarse aggregate gradation, this study proposes an intelligent portable detection method for coarse aggregate gradation. A knowledge distillation strategy is employed to lighten the network structure of the large visual model SAM, and the neural network classifier PP-HGNetV2 is embedded to provide the model with semantic judgment capabilities. A mathematical representation algorithm for the characteristic parameters of coarse aggregate particles is designed, and a mobile application is developed to enable high-throughput detection of coarse aggregate gradation. Tests were conducted on five different coarse aggregate gradation scenarios, and the results indicate that the proposed method achieves higher segmentation accuracy for coarse aggregate particles compared to the original SAM model. It also precisely removes background information, and the extracted key parameters of coarse aggregate particles are accurate and reliable.