Accurate identification of the continuous physiological states of crops and diseases and pests is crucial for agricultural production management. However, real agricultural monitoring scenarios often encounter problems such as high similarity in visual differences between classes, complex background interference, and unstable lighting and environmental conditions. Additionally, due to the limited computing power, storage space, and power consumption budget of edge devices, traditional methods with high parameter volume and high computational complexity are difficult to meet the requirements of real-time monitoring and end-device deployment. To address these issues, we propose an efficient and lightweight deep perception method for complex agricultural visual tasks, Frequency-Spatial Enhanced Lightweight ConvNeXt (FSEL-NeXt), which mainly consists of the Vivid Enhanced Spatial Attention Stem (VESAS) and the Frequency-Spatial Enhancement and Dilation (FSED). VESAS retains high-resolution texture information and extracts multi-scale semantic features through a dual-branch structure; FSED combines asymmetric spatial convolution with frequency-domain multi-layer perceptron to jointly model local texture, global morphology, and frequency information, thereby improving the model's ability to recognize subtle category differences with lower computational cost. We conducted comprehensive evaluations on multiple datasets including tobacco drying status, banana ripeness, and plant diseases.