By using sustainable yield index (SYI), this paper analyzed the sustainability of maize, rice, and wheat yields at 20 long-term fertilization experiment sites in China under different fertilization system and ecological conditions. The SYI value of test crops varied significantly with fertilization system, crop kind, effective accumulated temperature, and sunshine hour. Irrespective of fertilization, the SYI value of rice was generally higher than that of maize and wheat. Under long-term no fertilization, the SYI values of rice, wheat, and maize were lower, being 0.55, 0.44, and 0.43, respectively. Fertilization, especially a combined application of NPK and manure, promoted the sustainability of crop yields, with the SYI values of rice, wheat, and maize being 0.66, 0.58, and 0.57, respectively. Under the application of N and NK, the SYI values ranged in 0. 36-0. 47. SYI value > 0.55 represented the better sustainability of crop yields, while SYI < 0.45 represented a worse one. Longitude, altitude, and weather factors also affected the SYI values, especially that of maize. Under no fertilization, maize had the largest variation of SYI value, followed by wheat, and rice. Our results indicated that a combined application of chemical fertilizers with manure benefited high and stable yielding, being the optimal fertilization mode for maintaining the sustainability of grain yield.