Quality Diets for Better Health (QDBH) is a nutrition-sensitive agriculture project in southern Ethiopia that promotes vitamin A-rich orange-fleshed sweetpotato (OFSP) and community-based nutrition education to improve the quality of young child diets. The objective of this research is to examine the impact of the project and other predictors on caregiver knowledge of infant and young child feeding (IYCF) over time. Communities were randomized to a control group and intervention groups receiving OFSP planting materials and community-based nutrition education. Households with infants P < 0.05) and who reported receiving prenatal IYCF counseling (P < 0.0001), but also declined faster among those who had prenatal counseling (β = −1.6, P = 0.04). Knowledge scores in intervention and control groups were similar at baseline (P = 0.49), but scores of those in the intervention group declined less rapidly than those in the control group (β = 4.0, P < 0.0001). Caregivers’ knowledge of IYCF decreased in the postnatal period; decreases were fastest among those who received prenatal IYCF counseling and among those in the control group. Postnatal community-based nutrition education can help maintain knowledge of IYCF recommendations. This research is funded by the European Union in partnership with the International Potato Center.