Agriculture is a significant contributor to anthropogenic global warming. In recent years, agricultural greenhouse gas (GHG) emissions in China, which is the largest emitter, has been decreasing. In order to identify whether or not Chinese agricultural development has affected GHG emission, we used the logarithmic mean Divisia index (LMDI) factor decomposition model to study the effect of productivity factors on GHG emission and their characteristics in four phases of Chinese agricultural development. The results indicate that land productivity is the most significantly promotion factor which contributes 1.12 Gt CO2e GHG emission growth. On the contrary, technological input intensity exerts an obvious mitigating effect with 1.57 Gt CO2e GHG emission reduction. The effects of productivity factors on GHG emission differ between household-based farming systems and large-scale management systems. A more nuanced perspective on the significant role of agricultural large-scale management in GHG emission could aid climate change mitigation.