In the semi-arid and arid regions of northern China, geochemical behavior of the aeolian deposit is closely related to climatic and environmental changes, which was used to reconstruct the past history of environmental evolution and possibly forcing mechanisms. However, the related result was still scarce due to the lack of detailed geochemical analysis results in the desert sediments. In the present study, we systematically analyzed the geochemical components and parameters of the paleo-aeolian sand dune and modern mobile sand deposits in the Mu Us Desert and discuss the climatic variation inferred from the paleo-aeolian sand dune during the past 4.2 ka BP. The results indicated that (1) geochemical composition of the sandy deposits were dominated by SiO2, Al2O3 and Na2O and the deposits probably originated from the widespread upper continental crust (UCC) and were formed by long-term weathering, transport and re-deposition; (2) these sandy deposits were subjected to weaker weathering or uneven weathering under cold and dry conditions, and had highly similar material sources and degrees of weathering and leaching in general; and (3) the direct OSL (Optically Stimulated Luminescence) dating ages and geochemical parameters from the palaeosol-aeolian sand dune indicated that the regional climate change experienced several typically cold and warm intervals. These intervals are 4.2, 2.8 ka BP and Little Ice Age and Medieval Warm Period, which probably attributed to periodic variations of the Asian summer monsoonal strength and cold events of the northern Atlantic Ocean in low and high latitudes of the Northern Hemisphere. Our results suggest that the development of the sand dune in the Mu Us Desert provided a suitable archive for understanding the past local climatic change, which is linked to the global climatic change.