This study evaluated the effects of traditional Chinese medicine based exercise (TCE) on cognitive outcomes for patients with mild cognitive impairment (MCI), and tried to identify the most effective TCE modality. A systematic review and meta-analysis of randomized controlled trials were conducted. Ten studies met the inclusion criteria. TCE interventions were classified into three types: (a) Tai Chi, (b) Baduanjin, and (c) Qigong. The pooled analysis showed that, overall, TCE had beneficial effects on global cognition and memory. Subgroup analysis further demonstrated that Baduanjin had a larger effect size on global cognition than the other TCE modalities. By contrast, Tai Chi had a larger effect size on memory than the other modalities. This study implied that TCE is a promising exercise option for improving cognition in MCI. However, further studies with a more rigorous study design are warranted to support or falsify the findings of the present review.