The son of a pastor, Cheng studied the Chinese classics before continuing his education at the Anglo-Chinese Institute of the London Missionary Society (LMS) in Beijing. From 1896 to 1900, he did theological studies in Tianjin and graduated two weeks before the Boxer Uprising. Cheng risked his life as an interpreter and stretcher-bearer for the Allied forces. After the siege ended, he participated in relief work in Beijing.
At age 22, Cheng began to help George Owen of the LMS revise the Chinese translation of the New Testament. It was completed in 1906. He received further theological training at the Bible Institute in Glasgow, Scotland, and returned to China after his graduation in 1908. After his ordination in his home church, he pastored a newly independent church, the Mi-shih Hutung Church in the East City of Beijing, which was attended by a number of Chinese academics and professionals.
Cheng was one of the three Chinese delegates at the World Missionary Conference in Edinburgh in 1910. In a seven-minute address, he expressed his hope to see a "united Christian Church without any denominational distinctions" in China. Cheng was appointed secretary of the continuation committee of the National Missionary Conference in China, formed after John R. Mott's visit to China in 1913.
In 1917, Cheng led a campaign against the movement to allow only Confucian teachings for moral instruction in the schools. Realizing the future of the church in China lay in indigenous leadership, he formed the China for Christ movement (1919) and also helped form the indigenous interdenominational Chinese Home Mission Society to reach the ethnic groups in southwest China. He was general secretary of the National Christian Council from its establishment in 1922 until his resignation in 1933 because of poor health. In 1927, Cheng was elected the first moderator of the Church of Christ in China, a Protestant ecumenical organization comprising 16 denominations. He was on the executive committee of the International Missionary Conference from 1928 to 1938.
Cheng was deeply distressed by the Sino-Japanese war in 1937 because he knew many Japanese Christians and felt the war would do irreparable damage to Christian unity.
Cheng worked for an independent, unified Chinese Christian Church and desired universal Christian unity. A talented, tireless leader, he received honorary doctorates from Knox College, Toronto, Canada (1916); the College of Wooster, Ohio, USA (1923); and St. John's University, Shanghai (1929). He died in Shanghai after his visit to the mission work in southwest China and Guizhou in 1939.