Smith was born in Wellington, Somerset, England, was educated at Oxford, and in 1841 was appointed vicar of Goole in Yorkshire. In 1844 he and Thomas McClatchie sailed for China as the first two CMS missionaries in that country. (Smith is not to be confused with his namesake, a pioneer CMS missionary of the Foochow [Fuzhou] mission in the 1850s.)
In 1847 he published the results of a tour of Hong Kong and the treaty ports undertaken for the CMS. He and Bishop W.J. Boone of the Protestant Episcopal Church in the U.S.A., who had preceded him in 1844, reached agreement over their respective jurisdictions. In 1849 Smith became bishop of Hong Kong. After the 1858 Treaty of Tientsin opened inland China to missionary work, he appealed for help, and the Society for the Propagation of the Gospel responded with four missionaries in 1859.
After the Taiping rebels were defeated by the imperial troops under General Charles Gordon, Smith wrote public protests about the massacres perpetrated by the victors. He visited Australia, where he appealed for Christian ministry to the many Chinese immigrants there. He resigned his see in 1864 but attended the Lambeth Conference in 1867.