Born in Brighton, England, and raised in a Christian family, Hoste was converted when he attended the evangelistic meetings of D. L. Moody near his home in 1883. Accepted by the China Inland Mission (CIM) in 1884, he resigned his commission as a lieutenant in the Royal Artillery and sailed for China in 1885 as one of the famed Cambridge Seven, although he was not himself a Cambridge graduate. He was sent to Shansi (Shanxi) Province for language study, and there he developed a close relationship with the respected pastor Hsi Shengmo, a convened Confucian scholar who directed much of the CIM work in Shansi and founded many opium refuges.
Hoste worked until 1896 in Shansi, spending several months each year living in villages with an evangelist and preaching at country fairs. He exhibited wise leadership in the sensitivity that he showed in being a peacemaker among both his missionary associates and his Chinese colleagues. Following a sick leave in Australia in 1896, he was made superintendent of the CIM work in Honan (Henan) Province, and he was then appointed acting general director of the CIM in 1901 and general director in 1902. Although some had predicted the demise of the mission following the death of Hudson Taylor in 1905, Hoste gained the ongoing confidence of both missionaries amid Chinese. He was married to Amelia Gertrude Broomhall in 1894, and they had three sons. He retired from the CIM in 1935, was imprisoned by the Japanese (1941-1945), and died in London.