The origin of the hymn's text is a poem by diplomat Sir Cecil Spring Rice, written in 1908 or 1912, entitled "Urbs Dei" ("The City of God") or "The Two Fatherlands". The poem described how a Christian owes his loyalties to both his homeland and the heavenly kingdom.

In 1908, Spring Rice was posted to the British Embassy in Stockholm. In 1912, he was appointed as Ambassador to the United States of America, where he influenced the administration of Woodrow Wilson to abandon neutrality and join Britain in the war against Germany. After the United States entered the war, he was recalled to Britain. Shortly before his departure from the US in January 1918, he re-wrote and renamed "Urbs Dei", significantly altering the first verse to concentrate on the themes of love and sacrifice rather than "the noise of battle" and "the thunder of her guns", creating a more sombre tone in view of the dreadful loss of life suffered in the Great War. The first verse in both versions invoke Britain (in the 1912 version, anthropomorphised as Britannia with sword and shield; in the second version, simply called "my country"); the second verse, the Kingdom of Heaven.

According to Sir Cecil's granddaughter, the rewritten verse of 1918 was never intended to appear alongside the first verse of the original poem but was replacing it; the original first verse is nevertheless sometimes known as the "rarely sung middle verse" The text of the original poem was sent by Spring Rice to William Jennings Bryan in a letter shortly before his death in February 1918.

The poem circulated privately for a few years until it was set to music by Holst, to a tune he adapted from his Jupiter to fit the words of the poem. It was performed as a unison song with orchestra in the early 1920s, and it was finally published as a hymn in 1925/6 in the Songs of Praise hymnal (no. 188).

It was included in later hymnals, including