"Substitute" was primarily inspired by the 1965 soul single "The Tracks of My Tears" by Smokey Robinson and the Miracles. Pete Townshend became obsessed, particularly, with the line, "Although she may be cute/She's just a substitute." This had then led Townshend "to celebrate the word with a song all its own."

The riff used in the song's verses was derived from a November 1965 single by Robb Storme and The Whispers called Where Is My Girl, a fact later acknowledged by Townshend who recalled hearing the record whilst reviewing singles for the Melody Maker's 'Blind Date' column.

The guitar solo is played by bassist John Entwistle who later said: 'I played a Gibson SG medium scale bass with wire-wound strings. When it got to the solo, because we were recording and mixing it virtually live, I thought, yeah, this should be a bass solo, so I turned my volume up and they couldn’t mix me out, so it ended up as a bass solo.'

For the American single, released in April 1966, the line in the chorus “I look all white but my dad was black” was changed to “I try walking forward but my feet walk back.” The complete second verse and chorus were also edited from the US release, reducing the track's length to two minutes and fifty-nine seconds