The Rolling Stones landed in New York on June 1, 1964, for their first North American tour, around the time the Valentinos' recording was released. New York radio DJ Murray the K played the song to the Rolling Stones. He also played the Stones' "King Bee" (their Slim Harpo cover) the same night and remarked on their ability to achieve an authentic blues sound. After hearing "It's All Over Now" on that WINS show, the band recorded their version nine days later at Chess Studios in Chicago. Years later, Bobby Womack said in an interview that he had told Sam Cooke he did not want the Rolling Stones to record their version of the song, and that he had told Mick Jagger to get his own song. Cooke convinced him to let the Rolling Stones record the song. Six months later, after receiving the royalty check for the song, Womack told Cooke that Mick Jagger could have any song he wanted.

The Rolling Stones' version of "It's All Over Now" is the most famous version of the song. It was first released as a single in the UK, where it peaked at number 1 on the UK Singles Chart, giving the Rolling Stones their first number one hit. It was the band's third single released in America, and stayed in the Billboard Hot 100 for ten weeks, peaking at number 26.