"All of Me" first came to public awareness when a performance by Belle Baker was broadcast over the radio in 1931. Paul Whiteman and His Orchestra recorded the song on December 1 that year, with vocalist Mildred Bailey; this went to the top of the US pop charts. Within weeks, another two versions were in the charts, with a Louis Armstrong rendition also reaching No. 1, and Ben Selvin and His Orchestra peaking at No. 19. The song was used in the 1932 film "Careless Lady." In the view of critic Ted Gioia, the definitive version was sung by Billie Holiday in 1941: "she staked a claim of ownership that no one has managed to dislodge in subsequent years". Two years later, Lynne Sherman's recording with Count Basie and His Orchestra reached No. 14 in the charts.

Frank Sinatra recorded several versions of "All of Me". His 1948 release peaked at No. 21. He also sang it in the film "Meet Danny Wilson," which may have helped Johnnie Ray's rendition up to No. 12 in the charts that year. Sinatra's use of "All of Me" brought a non-jazz audience to the song, and so too did Willie Nelson, whose version was included in his Stardust album and reached No. 3 on the Hot Country Songs chart in 1978. In 2000, "All of Me" was given the Towering Song Award by the Songwriters Hall of Fame.