It is usually regarded as Reed's best-known solo work. The song reached #16 on the U.S. Billboard Hot 100, and #10 in the UK. In 2003, the song was ranked #221 on Rolling Stone magazine's list of The 500 Greatest Songs of All Time.
The lyrics tell of a series of individuals and their journeys to New York City, and is a thinly-veiled biography of several of the regular "superstars" at Andy Warhol's New York studio, The Factory, namely Holly Woodlawn, Candy Darling, Joe Dallesandro, Jackie Curtis and Joe Campbell (referred to in the song by his nickname Sugar Plum Fairy). Candy Darling was also the subject of Reed's song "Candy Says".
The mellifluous saxophone solo played over the fadeout of the song was performed by Ronnie Ross, who had previously taught David Bowie to play the saxophone during Bowie's childhood.
The song is also noted for its twin interlocking bass lines played by Herbie Flowers on double-bass and overdubbed fretless bass guitar, featuring the extensive use of a major tenth interval, which was unusual in pop music until then.
Like many of Reed's songs, the chord progression mostly consists of a Plagal cadence, in this case F to C.
In the 2001 documentary Classic Albums: Lou Reed: Transformer, Reed says that it was Nelson Algren's 1956 novel, A Walk on the Wild Side, that was the launching off point for the song, even though the song grew to be inhabited by characters from his own life. As with several other Reed songs from the 1970s, the title may also be an allusion to an earlier song, in this case Mack David and Elmer Bernstein's song of the same name, the Academy Award-nominated title song of the 1962 film based on Algren's novel. During his performance of the song on his 1978 Live: Take No Prisoners album, Reed humorously explains the song's development from a request that he write the music for the never completed musical version of Algren's novel.