Wolff was born in Nice in France, moving to the United States in 1941 and become an American citizen in 1946. He studied classics at Harvard University and upon graduating took up a teaching post there which he kept until 1970 when he began to teach classics and music at Dartmouth College. While at Harvard, Wolff associated himself with the composer John Cage and the group around him (Earle Brown, Morton Feldman). His early work includes a lot of silence. Later pieces often give a degree of freedom to the performers, and some works, such as Changing the System (1973), have an explicitly political element.