Born in Jalandhar in 1927, Khayyam ran away to Delhi when he was a young boy when his parents did not accept his love for music. He later travelled to Lahore to learn music under Baba Chishti and later under Pandit Amarnath. He had even enrolled in the army in 1943, where he was a part of the cultural troupe led by Faiz Amhed Faiz. After the war, he left for Mumbai to work in films.
He joined the music director duo, Husnlal - Bhagatram, who were Pandit Amarnath's brothers. He started his career with the pseudonym Sharmaji. His first film as music director was “Heer Ranjha” (1948) , for which he teamed up with Rehman Varma. The duo were credited as Sharmaji - Varmaji. He went solo with "Parda" (1949) but it was the film “Biwi” (1950) and the song “Akele Mein Woh Ghabrate Toh Honge”, which brought him notice. It wasn’t till 1953 and the release of “Footpath” that Khayyam started using his real name. His songs for “Phir Subah Hogi” (1958), brought his critical acclaim as well, especially the satirical number “Cheeno-Arab Hamara Hai.” After composing for “Aakhri Khat” (1966), he took a break from film music.
After composing music for non-filmi music for almost a decade, he staged a huge comeback with “Kabhi Kabhie” (1976). This film also won him his first Filmfare Award. He successful streak continued with “Umrao Jaan” (1981) for which he is credited for renewing singer Asha Bhosle's career.
Khayyam is the recipient of the Sangeet Natak Academy Award in 2007, the Filmfare Lifetime Achievement Award in 2010 as well as the Padma Bhushan in 2011.
Khayyam moved away from public life after the death of his son, Pradeep and his wife, Jagjit Kaur, an accomplished singer who supported Khayyam right from the days he was a struggling musician and even assisted him with his compositions.