He continued a long partnership with the Rascals, overseeing perennial hits like "Good Lovin'" and "Groovin'" and earning the title of the group's unofficial fifth member. In 1969, Mardin worked on the Dusty Springfield masterpiece Dusty in Memphis, and accepted a post as an Atlantic vice president; a year later, he moved on to produce Laura Nyro's Christmas and the Beads of Sweat, and also released his first solo LP, Glass Onion. His collaboration with Franklin continued on as well, yielding superb records like Spirit in the Dark, Young, Gifted & Black, and Amazing Grace. In 1971, Mardin shifted gears entirely to helm folkie John Prine's acclaimed self-titled debut. From there, he went on to work with a tremendous variety of new Atlantic signings, including Hall & Oates, Bette Midler, Roberta Flack, Donny Hathaway, Willie Nelson, and Doug Sahm.

In 1974, Mardin traveled to Miami to produce the Bee Gees, who were at the time struggling to recapture their late-'60s pop success. On the orders of manager Robert Stigwood, Mardin reinvented the group's sound to capture the flavor of American R&B; while his first attempt, Mr. Natural, was not a success, the 1975 follow-up, Main Course, found the Bee Gees pioneering the commercial rise of disco, scoring a number one hit with the single "Jive Talkin'." Mardin's knack for blue-eyed funk also came to the fore through his concurrent work with the Average White Band, most notably the number one hit "Pick Up the Pieces." Albums with Rod Stewart, George Benson, and Carly Simon followed, and in 1980 he also began an extended partnership with Chaka Khan, resulting in her biggest hit, "I Feel for You." Although Mardin's pace slowed in the years to follow, he continued to work with high-profile younger artists like Culture Club and Howard Jones, and also reunited with the likes of the Bee Gees, Hall & Oates, and Roberta Flack. His work with Phil Collins on albums like Face Value and No Jacket Required resulted in his biggest commercial successes of the '80s, and he also helped bring Bette Midler back to the top of the charts with "Wind Beneath My Wings."

Mardin remained active into the '90s, working most often with female solo artists: Midler, Carly Simon, Barbra Streisand, Patti LaBelle, Diana Ross, and Jewel. He also produced the original Broadway cast recordings of the hit musicals Smokey Joe's Cafe: The Songs of Leiber & Stoller and Rent in 1996 and 1997, respectively. In May 2001, amid numerous honors for his longevity and enormous legacy (12 Grammys and counting), Mardin retired as Atlantic's senior vice president. However, he wasn't finished -- he signed on with Manhattan/EMI several months later in several capacities, chief among them signing and producing new artists with adult appeal. His first major success with the label was the jazzy singer/songwriter Norah Jones, whose Blue Note debut album, Come Away with Me, topped the pop charts in early 2003.

In early 2005, Mardin was diagnosed with pancreatic cancer, and began working on a final album that would sum up his life's work as well as introduce some of his unknown compositions to listeners for the first time. The album was nearly complete at the time of his death on June 26, 2006. It was finished by his son, co-producer, and longtime collaborator Joe Mardin. Entitled All My Friends Are Here, it features jazz and pop compositions from throughout the elder Mardin's life, performed by a house band and an all-star lineup of performers including Hall & Oates, Chaka Khan, Norah Jones, Willie Nelson, Carly Simon, David Sanborn, Jon Faddis, Bette Midler, and Dr. John. It was released in 2010 along with the accompanying session documentary DVD, The Greatest Ears in Town, co-directed by Joe Mardin and Doug Biro.