On 29 July 1984, he won his first gold medal, breaking his own world record in the 200m freestyle. The next day he won the 100m butterfly, again setting a world record. Gross was expected to earn a third gold medal in the 200m butterfly, but he was upset by little-known Jon Sieben of Australia. Gross gained respect by using all of his time with the press to praise Sieben.
Between 1981 and 1988, Gross dominated the 200m butterfly, winning two world championships and four European championships and setting four world records. At the 1988 Games in Seoul he earned the one prize that had eluded him in that event: the Olympic gold medal. Gross also won two more medals as a member of West Germany's 4x200m freestyle relay team, a silver one in 1984 and a bronze in 1988.