Iceland were on the verge of going through in normal time against the Hungarians, leading 27-26 and having a penalty with just 15 seconds remaining, but it was missed and Hungary forced the additional period three seconds from the end.
With the tie still level in the second of two five-minute halves of extra-time, Hungary finally made their move to win by a single goal and send the Basketball Arena wild.
A tight first half of the additional period saw the score advance to 29-29 and that became 30-30 as a second set of two five-minute halves was required to separate the two teams.
Hungary finally earned some breathing space and scored two successive goals to lead 33-31 and it proved decisive as they were able to hold out to seal a famous victory.
They go on to meet Sweden, the three-time silver medallists who remain on course to add to that haul after edging a Scandinavian thriller with Denmark 24-22.
A tight contest throughout was effectively decided by Niklas Ekberg's shot into an empty net with just over a minute to play.
Dalibor Doder finished top-scorer with six for Sweden, who celebrated long after the final whistle, while Hans Lindberg also threw in six for Denmark in a losing cause.
In the other half of the draw, France stayed on course to successfully defend their title as Olympic champions after a 23-22 quarter-final win over Spain.
An even contest was heading for extra-time with the scores tied at 22-22 going into the final 30 seconds, but the Beijing 2008 gold medallists had time for one more attack and when a rebound fell to William Accambray he threw in to send his team through to the last four.
It was harsh on Spain as they led until the 45th minute yet must now pack their bags and head home.
France will now face two-time gold medallists Croatia, whose hopes of adding a third to their collection remain in tact as they beat Tunisia 25-23.
Croatia, winners at Atlanta 1996 and Athens 2004, were not tested by their African opponents and their unbeaten record at London 2012 was never threatened in a game that rounded off a superb day's action.