Xander Schauffele took gold and Rory Sabbatini won silver.
Schauffele withstood an Olympic record round of 61 from South Africa-born Rory Sabbatini, who now represents Slovakia, to triumph by one shot on 18 under pari, with Sabbatini finishing second.
Irish star McIlroy agonisingly missed a putt on the 18th that might have given him third outright, but he was among those who faced extra holes before the podium places could be completed.
He was joined by the likes of Great Britain’s Casey, recently-crowned Open champion Collin Morikawa and Japan’s Masters champion Hideki Matsuyama, all tied on 15 under.
Schauffele held his nerve following Sabbatini’s bid to turn the leaderboard on its head.
Sabbatini’s round included an eagle, 10 birdies and two bogeys on a day of impressive low scoring across the board.
Schauffele is the first American Olympic golf champion since Charles Sands in 1900.