Kayode Odejayi, a double from man-of-the match Steven Gillespie and a late John Melligan strike cancelled out early Rotherham goals from Marc Newsham and Ian Henderson.
Cheltenham, seeking the victory that would secure their survival, started brightly with Melligan fizzing a free-kick just over the bar.
Melligan turned provider on 12 minutes when he found Gillespie unmarked in the penalty area, but the midfielder headed straight at Gary Montgomery.
Relegated Rotherham doubled the lead on 22 minutes when Delroy Facey broke down the left and squared the ball for Henderson (left) who swept home his first goal for the Millers with a crisp sidefoot into the bottom corner.
Gillespie and Finnegan went close with long-range efforts before Odejayi brought a fine reflex save from Montgomery with a powerful header.
Odejayi then set up Meillgan who brought another good save from Montgomery, but from the resulting corner, Cheltenham's pressure got its reward.
Melligan's corner swung over and Odejayi leapt unopposed to head over Montgomery into the net.
Stephen Brogan had a chance to restore the Millers' two-goal cushion in stoppage time but his shot at the end of a jinking run was saved by Scott Brown.
And they began their survival party ten minutes later when Gillespie volleyed home from 20 yards as the Rotherham defence stood and watched in disbelief.
The Millers should have been level within minutes but substitute Michael Keane missed an open net after a good knock down by Facey.
Ian Sharps also headed wide before Melligan sealed the fightback with a dipping volley on 88 minutes with Montgomery flat footed.