After the game finished all-square at 4-4 after extra time, John Mullin smacked the clincher from the spot after Denis Irwin's agonising miss.
Chris Swailes, Marvin Bryan and Paul Hurst hit the Millers' other penalties, while Dean Sturridge and Nathan Blake netted for Wolves.
Rotherham made a lively start to normal time and went in front after only four minutes through Richie Barker, who got a touch from Andy Monkhouse's shot from five yards after good work from Darren Garner and John Mullin.
Mark Robins fired over from a good Barker throughball as the Millers continued to press and they should have had a second when a good move ended with Barker almost chipping keeper Matt Murray.
Robins then went close from Garner's corner as the Millers bossed it, with Wolves having little to offer, although an Alex Rae free-kick was just a foot wide.
But Wolves grabbed an equaliser a minute before the break when defender Guy Branston had a rush of blood in the penalty box when he pulled Paul Butler back and Shaun Newton crashed home the spot kick.
After the interval Wolves stepped the pace up as Blake fired a rasping shot over the bar from Kenny Miller's pass, while the troublesome Monkhouse had a good 20-yard run which ended with him shooting over the bar.
The second came after 56 minutes when Miller showed his class, with a crisp 15-yard shot after his initial effort had bobbled kindly into his path off a defender.
Then, 12 minutes later, Wolves made it 3-1 with a classy chip finish from Rae after Ivar Ingimarsson's throughball had been knocked down by Blake.
It looked all over for Rotherham, but they staged a magnificent fightback as Mark Robins made it 3-2 from the spot after 78 minutes, Blake having handled Hurst's cross, and then Swailes sent the game into extra time with a glancing header from Scott's long throw.
They snatched a fourth three minutes into the extra period when the battling Barker collected a Mullin knock-through and rounded Murray to clip the ball home.
But the tie went into penalties when Scott blasted past his own keeper in the last minute