Jorge Leitao came off the bench to produce an injury-time winner to give Walsall a first day win.
Leitao looped over a hopeful cross from the left seconds before the end and saw it go in off the post as Rotherham keeper Gary Montgomery lost the flight of the ball.
Walsall old boy Martin Butler had produced a leveller that looked set to earn Rotherham a point and rob his old side of the win before Leitao struck.
Walsall went ahead largely thanks to a piece of football magic by teenage sensation Matty Fryatt after just three minutes.
The 19 year old, who has had a £1million price tag slapped on him by player-manager Paul Merson, sparkled down the right.
He made a blistering run, skipped around Rotherham defender Shaun Barker and lashed a low drive for Kris Taylor to finish off with a crisp shot from six yards out.
The setback rocked Mick Harford's former Championship side and two minutes later Merson's men could have stretched their lead further.
Taylor wriggled through a gap in the centre and tried to place his shot but keeper Montgomery stretched to palm the effort away.
Another Walsall chance went begging when Julian Bennett hooked a shot over the top from a corner as the visitors kept up the pressure.
Only a spectacular save from Montgomery stopped Walsall going further in front after 25 minutes when Taylor whipped a superb free-kick in from 25 yards but the keeper prevented it from curling into the top corner.
Barker twice went close for Rotherham with two headers but the first looped over the top and the second was blocked after a Gregor Robertson corner.
New striker Deon Burton came close to a dramatic debut but an acrobatic overhead kick was well held by Andy Oakes and Paul McLaren tried a carbon copy effort and was inches over after Lee Williamson went close with a fierce low drive.
But three minutes before the break Rotherham's pressure told when Burton raced down the left and produced an inch-perfect cross for ex-Walsall striker Butler to power into the back of the net.
Seconds before the break Jamal Campbell-Ryce dazzled down the right and picked out Robertson with a cross but his shot was a fraction wide.
But after the break chances were few and far between as Rotherham kept a tight grip on Walsall dangerman Fryatt.
Burton charged down a clearance from Oakes, and as the ball broke loose into space, the keeper charged in to recover and hoof a huge clearance out of the danger area.
Michael Standing went close with a low drive and the game looked set for a stalemate until Leitao scored.