The Millers had lost four out of their five away games this season, but earned a deserved three points at Spotland thanks to Richard Wood's second-half header.
Paul Warne's men could have had the game sewn up by the half-time whistle as they squandered three great chances, with Will Vaulks heading wide, Anthony Forde shooting weakly in a great position and Richie Towell bringing a fine save out of Josh Lillis.
Rotherham's goal eventually came in the 57th minute as Wood beat his marker from Forde's corner to head home at the far post and become the first player to score against Dale in 523 minutes.
Rochdale offered a response and Steven Davies crashed a header against the crossbar from a corner, while Ian Henderson had an effort cleared off the line.
The Millers could not put the game to bed and had a let off deep into stoppage time when Bradden Inman embarrassingly skied over from just three yards.