Striker Shefki Kuqi grabbed a goal and made another as Wednesday claimed the honours and three vital points to start the New Year in style.
It left Wednesday boss Chris Turner full of praise for his side as they battle to haul themselves away from the relegation zone.
Said Turner: "I'm very pleased with the result. It's a good start to a very important period of 20 games for us.
"There are plenty of points to play for and enough to get us out of trouble.
"I'm not going to get carried away with a win against Rotherham but I thought we defended well and we won most of the battles on the pitch.
"Our main concern at the moment is that we are running out of players because of injuries."
Rotherham boss Ronnie Moore slammed his side for their second-half performance.
He said: "There wasn't much in it in the first half, but in the second they had the desire and the will but we didn't.
"We got what we deserved which was nothing, we didn't deserve the win because we didn't have the desire or the will."
It was Rotherham's third successive home defeat.
"I don't want Sheffield Wednesday to go down because these are the sort of games we want to play, but today it looks as though we were trying to help them."
Both sides squandered glorious chances just before half-time and Lee Bradbury hit the angle of the woodwork with a 25-yard effort.
But after 68 minutes Kuqi glanced a superb header past Mark Pollitt from Derek Geary's cross and seconds from the end Kuqi set up Adam Proudlock who smacked the second goal after Pollitt blocked his first attempt.