Bradford striker Peter Thorne bagged a double, but it was too little too late to save the Bantams' play-off dream.
Stuart McCall's under-achieving side won for the first time since thumping Aldershot 5-0 on March 7, but the main question for the home fans was whether the manager would still be around next season.
The former City favourite had vowed to walk away if they failed to finish in the top seven, but after a show of support from Bradford fans it seems likely that McCall will commit himself to another League Two campaign.
Rotherham, who would have been heading for promotion but for a 17-point penalty at the start of the season, claimed offside when Thorne nudged Dean Furman's volley home after just 12 minutes.
But after consulting his assistant, referee Andy Hall waved away their appeals and with the City players keen to impress McCall they suddenly form the form which had deserted them in the previous nine league games.
The home side created a host of chances and but for a Gordon Banks-style display by veteran keeper Andy Warrington, they could have been on the receiving end of a real hiding.
Thorne doubled his tally just after the half-hour when midfielder Nicky Law out-paced Marc Joseph and sent over a perfect cross for City's leading scorer to volley home from eight yards.
Bradford keeper Rhys Evans was rarely stretched by the visitors and Thorne was only denied a hat-trick by Warrington's excellent block in injury time.
With Bradford well on top, Warrington produced two stunning saves in the space of six minutes to deny Steve Jones, but the lively City man got his reward got his reward with Bradford's third soon after.
Shortly before Jones' match-sealing strike, Warrington produced his best moment of the game with a superb double save to keep out Law's drive and Thorne follow-up effort.
But his resistance was broken by Jones after 73 minutes when he raced onto Joe Colbeck's neat pass before driving an 18-yard shot beyond the Rotherham shot-stopper.