Bradden Inman gave Crewe the lead after 12 minutes after AJ Leitch-Smith had seen a shoot hit the post.
The Millers equalised through Kieran Agard just before the half-hour mark but Crewe's advantage was restored in the 61st minute when Kelvin Mellor spun in the area and fired home.
Rotherham levelled again in the 68th minute with a Lee Frecklington penalty before Harry Davis saw his penalty for Crewe saved by Adam Collin.
The hosts roared up to the other end and Ben Pringle's superb cross was powerfully headed in by Alex Revell to give the home side the lead for the first time.
Chuks Aneke went close to an equaliser with a diving header that flew just wide.
But Rotherham made sure of the win in the dying seconds when Frecklington pounced to shoot home his second as Crewe pushed forward.