The Welsh full-back reacted the quickest when Ryan Lowe's right-sided corner was flicked on at the near post by Gary Roberts. Moss volleyed in from close-range giving the visiting keeper, Neil Cutler no chance.
That agile piece of goalkeeping maintained Crewe's advantage and Williams had an easier job dealing with Ian Sharps' header from the resulting corner.
Crewe had Watford winger Anthony McNamee making his debut and, while the 23-year-old offered a stunning array of step-overs and tricks, he never really tore the Millers' defence apart.
Michael Higdon struggled to have any effect up front for Crewe but he wasn't guilty of wasting a juicy chance which fell the hosts' way later on in the game. Lowe moved onto Higdon's flick-on with just Cutler to beat but scuffed his attempted volley and the ball trundled into the keeper's hands.
Rotherham faired no better with Rossi Jarvis firing well over in the dying minutes and then straight at Williams as Crewe held on which takes Crewe over the 50-point mark and more or less guarantees that they will be playing League One football next season.