The hosts, under the permanent guidance of new manager Mark Robins for the first time, trailed 2-0 with 15 minutes to go before the young strike duo earned them the points.
The result leaves Gillingham still needing at least a win to ensure their own survival.
Robins' reign got off to the perfect start when on seven minutes Delroy Facey headed home Martin Woods' left-wing cross.
Felix Bastians levelled for Gillingham on 22 minutes when he cut in from the left and unleashed a curling 20-yard shot into the top corner.
Bastians turned the game on its head two minutes later when he set up the second with a corner that was powerfully headed home by Andrew Crofts.
Gillingham defended deep in the second half and were forced on to the back foot as Rotherham sought an equaliser.
Woods, Newsham and Ian Henderson all went close, while at the other end Guylain Ndumbu-Nsungu went close to stretching Gillingham's lead.
Rotherham got the equaliser they deserved on 75 minutes when Facey headed into the path of O'Grady who rolled his marker and hooked the ball beyond Kelvin Jack with a clever shot with the outside of his foot.
Rotherham continued to look for the winner knowing that a draw would be no good in their plight.
And they were rewarded four minutes from the end when Woods burst down the right wing.
His cross was met with a downward header from Newsham at the front post who celebrated his first league goal for the club.