The Millers went in front on 20 minuted after Joe Newell volleyed in after a poor defensive header by Canice Carroll.
But the U's stormed back to lead 3-1, despite missing a penalty.
Jack Payne fired into the bottom corner to level 60 seconds after Newell's strike.
Josh Ruffels then beat goalkeeper Marek Rodak at his near post on the half-hour to fire Oxford in front.
Moments later, Will Vaulks blocked Ricardinho's cross with a hand, but Rodak saved Ryan Ledson's weak penalty down to his left.
However, the U's did go two goals in front after 40 minutes when Curtis Nelson stabbed home at a corner, only for Ryan Williams to make it 3-2 from 12 yards a minute before half-time.
Rotherham striker Kieffer Moore lobbed onto the bar in the second half before Moore converted Williams' left-wing cross on 78 minutes to earn a draw.