First-half play was scrappy with Nathan Jarman forcing a good stop out of keeper Andy Warrington and then Adam Proudlock went even closer for Grimsby when his excellent effort from distance struck the post in first-half stoppage time.
The second period then took on the feel of a cup tie with both sides going close early on.
Grimsby keeper Phil Barnes and his opposite number Warrington both made fine blocks before home debut boy Joe Widdowson had the crowd on their feet with a stunning opening goal on 56 minutes.
The left-back forced his way into the box after a strong run and shot past Warrington with a fierce cross-shot.
Grimsby's second goal on 71 minutes came from the spot after former Mariner Nick Fenton was adjudged to have fouled Proudlock in the box.
The Grimsby number nine stepped up to send Warrington the wrong way from 12 yards.
The Grimsby crowd were enjoying the rare luxury of a 2-0 lead although Rotherham refused to give up.
The icing on the cake came in the 84th minute with a third goal from midfielder Dean Sinclair who played a one-two with Proudlock before becoming the second player on his home debut to find the net.
The third goal was a signal for many of the travelling 848 fans to make an early exit.