Wes Thomas gave Rotherham the lead before Lee Miller's overhead kick enabled Carlisle to draw level in the third minute of added time in the first half, only for Agard to pounce at the death for his 16th goal of the season.
Rotherham could have been two goals up in the early stages with Thomas twice failing to hit the target when presented with clear-cut opportunities.
But the striker made amends in the 24th minute when benefiting from a chaotic mix-up in the Carlisle defence.
Goalkeeper Jordan Pickford raced a long way out of his area in an effort to head a hopeful long ball clear and in the ensuing collision Thomas was first to regain his balance and roll the ball into the unguarded net from just outside the area.
Ben Pringle nearly doubled the lead but Pickford tipped his 25 yard curler over for a corner.
But against the run of play Carlisle levelled through an inspired overhead shot by Miller from 12 yards, after both Brad Potts and Sean O'Hanlon had seen headers bobble about in the box.
Rotherham continued to press after the break, but their frustrations grew as Thomas again missed a great chance - hitting the legs of Pickford and then having a penalty appeal waved away.
But great play by Alex Revell on the right saw his cross flicked home by Agard from six yards in the second minute of stoppage time to seal the points.
Source PA