In front of a crowd of 1390, the visitors scored in the tenth minute with a Peter Wackett try.
Coventry hit back through Ben Russell who converted two penalties in the 30th and 34th minute.
Chris Lewis ran in a try four minutes later which Russell converted.
The second half, Coventry increase their lead with a Tommy Hayes drop goal and a 68th minute Will Hurrell try.
Rotherham scored a try in the 77th minute through Colin Quigley which was converted by Mike Whitehead but it was Coventry who ran out worthy winners.
Picture courtesy of Joanne Davies