Rafferty helped the Loyola College soccer team to one South Atlantic Region championship and two ECAC Metro Conference championships. At the time of his induction he was the fifth-leading scorer in Loyola's history with a total of 47 goals and 17 assists. Rafferty was Loyola's top scorer in 1981, 82, and 84. He was selected to the All-South first team in his senior season of 1984. After his senior year, Rafferty was nominated for the East-West Collegiate All-Star Game. He was an All-Maryland Team member during his sophomore and junior years.