The castle was once the refuge of Phelim O’Neill, leader of the Irish Rebellion of 1641 in Ulster. He was captured there in 1653 and taken to Dublin, where he was hanged for treason. Robert Stewart, of Irry inherited the castle from his brothers and died here in 1662.