The Celebrity Big Brother star, 71, has hit back at the Britain's Got Talent judge, 43, as their spat escalates