I have already taken all the belts off - that's how I pinpointed the noise in the first place (I started searching for what I thought might be a dodgy idler bearing).
The whole balancer is rock solid, as is the nut that holds it on. That's what makes it so bizarre