I really cant understand how the nut can come loose without spreading the fins on the retainer! Even if you only did the nut finger tite, the retainer should stop the nut from spinning!
Make sure the fins are pressed against the flat edges of the nut!
I'll double check the retainer as thats a really good point. Bear in mind it's only coming loose about 1/8 of a turn or less. There seems to be a small amount of slack in the retainer when installed? What do Toyota want for new retainers?