My cat also has a similar reaction. We don't exactly know why, but we suspect it's fleas as he will occasionally get scabby around the common flea areas like the base of the tail. He then starts to over-groom and pull off his fur. Our vet gives him an anti-inflammatory injection, which helps immediately, then the same in pill form to keep the itching at bay and promote hair regrowth.
So if it is fleas, it could be that the allergic reaction only lasts a small while, but the resulting dermatitis lasts longer. The other option is a stressful event that causes a bit of miliary dermatitis, that then snowballs.
Would be interested to hear what your vet says/does.