Your observations are not unique, I think the explanation could be fairly simple. I have noticed similar phenmenon when I lived in KY. Every night in summer I would watch anywhere from 3-15 bucks hit ag fields of clover or alfalfa. Rarely more than 1 or two does. The number of bucks dwindle as velvet shed. As fawns grew, more does would should up.
You mentioned improving your habitat with food plots and select cut. You probably have the best food in the area, which attracts the bucks in summer. In summer bucks are more tolerant of each other. As summer fades to fall, that tolerance diminishes, and bucks are unwilling to maintain the same level of testosterone in the same general area....they move on.
Conversly, does with young fawns are protective, they keep them secluded and then slowly socialize them. If you have a high density of bucks in the summer, the does are reluctant to mingle. As the fawns grow and they are slowly socialized, eventually this inhibition disappeares. And of course by fall there is no longer the same density of bucks hitting your food sources, which creates more room for the does.
In short, your observing the interplay of food sources attracting deer and the balancing effect of the social structure of the herd.
