Post-workout I'd go for full fat, shown to increase net protein balance more than lower fat milk.
Rest of the time then it's up to you. I prefer the taste of full fat so usually get gold top, but occasionally get skimmed if I want to spend calories elsewhere that particular day. Why not have the whole you're currently having and add the extra as (semi-)skimmed? You're allowed to mix and match you know